awswrangler.emr.submit_step

awswrangler.emr.submit_step(cluster_id: str, command: str, name: str = 'my-step', action_on_failure: Literal['TERMINATE_JOB_FLOW', 'TERMINATE_CLUSTER', 'CANCEL_AND_WAIT', 'CONTINUE'] = 'CONTINUE', script: bool = False, boto3_session: Session | None = None) → str

Submit new job in the EMR Cluster.

Parameters:

  • cluster_id (str) – Cluster ID.

  • command (str) – e.g. ‘echo “Hello!”’ e.g. for script ‘s3://…/script.sh arg1 arg2’

  • name (str) – Step name.

  • action_on_failure (Literal['TERMINATE_JOB_FLOW', 'TERMINATE_CLUSTER', 'CANCEL_AND_WAIT', 'CONTINUE']) – ‘TERMINATE_JOB_FLOW’, ‘TERMINATE_CLUSTER’, ‘CANCEL_AND_WAIT’, ‘CONTINUE’

  • script (bool) – True for raw command or False for script runner. https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-commandrunner.html

  • boto3_session (Session | None) – The default boto3 session will be used if boto3_session is None.

Return type:

str

Returns:

Step ID.

Examples

>>> import awswrangler as wr
>>> step_id = wr.emr.submit_step(
...     cluster_id=cluster_id,
...     name="step_test",
...     command="s3://...script.sh arg1 arg2",
...     script=True,
... )