awswrangler.emr.build_step

awswrangler.emr.build_step(command: str, name: str = 'my-step', action_on_failure: Literal['TERMINATE_JOB_FLOW', 'TERMINATE_CLUSTER', 'CANCEL_AND_WAIT', 'CONTINUE'] = 'CONTINUE', script: bool = False, region: str | None = None, boto3_session: Session | None = None) → dict[str, Any]

Build the Step structure (dictionary).

Parameters:

  • command (str) – e.g. ‘echo “Hello!”’ e.g. for script ‘s3://…/script.sh arg1 arg2’

  • name (str) – Step name.

  • action_on_failure (Literal['TERMINATE_JOB_FLOW', 'TERMINATE_CLUSTER', 'CANCEL_AND_WAIT', 'CONTINUE']) – ‘TERMINATE_JOB_FLOW’, ‘TERMINATE_CLUSTER’, ‘CANCEL_AND_WAIT’, ‘CONTINUE’

  • script (bool) – False for raw command or True for script runner. https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-commandrunner.html

  • region (str | None) – Region name to not get it from boto3.Session. (e.g. us-east-1)

  • boto3_session (Session | None) – The default boto3 session will be used if boto3_session is None.

Return type:

dict[str, Any]

Returns:

Step structure.

Examples

>>> import awswrangler as wr
>>> steps = []
>>> for cmd in ['echo "Hello"', "ls -la"]:
...     steps.append(wr.emr.build_step(name=cmd, command=cmd))
>>> wr.emr.submit_steps(cluster_id="cluster-id", steps=steps)