awswrangler.emr.build_step¶
- awswrangler.emr.build_step(command: str, name: str = 'my-step', action_on_failure: str = 'CONTINUE', script: bool = False, region: str | None = None, boto3_session: Session | None = None) Dict[str, Any] ¶
Build the Step structure (dictionary).
- Parameters:
command (str) – e.g. ‘echo “Hello!”’ e.g. for script ‘s3://…/script.sh arg1 arg2’
name (str, optional) – Step name.
action_on_failure (str) – ‘TERMINATE_JOB_FLOW’, ‘TERMINATE_CLUSTER’, ‘CANCEL_AND_WAIT’, ‘CONTINUE’
script (bool) – False for raw command or True for script runner. https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-commandrunner.html
region (str, optional) – Region name to not get it from boto3.Session. (e.g. us-east-1)
boto3_session (boto3.Session(), optional) – Boto3 Session. The default boto3 session will be used if boto3_session receive None.
- Returns:
Step structure.
- Return type:
Dict[str, Any]
Examples
>>> import awswrangler as wr >>> steps = [] >>> for cmd in ['echo "Hello"', "ls -la"]: ... steps.append(wr.emr.build_step(name=cmd, command=cmd)) >>> wr.emr.submit_steps(cluster_id="cluster-id", steps=steps)