- awswrangler.s3.wait_objects_exist(paths: List[str], delay: float | None = None, max_attempts: int | None = None, use_threads: bool | int = True, boto3_session: Session | None = None) None ¶
Wait Amazon S3 objects exist.
Polls S3.Client.head_object() every 5 seconds (default) until a successful state is reached. An error is returned after 20 (default) failed checks. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Waiter.ObjectExists
In case of use_threads=True the number of threads that will be spawned will be gotten from os.cpu_count().
Following arguments are not supported in distributed mode with engine EngineEnum.RAY:
paths (List[str]) – List of S3 objects paths (e.g. [s3://bucket/key0, s3://bucket/key1]).
delay (Union[int,float], optional) – The amount of time in seconds to wait between attempts. Default: 5
max_attempts (int, optional) – The maximum number of attempts to be made. Default: 20
use_threads (bool, int) – True to enable concurrent requests, False to disable multiple threads. If enabled os.cpu_count() will be used as the max number of threads. If integer is provided, specified number is used.
boto3_session (boto3.Session(), optional) – Boto3 Session. The default boto3 session will be used if boto3_session receive None.
- Return type:
>>> import awswrangler as wr >>> wr.s3.wait_objects_exist(['s3://bucket/key0', 's3://bucket/key1']) # wait both objects