awswrangler.s3.list_objects

awswrangler.s3.list_objects(path: str, suffix: str | list[str] | None = None, ignore_suffix: str | list[str] | None = None, last_modified_begin: datetime | None = None, last_modified_end: datetime | None = None, ignore_empty: bool = False, chunked: bool = False, s3_additional_kwargs: dict[str, Any] | None = None, boto3_session: Session | None = None) → list[str] | Iterator[list[str]]

List Amazon S3 objects from a prefix.

This function accepts Unix shell-style wildcards in the path argument. * (matches everything), ? (matches any single character), [seq] (matches any character in seq), [!seq] (matches any character not in seq). If you want to use a path which includes Unix shell-style wildcard characters (*, ?, []), you can use glob.escape(path) before passing the path to this function.

Note

The filter by last_modified begin last_modified end is applied after list all S3 files

Note

Following arguments are not supported in distributed mode with engine EngineEnum.RAY:

  • boto3_session

  • s3_additional_kwargs

Parameters:

  • path (str) – S3 path (e.g. s3://bucket/prefix).

  • suffix (Union[str, List[str], None]) – Suffix or List of suffixes for filtering S3 keys.

  • ignore_suffix (Union[str, List[str], None]) – Suffix or List of suffixes for S3 keys to be ignored.

  • last_modified_begin – Filter the s3 files by the Last modified date of the object. The filter is applied only after list all s3 files.

  • last_modified_end (datetime, optional) – Filter the s3 files by the Last modified date of the object. The filter is applied only after list all s3 files.

  • ignore_empty (bool) – Ignore files with 0 bytes.

  • chunked (bool) – If True returns iterator, and a single list otherwise. False by default.

  • s3_additional_kwargs (dict[str, Any], optional) – Forwarded to botocore requests. e.g. s3_additional_kwargs={‘RequestPayer’: ‘requester’}

  • boto3_session (boto3.Session(), optional) – Boto3 Session. The default boto3 session will be used if boto3_session receive None.

Returns:

List of objects paths.

Return type:

Union[List[str], Iterator[List[str]]]

Examples

Using the default boto3 session

>>> import awswrangler as wr
>>> wr.s3.list_objects('s3://bucket/prefix')
['s3://bucket/prefix0', 's3://bucket/prefix1', 's3://bucket/prefix2']

Using a custom boto3 session

>>> import boto3
>>> import awswrangler as wr
>>> wr.s3.list_objects('s3://bucket/prefix', boto3_session=boto3.Session())
['s3://bucket/prefix0', 's3://bucket/prefix1', 's3://bucket/prefix2']