awswrangler.s3.read_deltalake¶

awswrangler.s3.read_deltalake(path: str, version: int | None = None, partitions: list[tuple[str, str, Any]] | None = None, columns: list[str] | None = None, without_files: bool = False, dtype_backend: Literal['numpy_nullable', 'pyarrow'] = 'numpy_nullable', use_threads: bool = True, boto3_session: Session | None = None, s3_additional_kwargs: dict[str, str] | None = None, pyarrow_additional_kwargs: dict[str, Any] | None = None) → DataFrame¶

Load a Deltalake table data from an S3 path.

This function requires the deltalake package. See the How to load a Delta table guide for loading instructions.

Note

This function has arguments which can be configured globally through wr.config or environment variables:

dtype_backend

Check out the Global Configurations Tutorial for details.

Parameters:

path (str) – The path of the DeltaTable.
version (Optional[int]) – The version of the DeltaTable.
partitions (Optional[List[Tuple[str, str, Any]]) – A list of partition filters, see help(DeltaTable.files_by_partitions) for filter syntax.
columns (Optional[List[str]]) – The columns to project. This can be a list of column names to include (order and duplicates are preserved).
without_files (bool) – If True, load the table without tracking files (memory-friendly). Some append-only applications might not need to track files.
dtype_backend (str, optional) –
Which dtype_backend to use, e.g. whether a DataFrame should have NumPy arrays, nullable dtypes are used for all dtypes that have a nullable implementation when “numpy_nullable” is set, pyarrow is used for all dtypes if “pyarrow” is set.

The dtype_backends are still experimential. The “pyarrow” backend is only supported with Pandas 2.0 or above.
use_threads (bool) – True to enable concurrent requests, False to disable multiple threads. When enabled, os.cpu_count() is used as the max number of threads.
boto3_session (Optional[boto3.Session()]) – Boto3 Session. If None, the default boto3 session is used.
s3_additional_kwargs (Optional[Dict[str, str]]) – Forwarded to the Delta Table class for the storage options of the S3 backend.
pyarrow_additional_kwargs (Optional[Dict[str, str]]) – Forwarded to the PyArrow to_pandas method.

Returns:

df – DataFrame with the results.

Return type:

pd.DataFrame