RaySettings

class awswrangler.typing.RaySettings

Bases: TypedDict

Typed dictionary defining the settings for distributing calls using Ray.

Attributes

parallelism	The requested parallelism of the read.
override_num_blocks	Override the number of output blocks from all read tasks.

Attributes Documentation

parallelism: NotRequired[int]

The requested parallelism of the read. Parallelism may be limited by the number of files of the dataset. Auto-detect by default.

override_num_blocks: NotRequired[int]

Override the number of output blocks from all read tasks. By default, the number of output blocks is dynamically decided based on input data size and available resources. You shouldn’t manually set this value in most cases.