Acero - Streaming Execution Engine#

Warning

Acero is experimental and a stable API is not yet guaranteed.

Acero is a streaming query engine, which allows the computation to be expressed as an “execution plan” (constructed using the Declaration interface). This enables to create a computation composed of pyarrow.compute functions and to execute this efficiently in a batched manner.

`Declaration`(factory_name, ...[, inputs])	Helper class for declaring the nodes of an ExecPlan.
`ExecNodeOptions`	Base class for the node options.
`TableSourceNodeOptions`(table)	A Source node which accepts a table.
`ScanNodeOptions`(dataset, **kwargs)	A Source node which yields batches from a Dataset scan.
`FilterNodeOptions`(filter_expression)	Make a node which excludes some rows from batches passed through it.
`ProjectNodeOptions`(expressions[, names])	Make a node which executes expressions on input batches, producing batches of the same length with new columns.
`AggregateNodeOptions`(aggregates[, keys])	Make a node which aggregates input batches, optionally grouped by keys.
`OrderByNodeOptions`([sort_keys, null_placement])	Make a node which applies a new ordering to the data.
`HashJoinNodeOptions`(join_type, left_keys, ...)	Make a node which implements join operation using hash join strategy.