numbarrow.core.adapters

Overview

Type-dispatched adapters that convert PyArrow arrays into NumPy arrays for use in Numba @njit compiled functions.

Uses functools.singledispatch to route each PyArrow array type to a handler that extracts the underlying data buffer as a NumPy view (where possible) and the validity bitmap as a uint8 array.

Supported types:

  • BooleanArray (requires copy due to bit-packed layout)

  • Int32Array, Int64Array, DoubleArray (zero-copy view)

  • Date32Array (copy: int32 days → datetime64[D])

  • Date64Array (zero-copy view as datetime64[ms])

  • TimestampArray (zero-copy view as datetime64[unit])

  • StringArray (copy into fixed-width NumPy Unicode array; bitmap is not returned)

  • StructArray (returns tuple of two dicts: bitmaps and data, keyed by field name)

  • ListArray (delegates to StructArray adapter for list-of-struct)

Module

Type-dispatched adapters that convert PyArrow arrays into NumPy arrays for use in Numba @njit compiled functions.

Uses functools.singledispatch() to route each PyArrow array type (BooleanArray, Int32Array, Date32Array, etc.) to a handler that extracts the underlying data buffer as a NumPy array and the validity bitmap as a uint8 array. Where possible, data is viewed without copying; types that require layout changes (e.g. Date32 → datetime64[D]) produce a copy.

numbarrow.core.adapters.arrow_array_adapter(pa_array: Array)[source]
numbarrow.core.adapters.arrow_array_adapter(pa_array: BooleanArray)
numbarrow.core.adapters.arrow_array_adapter(pa_array: Date32Array)
numbarrow.core.adapters.arrow_array_adapter(pa_array: Date64Array)
numbarrow.core.adapters.arrow_array_adapter(pa_array: DoubleArray | Int32Array | Int64Array)
numbarrow.core.adapters.arrow_array_adapter(pa_array: DoubleArray | Int32Array | Int64Array)
numbarrow.core.adapters.arrow_array_adapter(pa_array: DoubleArray | Int32Array | Int64Array)
numbarrow.core.adapters.arrow_array_adapter(pa_array: ListArray)
numbarrow.core.adapters.arrow_array_adapter(pa_array: StructArray)
numbarrow.core.adapters.arrow_array_adapter(pa_array: StringArray)
numbarrow.core.adapters.arrow_array_adapter(pa_array: TimestampArray)

Dispatcher for PyArrow array adapters of various types.

numbarrow.core.adapters.cast_64bit_date_arrow_to_numpy_array(pa_array: Array, np_dtype: dtype)[source]

Can be used to cast PyArrow arrays of date types that are represented by 64-bit integers to numpy arrays of various date types (np.datetime64[…], which are always represented by 64-bit integers whose meaning is determined by the precision, such as, ‘s’, ‘ms’, ‘us’).

Since underlying data layout of both arrays in int64, a copy is avoided,

The associated bitmap (if any) is also returned.