Apache Arrow
Judgement: Adopt
In-memory columnar standard powering Flight, ADBC, DataFusion, Polars.
Language-agnostic columnar in-memory format for analytical data. Designed for zero-copy reads and efficient interchange between processes, languages, and engines.
Foundation for ADBC, the basis of pandas 2.x, the wire format of Arrow Flight, and the in-memory layout of DuckDB, Polars, and many others. Arrow is the connective tissue of modern analytics.
Why it counts as a standard
Arrow's columnar memory layout and IPC format are an open specification, not a library. DuckDB, Polars, pandas 2.x, Spark, and ADBC all align on the same in-memory representation, which is what enables zero-copy interchange across engines and languages. The format spec is the standard.
At a glance
- Category
- In-Memory Format
- Governance
- Apache Software Foundation
- Status
- Stable; broadly adopted across the analytics stack
- First released
- 2016
Links
Related standards
Other standards in In-Memory Format.
- DataFrame API — Python Dataframe API Standard
See Apache Arrow in context
Open the interactive Data Landscape to compare Apache Arrow against every other open standard, or grab the raw JSON. Spotted something wrong? Open an issue.