Discovery — how data is found and traced

Catalog APIs

5 open standards for Catalog APIs in a modern data architecture, each with an opinionated judgement: Adopt, Situational, Assess, or Caution.

Start with Iceberg Catalog and Schema Registry.

Adopt 2 standards

The standard to reach for in new work. Proven, multi-vendor, clearly the default for its slot.

  • Iceberg Catalog — Iceberg REST Catalog

    Canonical catalog API for Iceberg; multi-vendor implementations.

    Apache Software Foundation

  • Schema Registry — Confluent Schema Registry

    The default Kafka schema-management surface.

    Confluent (open API; multiple compatible implementations)

Situational 1 standard

The right answer in some contexts but not others. Pick deliberately based on the constraint.

  • Unity Catalog

    LF-open-sourced but still Databricks-tilted in practice.

    Linux Foundation (originally Databricks)

Assess 1 standard

Promising but not yet proven for production-default use. Track it and prototype, but don't commit your architecture.

  • DuckLake

    DuckDB-Labs catalog; v1.0 production-ready, ecosystem still small.

    DuckDB Labs (vendor-driven open source)

Caution 1 standard

We'd avoid it for new work — superseded or fading, but still encountered in existing systems.

  • Hive Metastore — Apache Hive Metastore

    The catalog Iceberg REST is displacing; maintain only.

    Apache Software Foundation

More in Discovery

Discovery covers how data is found and traced.

See Catalog APIs in context

These standards are one panel of the interactive Data Landscape, which maps every open standard a modern data architecture is built on. The underlying data is a single JSON file; disagree with a judgement? Open an issue.