Storage — where data lives

Storage Systems

2 open standards for Storage Systems in a modern data architecture, each with an opinionated judgement: Adopt, Situational, Assess, or Caution.

Start with S3.

Adopt 1 standard

The standard to reach for in new work. Proven, multi-vendor, clearly the default for its slot.

S3 — Amazon S3 API
The object-storage API everyone implements.

AWS (vendor-controlled, but de-facto standard)

Caution 1 standard

We'd avoid it for new work — superseded or fading, but still encountered in existing systems.

HDFS — Hadoop Distributed File System
Hadoop-era distributed FS; object storage replaced it for nearly all new builds.

Apache Software Foundation

More in Storage

Storage covers where data lives.

File Formats
Open Table Formats

See Storage Systems in context

These standards are one panel of the interactive Data Landscape, which maps every open standard a modern data architecture is built on. The underlying data is a single JSON file; disagree with a judgement? Open an issue.