Özer Cem KELAHMETKELAHMET
Topics · Big data

The lakehouse: medallion layers, catalogs and governance

A lakehouse gives warehouse features on open files. This page explains how teams organise one: the medallion layers, the catalog that names every table, and the contracts and lineage that keep data trustworthy.

Overview

A lakehouse keeps data in an open table format on object storage and lets any engine query it with SQL and transactions. The technology is only half of it; the other half is organisation: where raw data lands, how it is cleaned, who owns each table and how changes are tracked.

The medallion layers

Layer Contents Who uses it Typical checks
Bronze Raw data as it arrived, append-only Data engineers Completeness, arrival time
Silver Cleaned, de-duplicated, joined, typed Analysts, ML teams Schema, uniqueness, valid values
Gold Business-ready aggregates and features Dashboards, products, models Business rules, reconciliation

The idea is simple: never change raw data, so any mistake downstream can be rebuilt from bronze. The medallion architecture page covers the variations.

The catalog: one name for every table

A catalog maps table names to their current metadata and controls who can read or write them. Without one, every engine needs to be told where each table lives. Options include REST catalogs for Iceberg, AWS Glue Data Catalog and the platform catalogs of Databricks and Snowflake. Permissions set in the catalog then apply to every engine.

Trust: contracts, lineage and quality

  • Data contracts state the schema, meaning and quality promises of a table, and break the pipeline when an upstream change violates them (data contract validation).
  • Lineage records which job produced each table from which inputs, so the impact of a change or an error can be traced (data lineage).
  • Schema drift detection catches new, renamed or retyped columns before they reach gold (schema drift detection).
  • Transformations as code with tests, using tools such as dbt, make every gold table reproducible.

Who owns what

Central data teams often become a bottleneck as companies grow. Data mesh assigns each domain ownership of its data products, with shared standards for formats, catalogs and quality. Most organisations land between the two: shared platform, domain-owned gold tables.

Frequently asked questions

Is the medallion architecture required?

No, but some layering is: raw data kept unchanged, a cleaned layer, and business-ready tables. The names matter less than the rule that raw data is never overwritten.

How does AI use the lakehouse?

Gold tables feed dashboards, feature stores and, increasingly, tabular foundation models that predict directly from a table; see AI and machine learning for tables.

Summary

Bronze keeps the truth, silver cleans it, gold serves it; the catalog names and protects every table; contracts, lineage and tests keep it trustworthy.

Back to the topic

This page is part of Big data, which maps every concept, tool, simulation and article on the subject.

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open