Skip to main content
All projects
Production
data engineering

Modern Data Stack: DuckDB Semantic Layer

A local-first Modern Data Stack reference using dlt for ingestion, dbt for transformation, and DuckDB as the engine — with a MetricFlow semantic layer and Rill dashboards.

Tech Stack

dbtDuckDBdltMetricFlowRillPython
View on GitHub

A fully local Modern Data Stack reference implementation — no cloud account required. Everything runs against a single DuckDB file.

The pipeline follows a three-layer architecture:

Layer Tool What it does
Ingestion dlt Loads raw data into DuckDB bronze tables
Transformation dbt-core + dbt-duckdb Stages, transforms, and materialises mart tables
Presentation Rill Connects directly to the semantic layer for dashboards

A MetricFlow semantic layer sits on top of the marts, providing standardised metric definitions across revenue, customer segmentation, and store performance.

Quick Start

Requirements: Python 3.11+, uv

uv venv && source .venv/bin/activate
make install && make run
make rill-dev   # dashboards at localhost:9009