📊 Data Engineering & Analytics · C4 Architecture · Pro
Modern Data Platform Architecture
A typical modern data stack: data is loaded from apps and SaaS tools into a cloud warehouse, modelled with dbt, orchestrated with Airflow and served to BI dashboards.
C4 Architecture diagrams are part of Pro. Anyone can view this one; generating and editing it needs Pro.
Drawing diagram…
What this diagram shows
- Raw data is loaded first and transformed inside the warehouse
- dbt models are tested and documented like code
- One semantic layer keeps metric definitions consistent
Prompt used
Modern data platform: Fivetran loads data from the product database, Salesforce and Google Ads into Snowflake raw schemas. dbt transforms raw tables into staging and mart models, orchestrated by Airflow, with tests on every run. A semantic layer defines metrics once. Analysts use Looker dashboards and data scientists use notebooks. A data catalog documents tables and owners.
Mermaid code
C4Container
title Modern Data Platform
Person(analyst, "Analyst", "Builds dashboards")
Person(ds, "Data Scientist", "Explores data")
System_Boundary(dp, "Data Platform") {
Container(el, "Loader", "Fivetran", "Extract and load")
ContainerDb(wh, "Warehouse", "Snowflake", "Raw, staging and mart schemas")
Container(dbt, "Transformations", "dbt", "Models and tests")
Container(af, "Orchestrator", "Airflow", "Schedules and retries")
Container(sem, "Semantic Layer", "dbt metrics", "One definition per metric")
Container(cat, "Data Catalog", "DataHub", "Tables, owners, lineage")
}
System_Ext(app, "Product Database", "PostgreSQL")
System_Ext(saas, "SaaS Tools", "Salesforce, Google Ads")
System_Ext(bi, "BI Tool", "Looker")
Rel(el, app, "Reads")
Rel(el, saas, "Reads")
Rel(el, wh, "Loads raw data")
Rel(af, el, "Triggers syncs")
Rel(af, dbt, "Runs models")
Rel(dbt, wh, "Builds models")
Rel(sem, wh, "Queries")
Rel(bi, sem, "Metrics")
Rel(analyst, bi, "Uses")
Rel(ds, wh, "Queries")
Rel(cat, wh, "Scans metadata")Related templates
Change Data Capture Pipeline
How every insert, update and delete in an operational database is streamed to the data lake and search index in near real time using change data capture.
Sales Star Schema
A classic star schema for sales analytics: one fact table of order lines surrounded by date, customer, product, store and promotion dimensions.
Nightly Batch ETL Workflow
The steps of a nightly batch pipeline with data quality gates: extract, validate, transform, load and publish, stopping safely when checks fail.
Real-time Streaming Analytics Deployment
Where a real-time analytics stack runs: Kafka for events, Flink for stream processing, a real-time OLAP database and live dashboards.
Data Pipeline Run States
The states of a single pipeline run in an orchestrator like Airflow, including retries, upstream failures and manual reruns.
Data Lake Zones (Bronze, Silver, Gold)
The medallion pattern for a data lake: raw data lands in bronze, is cleaned in silver, and turned into business-ready tables in gold.