CloudSketch AI Logo
FeaturesTemplatesPricingEnterpriseAboutContactLog in
πŸŒ™β˜€οΈ
Log inStart free
Home/Templates/Data Engineering & Analytics

πŸ“Š Data Engineering & Analytics Β· Data Flow

Data Lake Zones (Bronze, Silver, Gold)

The medallion pattern for a data lake: raw data lands in bronze, is cleaned in silver, and turned into business-ready tables in gold.

More Data Engineering & Analytics templates

Opening a template is free and never counts toward your daily diagrams. Asking the AI to change it counts as one.

Drawing diagram…

What this diagram shows

  • Bronze keeps raw data exactly as received
  • Silver removes duplicates and fixes types
  • Gold holds aggregated tables for reports and ML

Prompt used

Medallion architecture data flow: batch files, API data and streaming events land unchanged in the bronze zone of a Delta Lake. A cleaning job de-duplicates, standardises types and masks personal data into silver. Aggregation jobs build gold tables such as daily revenue and customer 360. BI tools and ML features read from gold. Number the processes and label every flow.

Mermaid code
flowchart LR
subgraph EXT ["External entities"]
batch_sys["Batch Source Systems"]
api_sys["API Data Sources"]
stream_sys["Streaming Event Sources"]
bi_tools["BI Tools"]
ml_sys["ML Feature Store / ML Systems"]
end
subgraph PROC ["Processes"]
p1("1.0 Ingest Raw Data")
p2("2.0 Clean and Standardize Data")
p3("3.0 Aggregate Business Metrics")
p4("4.0 Serve Analytics and Features")
end
subgraph STORE ["Data stores"]
d1[("D1 Bronze Zone - Raw Delta Tables")]
d2[("D2 Silver Zone - Cleaned Delta Tables")]
d3[("D3 Gold Zone - Aggregated Delta Tables")]
end
batch_sys -->|"Raw Batch Files"| p1
api_sys -->|"Raw API Data Payload"| p1
stream_sys -->|"Raw Streaming Events"| p1
p1 -->|"Unchanged Raw Data Records"| d1
d1 -->|"Unprocessed Bronze Records"| p2
p2 -->|"Deduplicated, Standardized & Masked Data"| d2
d2 -->|"Cleaned Silver Records"| p3
p3 -->|"Daily Revenue & Customer 360 Aggregations"| d3
d3 -->|"Gold Aggregated Tables Data"| p4
p4 -->|"Analytics Reports & Dashboard Data"| bi_tools
p4 -->|"Machine Learning Features"| ml_sys

Inside this diagram

A data flow diagram with 9 processes and outside parties, 3 data stores and 11 data flows.

Processes and outside parties

  • Batch Source Systems
  • API Data Sources
  • Streaming Event Sources
  • BI Tools
  • ML Feature Store / ML Systems
  • 1.0 Ingest Raw Data
  • 2.0 Clean and Standardize Data
  • 3.0 Aggregate Business Metrics
  • 4.0 Serve Analytics and Features

Data stores

  • D1 Bronze Zone - Raw Delta Tables
  • D2 Silver Zone - Cleaned Delta Tables
  • D3 Gold Zone - Aggregated Delta Tables

Data flows

  • Batch Source Systems β†’ 1.0 Ingest Raw Data: Raw Batch Files
  • API Data Sources β†’ 1.0 Ingest Raw Data: Raw API Data Payload
  • Streaming Event Sources β†’ 1.0 Ingest Raw Data: Raw Streaming Events
  • 1.0 Ingest Raw Data β†’ D1 Bronze Zone - Raw Delta Tables: Unchanged Raw Data Records
  • D1 Bronze Zone - Raw Delta Tables β†’ 2.0 Clean and Standardize Data: Unprocessed Bronze Records
  • 2.0 Clean and Standardize Data β†’ D2 Silver Zone - Cleaned Delta Tables: Deduplicated, Standardized & Masked Data
  • D2 Silver Zone - Cleaned Delta Tables β†’ 3.0 Aggregate Business Metrics: Cleaned Silver Records
  • 3.0 Aggregate Business Metrics β†’ D3 Gold Zone - Aggregated Delta Tables: Daily Revenue & Customer 360 Aggregations
  • D3 Gold Zone - Aggregated Delta Tables β†’ 4.0 Serve Analytics and Features: Gold Aggregated Tables Data
  • 4.0 Serve Analytics and Features β†’ BI Tools: Analytics Reports & Dashboard Data
  • 4.0 Serve Analytics and Features β†’ ML Feature Store / ML Systems: Machine Learning Features

Boundaries

  • External entities: Batch Source Systems, API Data Sources, Streaming Event Sources, BI Tools and ML Feature Store / ML Systems
  • Processes: 1.0 Ingest Raw Data, 2.0 Clean and Standardize Data, 3.0 Aggregate Business Metrics and 4.0 Serve Analytics and Features
  • Data stores: D1 Bronze Zone - Raw Delta Tables, D2 Silver Zone - Cleaned Delta Tables and D3 Gold Zone - Aggregated Delta Tables

Make your own diagram like this

Describe your own system in plain English and CloudSketch AI draws it in seconds. See all AI diagram generators.

Related templates

C4 ArchitectureProData Engineering & Analytics

Modern Data Platform Architecture

A typical modern data stack: data is loaded from apps and SaaS tools into a cloud warehouse, modelled with dbt, orchestrated with Airflow and served to BI dashboards.

Data FlowData Engineering & Analytics

Change Data Capture Pipeline

How every insert, update and delete in an operational database is streamed to the data lake and search index in near real time using change data capture.

Database ERDData Engineering & Analytics

Sales Star Schema

A classic star schema for sales analytics: one fact table of order lines surrounded by date, customer, product, store and promotion dimensions.

FlowchartData Engineering & Analytics

Nightly Batch ETL Workflow

The steps of a nightly batch pipeline with data quality gates: extract, validate, transform, load and publish, stopping safely when checks fail.

DeploymentProData Engineering & Analytics

Real-time Streaming Analytics Deployment

Where a real-time analytics stack runs: Kafka for events, Flink for stream processing, a real-time OLAP database and live dashboards.

State MachineData Engineering & Analytics

Data Pipeline Run States

The states of a single pipeline run in an orchestrator like Airflow, including retries, upstream failures and manual reruns.