
π Data Engineering & Analytics Β· Mindmap
The main parts of a data governance programme, from ownership and quality to privacy and access, for teams setting up one.
Drawing diagramβ¦
Mind map of data governance: ownership (data owners, stewards, RACI), quality (rules, monitoring, issue process), catalog and lineage, privacy (PII classification, consent, retention), access (role-based access, approvals, audits) and standards (naming, definitions, metric glossary).
mindmap
root((Data Governance))
Ownership
Data owners
Data stewards
RACI per domain
Quality
Rules and thresholds
Monitoring
Issue process
Catalog
Table descriptions
Lineage
Search
Privacy
PII classification
Consent
Retention periods
Access
Role-based access
Approval workflow
Access audits
Standards
Naming conventions
Metric glossaryA typical modern data stack: data is loaded from apps and SaaS tools into a cloud warehouse, modelled with dbt, orchestrated with Airflow and served to BI dashboards.
How every insert, update and delete in an operational database is streamed to the data lake and search index in near real time using change data capture.
A classic star schema for sales analytics: one fact table of order lines surrounded by date, customer, product, store and promotion dimensions.
The steps of a nightly batch pipeline with data quality gates: extract, validate, transform, load and publish, stopping safely when checks fail.
Where a real-time analytics stack runs: Kafka for events, Flink for stream processing, a real-time OLAP database and live dashboards.
The states of a single pipeline run in an orchestrator like Airflow, including retries, upstream failures and manual reruns.