10 min read
CI/CD Pipelines for Data Engineers
Data pipelines are production software. Here's how to build CI/CD that catches bad transforms before they corrupt dashboards: testing strategy, environment promotion, slim runs, and rollback patterns.
Tag
Data Engineering
Articles tagged Data Engineering on The Augmented Dev.
4 articles
10 min read
Columnar Storage: How Analytical Databases Work Under the Hood
Why column-oriented databases run analytical queries 100x faster than row-oriented ones — covering physical layout, compression algorithms, vectorized execution, and predicate pushdown with concrete examples.
10 min read
dbt Best Practices for Production Data Modeling
Hard-won lessons for structuring dbt projects that scale: layer conventions, testing strategies, slim CI runs, incremental models, and documentation that doesn't rot.
12 min read
Snowflake Security in Layers: Roles, Column Masking, Row Policies, and Environment Controls
A practical walkthrough of Snowflake's four security layers — role hierarchy, column-level masking policies, row access policies, and environment isolation — with real SQL examples.