All writing

Writing topic

Data Engineering

Shared definitions, lineage and quality gates in the pipelines that feed training, inference and reporting.

Data Engineering writing

5 min read

A DAG Is a Failure Contract, Not a Schedule

Orchestration tutorials sell the schedule. The parts that decide whether a pipeline survives are retry semantics, write shape, backfill parameterisation and what a half-finished run leaves behind.

  • Data Engineering
  • Airflow
  • Idempotency
  • Reliability
Read
5 min read

One Definition Per Feature

When every pipeline rebuilds the same customer feature, the definitions drift apart silently. Making the feature a shared owned artifact fixes that and creates a change-impact problem that column lineage has to answer.

  • Data Engineering
  • dbt
  • Feature Platform
  • Lineage
Read