Ravi Writes

Data Engineer · learning in public

Ravi Writes

Hi, I'm Ravi 👋

Data Engineer by profession, teacher at heart.

Engineering data is what I do; teaching is why I write. I start from fundamentals and build up — the way I actually understand things, written down so it's useful to someone else. And that someone is sometimes me.

From Visual ETL to Code ETL. I spent years building data pipelines visually in Ab Initio — these notes work out what happens when the same ideas (partitioning, re-partitioning, joins, sort etc.) are written as Spark, SQL and Python instead.

Ab Initio graph→Spark code

Start here

🧪Try it LiveThe PlaygroundRunnable examples — open one in Colab and run it in your browser, nothing to install. 📓Ab Initio to PySparkThe core series — Ab Initio Graph Development concepts mapped to PySpark Scripting

Two ways in — Notes: reference & structured series · The Playground: hands-on runnable examples.

Home Notes Playground

Find me

ETL Spark Parquet Shuffle Kafka DAG Airflow SQL ELT Partition RDD Catalyst Repartition Broadcast Schema MapReduce Stream Lakehouse Warehouse Batch