Free notes · no sign-up to browse
Spark, SQL, Kafka, Snowflake, Airflow, dbt, warehousing and system design, written as revision notes, not textbooks.
Also included in any paid plan and every cohort seat.
New · the complete notes · sold separately from the vault
Twenty-eight chapters, Hadoop to Terraform, in one 157-page PDF. Sold on its own: the vault and the Learn Pass do not include it.
Data Engineering End to End Notes
What the 28 chapters give you:
₹499 · PDF · 20 pages free to read
New · data modelling from zero to hero · sold separately from the vault
Twenty chapters in seven parts, ER diagrams to replication, then six case studies, in one 93-page PDF. Sold on its own: the vault and the Learn Pass do not include it.
Data Modelling: Zero to Hero
What the 20 chapters give you:
₹299 · PDF · 20 pages free to read
Just launched · sold separately from the vault
The 11 decks:
₹499 · PDF · 20 pages free to read
New · for data analyst interviews · sold separately from the vault
The 11 decks:
₹499 · PDF · 20 pages free to read
New book · sold separately from the vault
₹499. PDF, 111 pages.
Just added
Just added — newest first.
100 questions from easy to expert for product-based company interviews, each solved on one store dataset with the PostgreSQL 16 query and its real output.
217 pages3.0 MB
Spark's engine, execution model and optimiser, then PySpark hands-on from RDDs to streaming. Chapters 6 and 7 of the Data Engineering End to End Notes.
14 pages537 KB
Topics, partitions, brokers, replication, producers, consumers, Schema Registry and real setups. Chapter 10 of the Data Engineering End to End Notes.
7 pages456 KB
DAGs, operators, XCom, TaskFlow, scheduling, catchup and backfill, connections and alerts. Chapter 11 of the Data Engineering End to End Notes.
7 pages464 KB
The library
100 questions from easy to expert for product-based company interviews, each solved on one store dataset with the PostgreSQL 16 query and its real output.
217 pages3.0 MB
Spark's engine, execution model and optimiser, then PySpark hands-on from RDDs to streaming. Chapters 6 and 7 of the Data Engineering End to End Notes.
14 pages537 KB
Topics, partitions, brokers, replication, producers, consumers, Schema Registry and real setups. Chapter 10 of the Data Engineering End to End Notes.
7 pages456 KB
DAGs, operators, XCom, TaskFlow, scheduling, catchup and backfill, connections and alerts. Chapter 11 of the Data Engineering End to End Notes.
7 pages464 KB
Identity, compute, storage, networking, databases, DevOps and monitoring on Azure. Chapter 16 of the Data Engineering End to End Notes.
8 pages459 KB
IAM, compute, storage, databases, networking, Vertex AI, operations and cost control. Chapter 17 of the Data Engineering End to End Notes.
7 pages449 KB
OLTP vs OLAP, warehouse architecture, dimensional modelling, lakes and lakehouses, and performance. Chapter 20 of the Data Engineering End to End Notes.
5 pages426 KB
Why one machine stops being enough, Hadoop and HDFS inside out, MapReduce and YARN. Chapters 1 to 3 of the Data Engineering End to End Notes.
16 pages293 KB
SQL on big data: architecture, setup, tables, partitions, buckets, functions and joins. Chapter 4 of the Data Engineering End to End Notes.
7 pages203 KB
NoSQL on Hadoop: random, real-time reads and writes on top of HDFS. Chapter 5 of the Data Engineering End to End Notes.
6 pages166 KB
Pipeline anatomy, ETL vs ELT, batch vs real-time, windows and the Lambda architecture. Chapter 8 of the Data Engineering End to End Notes.
6 pages150 KB
Images, containers, Dockerfiles, debugging and multi-container stacks with Compose. Chapter 9 of the Data Engineering End to End Notes.
6 pages164 KB
IAM and EC2, S3 and Route 53, Lambda and VPC, Glue and Athena with the Data Catalog. Chapters 12 to 15 of the Data Engineering End to End Notes.
19 pages307 KB
CAP, consistency, replication vs sharding; Cassandra's masterless design, modelling and CQL. Chapters 18 and 19 of the Data Engineering End to End Notes.
12 pages222 KB
MPP columnar warehouse on AWS: architecture, loading, table design, tuning, Spectrum and querying. Chapter 21 of the Data Engineering End to End Notes.
7 pages169 KB
Serverless warehouse: Dremel, datasets and tables, loading, partitioning, clustering and SQL. Chapter 22 of the Data Engineering End to End Notes.
6 pages163 KB
Architecture, GKE, core objects, storage, autoscaling and stateful data workloads. Chapter 23 of the Data Engineering End to End Notes.
7 pages186 KB
Architecture, objects and RBAC, Time Travel, warehouses, stages, Snowpipe, Streams and Tasks. Chapter 24 of the Data Engineering End to End Notes.
8 pages195 KB
The DevSecOps platform, runners and .gitlab-ci.yml from the first job to scheduled pipelines. Chapter 25 of the Data Engineering End to End Notes.
8 pages190 KB
Infrastructure as Code: core workflow, state, modules, variables, expressions and real AWS builds. Chapter 26 of the Data Engineering End to End Notes.
8 pages193 KB
All 19 topics for making Databricks and Spark queries fast: warehouses and Photon, Query Profile and EXPLAIN, joins, AQE, skew, pruning, stats and caching.
25 pages432 KB
The log, ACID and OCC, time travel, VACUUM, OPTIMIZE, ZORDER, liquid clustering, MERGE, CDF and deletion vectors. Every block ran on a real Delta table.
27 pages518 KB
The 6-step framework interviewers grade, metric trees with worked arithmetic, Simpson's paradox with real numbers, 13 cases from food delivery to UPI.
19 pages1.7 MB
Raw CSVs to a published, secured report: a real brief with 8 questions, every M and DAX formula written out, an April to March fiscal year, RLS, refresh.
19 pages1.7 MB
How Snowflake, Databricks, BigQuery and AWS actually bill, the queries that find the expensive 10%, hidden network costs, a worked case and FinOps habits.
19 pages1.7 MB
The pipelines behind RAG, agents and LLM apps: parsing, six chunking strategies, versioned embeddings, pgvector hybrid search, fresh indexes, evals, ACLs.
19 pages1.8 MB
25 data pipeline incidents on Spark, Airflow, Kafka, CDC, warehouses and S3: symptom, first 10 minutes, root cause, fix, prevention, reruns, postmortems.
19 pages1.8 MB
The same 18 business questions answered in SQL, pandas and Excel side by side on one dataset. SQL ran on DuckDB, pandas on 3.0, and the results match.
18 pages357 KB
DAU/MAU, ordered funnels, sessions, the retention triangle, four retention definitions, LTV, growth accounting and RFM, as SQL that runs on real data.
22 pages396 KB
Package a PySpark job, run a local stack with Compose, then schedule it with CronJobs, the Spark Operator and Airflow 3 on Kubernetes, laptop to cluster.
25 pages439 KB
How a database change becomes a correct row in your lake: write-ahead logs, Debezium, Kafka, ordering, deletes, MERGE, SCD 2 and the 2 AM failures, in SQL.
25 pages455 KB
Why a folder of Parquet is not a table, and how Iceberg, Delta and Hudi fix it. Snapshots, logs and deletes run for real on PyIceberg and delta-rs.
26 pages510 KB
From installing Python to DSA, Big-O and the coding patterns DE interviews keep reusing, with a 10-week plan to get there. Every topic comes with the approach, code that runs, and the real output.
80 pages477 KB
From your first SELECT to the patterns interviewers keep reusing. Every topic comes with the approach to solve it, a query that runs, and the real output.
40 pages329 KB
Lecture Zero plus fourteen chapters — Linux, SQL, Python, data modeling, storage, cloud, pipelines, Spark, Airflow, Kafka, data quality, a portfolio project and the interview loop. Every figure measured. Free, and it never uses your free note.
244 pages1.4 MB
Twenty pages on how your code actually reaches production: Git and merge requests, .gitlab-ci.yml line by line, rules and runners, dbt Slim CI without dbt Cloud, shipping Airflow DAGs and Databricks asset bundles, plus 25 rapid-fire interview questions.
20 pages1.6 MB
The full written notes from the 29 August live build: loading raw IPL match data into Databricks, DataFrame analysis with groupBy, window functions, joins and UDFs, tuning with caching and partitioning, and writing the processed outputs to Delta.
20 pages1.8 MB
Two free ways to see where you stand
After the notes
Read them in order if you like — the free library first, then the deep-dive modules, then the live cohort when you want someone reviewing your work.
Free · no payment
SQL, Python, Power BI, PySpark and Excel — full video lessons with MCQs and a proctored skill test at the end. The notes are the summary; this is the teaching.
Open the free libraryLearn Pass · ₹499 for 6 months
Every note in this vault for 6 months, plus the complete PySpark and Databricks modules recorded end to end with their project notebooks. One payment, nothing renews.
See what's insideAI-Powered Data Engineering Cohort 4 · live cohort
Live weekend classes (Sat–Sun · 7 AM–9:30 AM IST), real projects, graded assignments, and 6 months of Pro Auto-Pilot included. The full path, taught with you in the room.
See the curriculumYour first note and the free book cost nothing. The rest of the vault comes with the Learn Pass, any paid plan or a cohort seat.