Data Engineering · Hands-On Build

Tesla Telemetry Pipeline S3 → Airflow → Snowflake

Raw JSONL events in an S3 bucket become five Snowflake report tables — and you leave with the whole thing running on your own laptop.

Electric cars emit small events all day — speed, battery level, GPS position, alerts. This session takes those raw JSONL events out of an S3 bucket and turns them into five report tables in Snowflake, with Airflow orchestrating the whole thing inside Docker. We pull only the new files, tracked by key and ETag, so nothing loads twice. Every event is validated with pydantic before pandas builds the hourly, daily, trip and alert rollups as Parquet. The load into Snowflake runs through a staging layer with PUT, COPY INTO and MERGE, and we cover why the pipeline authenticates with a key pair instead of a password, and why the AWS user can read exactly one folder. Setup is one command and cleanup is another. Almost every line of code carries a plain-English comment, so if you know a little Python and SQL you can follow along live and explain the project in an interview afterwards. The code is public at github.com/analyticsdurgesh/tesla-vehicle-telemetry-etl. The vehicle data is synthetic and this project is not affiliated with Tesla.