AWS Glue Interview Questions and Answers (2026)

AWS Glue is a serverless data integration service for extract, transform, and load work. It provisions and manages Spark for you, provides a central metadata store called the Data Catalog, and runs jobs that read from sources like S3, transform data, and write results back — without you managing any servers or clusters.

It stores metadata — table schemas, column types, and partitions — for your data. Crawlers populate it, Glue jobs read it, and other AWS services such as Athena and Redshift Spectrum query against the same catalog, so a single schema definition is shared across the analytics stack.

A crawler connects to a data store, scans a sample of the data, infers the schema and partition structure, and creates or updates table definitions in the Data Catalog. It automates schema discovery so you do not have to define every table by hand.

Spark jobs, which run distributed transformations across managed workers for large datasets, and Python shell jobs, which run a single-node Python script for small or lightweight tasks like calling an API or a small file transform. You pick based on data size and whether you need Spark's distribution.

A DPU, or Data Processing Unit, is Glue's unit of compute capacity combining CPU and memory. Glue jobs are billed per DPU-hour, so the cost of a job depends on how many DPUs (workers) it uses and how long it runs. Fewer DPU-hours means a cheaper job.

A DynamicFrame is Glue-specific and schema-flexible: it can hold multiple candidate types for a field and defer resolving them, which handles inconsistent source data gracefully. A DataFrame needs a fixed schema and gives you the full Spark SQL API. A common pattern is to read as a DynamicFrame, resolve types, convert to a DataFrame for rich transformations, then convert back to write.

Glue persists state about the data a job has already handled — for example which S3 files or records were processed. On each subsequent run, the job reads that bookmark and processes only new data, skipping what it saw before. This lets you build incremental pipelines without manually filtering on a date or watermark column.

resolveChoice handles fields where a DynamicFrame has detected more than one possible type, or ambiguity such as the same column appearing as both int and string. You tell Glue how to resolve it — cast to a single type, keep both as a struct, project one type, or make separate columns — so downstream steps get a clean, predictable schema.

Reduce DPU-hours: right-size the number of workers, avoid over-provisioning, and shorten runtime. Partition output data and use predicate pushdown so jobs read only the partitions they need. Filter early to move less data through the job. Use job bookmarks so you process incremental data instead of full reloads, and prefer Python shell jobs for small tasks that do not need Spark.

Partitioning organizes S3 data into folders by column values such as date, so queries and jobs can skip irrelevant partitions. Predicate pushdown means Glue applies a filter at read time, loading only the matching partitions rather than scanning everything. Together they cut the amount of data read, which lowers both runtime and cost and is a frequent interview point.

They share the Glue Data Catalog. Glue crawls and transforms data into S3 and registers tables in the catalog. Athena runs serverless SQL directly over those catalog tables in S3. Redshift Spectrum lets a Redshift cluster query the same external tables. So one catalog definition, built once, is reused for ETL, ad-hoc SQL, and warehouse queries.

relationalize flattens nested or semi-structured data — such as JSON with arrays and nested objects — into a set of flat, relational tables suitable for loading into a warehouse. It unnests arrays into separate tables with keys linking them back, which is how you get hierarchical source data into a form a relational store like Redshift can use.

Choose Glue for standard batch Spark ETL when you want no cluster management, fast setup, and tight Data Catalog integration, and when workloads are intermittent. Choose EMR when you need cluster control — custom Spark tuning or libraries, long-running or interactive jobs, non-Spark engines like Hive or Presto, or sustained usage where a persistent right-sized cluster costs less than repeated serverless runs. The deciding factors are control, startup latency, tooling, and cost at your volume.

Avoid blindly re-crawling, which can overwrite a corrected schema. Options include using DynamicFrames with resolveChoice to tolerate new or changed types, enabling schema-change handling on the crawler with an update policy that only adds new columns, or managing schema explicitly in the job and validating incoming data. For frequently evolving data, storing in a format and catalog that supports evolution, and version-controlling the schema, is more robust than crawler auto-inference.

Check the Spark UI and Glue metrics for skew, spills, and stage times. Confirm the job reads only needed partitions via pushdown rather than scanning all of S3. Look for many small files, which hurt Spark throughput, and compact them. Verify worker count and type match the data size — both under- and over-provisioning waste money. Ensure job bookmarks are on so it processes incremental data, and check that expensive wide transformations or unnecessary shuffles are minimized.

Use Glue Workflows or triggers to chain jobs and crawlers, where a trigger fires the next job on success or on a schedule. For more complex dependencies, branching, and cross-service steps, orchestrate with Step Functions or Managed Airflow, which give retries, conditional paths, and visibility. The senior point is separating orchestration from transformation logic so each job stays idempotent and independently retriable.