AWS Glue Interview Questions and Answers (2026)

AWS Glue is AWS's serverless data integration service: you submit ETL jobs and Glue provisions, scales, and tears down the underlying Apache Spark environment, billed per DPU-hour with no clusters to manage. Interviewers expect you to name its three core pieces: the Data Catalog (a central, Hive-compatible metastore shared with Athena, EMR, and Redshift Spectrum), crawlers (which infer schemas and register tables), and jobs (distributed Spark, Spark streaming, or single-node Python shell). The classic trap is describing Glue as storage or a database — it holds only metadata; the data itself stays in S3, JDBC sources, and so on. Pre-empt the follow-up: Glue suits intermittent batch ETL; long-running, heavily tuned, or non-Spark workloads often fit EMR better.

The Data Catalog is a central, Hive-metastore-compatible store of metadata: databases, tables, column types, partition keys and locations, plus connection definitions. It never holds data — only schemas and pointers describing data that lives in S3 or JDBC stores. Its value is that one definition is shared everywhere: crawlers and Glue jobs write to it, while Athena, Redshift Spectrum, and EMR read from it, so a table defined once serves every engine. Partitions are first-class catalog entries, which is exactly what lets query engines prune S3 prefixes instead of scanning everything. Two follow-ups worth pre-empting: it can replace a self-managed Hive metastore on EMR, and Lake Formation layers fine-grained, column-level permissions on top of catalog resources.

A crawler connects to a data store (an S3 path, JDBC database, or DynamoDB table), samples the data, uses classifiers to detect the format — JSON, CSV, Parquet, or a custom grok/regex classifier — infers the schema and the partition layout from the folder structure, and creates or updates tables in the Data Catalog. Scheduled crawls pick up new partitions automatically. The nuance interviewers probe: inference is sample-based, so mixed-type columns or inconsistent files can produce wrong schemas, and a re-crawl can overwrite manual corrections unless you set the update behavior to log-only or add-new-columns-only. Also know the cheaper alternative: for stable schemas, registering new partitions via ALTER TABLE ADD PARTITION in Athena or a Glue API call beats re-crawling the whole store.

Spark ETL jobs and Python shell jobs. Spark jobs run distributed PySpark or Scala across a fleet of workers and are the default for large-scale transforms; a streaming variant of the Spark job reads micro-batches from Kinesis or Kafka. Python shell jobs run a single plain-Python process — no Spark at all — sized at a fraction of a DPU, which makes them right for small files, API calls, orchestration glue, or loading a lookup table. The classic follow-up is 'when would you NOT use Spark?': if the data fits comfortably on one node, a Python shell job starts faster and costs far less, because a Spark job consumes a minimum number of DPUs regardless of how little work it does. Spinning up distributed Spark for a 50 MB CSV is the trap answer.

A DPU (Data Processing Unit) is Glue's unit of compute and billing — a bundle of 4 vCPUs and 16 GB of memory for a standard DPU. Jobs are billed per DPU-hour, metered by the second, so cost is workers times runtime. Worker types map onto DPUs: a G.1X worker is 1 DPU, G.2X is 2, with larger sizes for memory-heavy or skewed workloads, while Python shell jobs can run on a fraction of a DPU. The follow-up interviewers want is the tuning trade-off: doubling workers halves runtime only if the job parallelises cleanly, so at best cost stays flat — with skew, stragglers, or small data, the extra DPUs idle and you pay for nothing. The strong answer is right-sizing from executor-utilisation metrics on real runs, not defaulting to 'add more workers'.

A DynamicFrame is Glue's schema-flexible wrapper: each record carries its own schema, so a field can hold multiple candidate types (a 'choice' type) without failing the read — built for messy JSON and drifting sources. A Spark DataFrame commits to one fixed schema up front and gives you the full Spark SQL API with Catalyst optimisation, which generally makes it faster. DynamicFrames add ETL-specific transforms — resolveChoice, relationalize, applyMapping — plus native integration with the Data Catalog and job bookmarks. The standard pattern: read as a DynamicFrame, resolve type ambiguity, call toDF() for complex joins and window functions, then fromDF() back to write through Glue sinks. The trade-off: DynamicFrames trade some performance for tolerance of dirty data.

Cost is DPU-hours, so attack both factors. Read less: partition the S3 data and use pushdown predicates so the job loads only relevant partitions, and prune columns early — cheap with Parquet. Process less: enable job bookmarks so each run is incremental rather than a full reload. Right-size: check executor utilisation in the job metrics; if half the workers sit idle, cut the count, or use auto scaling so Glue releases idle executors. Use the cheap tiers: Python shell jobs for single-node work, and Flex execution for non-urgent batch. Compact small files, which waste listing and task overhead. The trap: 'just reduce workers' — on a well-parallelised job that stretches runtime and leaves DPU-hours unchanged. Measure DPU-hours per run, not worker count.

They are three engines sharing one metadata layer, the Glue Data Catalog. Glue owns ETL: crawl sources, transform, write curated data — ideally partitioned, compressed Parquet — to S3, and register the tables. Athena runs serverless SQL directly over those same catalog tables, billed per TB scanned. Redshift Spectrum lets an existing Redshift cluster join its local warehouse tables against the same S3 external tables through an external schema mapped to the catalog. Define a table or add a partition once and all three see it immediately. The point to land: Glue prepares, Athena serves ad-hoc exploration, Spectrum serves warehouse joins. And since Athena and Spectrum bill by data scanned, Glue's conversion to partitioned columnar formats is precisely what makes the whole stack cheap.

Glue wins for managed, intermittent batch ETL: no cluster operations, per-second serverless billing, catalog and bookmark integration built in, minutes to stand up. EMR wins when you need control: specific Spark or Hadoop versions, custom libraries and AMIs, non-Spark engines like Trino, Hive, HBase, or Flink, long-running or interactive clusters, deep JVM tuning, or spot-instance strategies that undercut Glue's DPU pricing at sustained volume. The cost crossover is real: a job running most of every day is usually cheaper on a right-sized EMR cluster, while a 20-minute nightly job is cheaper on Glue. Team shape matters too: Glue suits teams that do not want to own infrastructure. Default to Glue for standard scheduled Spark ETL; move to EMR only on a concrete limitation you can name.

First decide who owns the schema. If the crawler owns it, set the update behavior so new columns are added but existing definitions are never overwritten — a naive re-crawl clobbering a manually corrected schema is the classic incident. If the job owns it, read with DynamicFrames, absorb type drift with resolveChoice, and use applyMapping to project the contract you promise downstream. Validate at the boundary: quarantine records that break the contract rather than letting nulls propagate. For fast-evolving data, the stronger answer is an open table format — Iceberg, Hudi, or Delta on S3, all usable from Glue — which gives real schema evolution with history. The principle: additive changes are safe; renames and type narrowing are breaking and should be versioned, not patched in place.

Start with evidence: enable the Spark UI and job metrics and see where time goes. One straggling task means skew — salt the hot keys or repartition. Heavy shuffle volume means wide transforms to reduce or reorder. Disk spill means memory pressure — larger workers, not more of them. Check the read path: is the pushdown predicate actually pruning partitions, or is the job listing all of S3? Many small files are the most common silent killer — compact them or use groupFiles/groupSize on the read. Idle executors mean over-provisioning — cut workers or enable auto scaling. Confirm bookmarks are on so you skip already-processed data, and check formats: gzipped CSV is unsplittable, so one file pins one core; convert to Parquet. The point is order: measure, fix the dominant bottleneck, re-measure.

Within Glue, use triggers and Workflows: a trigger fires a job or crawler on a schedule, on demand, or conditionally on the completion of others, and a Workflow groups the DAG with shared run properties and one view of execution. That covers linear crawl-transform-load chains well. For branching, retries with backoff, error fan-out, or steps in other services (Lambda, ECS, Athena), move up to Step Functions, which has native Glue integration, or Managed Airflow when the team thinks in DAG code and needs rich backfills. Use EventBridge for event-driven starts, e.g. on S3 object arrival, instead of polling schedules. The senior points: keep every job idempotent so retries are safe, and keep dependency logic in the orchestrator, not buried inside jobs, so each stays independently runnable.