Scala Interview Questions for Data Engineers (2026): 16 Answers That Hold Up

val declares an immutable reference — once assigned, it cannot be reassigned; var can be reassigned. The subtlety interviewers probe: val makes the reference immutable, not the object. A val holding an ArrayBuffer still allows appending elements; it only prevents rebinding the name to a different buffer. Idiomatic Scala defaults to val everywhere and reaches for var only in tight local scopes with a measured reason. Why it matters: immutable references make code easier to reason about, are safe to share across threads, and fit the functional style Spark code is written in — transformations produce new values instead of mutating state. A good closing line in an interview: "I use val by default, immutable collections by default, and treat any var as a code-review flag."

A case class is a class with compiler-generated machinery for modeling immutable data: constructor parameters become public vals, and you get structural equals/hashCode, a readable toString, a copy method, an apply factory in the companion (no new needed), and unapply, which is what makes pattern matching work. Compare Person("A") == Person("A"): true for case classes (field-by-field), false for plain classes (reference equality). In data engineering they are everywhere — records flowing through a pipeline, typed Dataset rows in Spark, decoded Kafka payloads. copy deserves a mention: record.copy(status = "done") gives you an updated record without mutation, which is exactly how immutable data is evolved. Mention that case classes are ideal leaves of sealed trait hierarchies, which is how Scala models algebraic data types.

Option[A] is a container with two cases: Some(value) or None. It moves "this might be absent" from documentation into the type system, so the compiler forces callers to handle the missing case — the root fix for NullPointerException-style bugs. Idiomatic use avoids .get (which throws on None and defeats the purpose): use getOrElse for defaults, map/flatMap to transform if present, fold or pattern matching to handle both branches, and for-comprehensions to chain several optional lookups. Example: userMap.get(id).map(_.email).getOrElse("unknown"). In pipelines, Option is how you represent nullable columns or failed lookups without sentinel values like empty strings or -1, which silently corrupt downstream logic. If asked when Option is insufficient, say: when you need the reason for absence, upgrade to Either.

A higher-order function takes a function as a parameter, returns a function, or both. Scala's collections API is built on them: map transforms each element, filter keeps matches, foldLeft accumulates. Example: salaries.filter(_ > 500000).map(_ * 1.1) reads as a declaration of intent rather than a loop with mutable state. The underscore syntax (_ * 2) is shorthand for an anonymous function (x => x * 2). Functions are first-class values, so you can store and pass them: val isSenior: Employee => Boolean = _.years >= 5. This is worth internalizing for data work — a Spark pipeline is literally higher-order functions applied to distributed collections, so fluency with these combinators on List transfers directly to writing and reading production jobs.

Scala defaults to immutable collections (List, Vector, Map, Set from scala.collection.immutable); operations return new collections, sharing structure internally so it is cheaper than a full copy. Mutable variants (ArrayBuffer, mutable.Map) live in a separate package you must import deliberately — that friction is by design. Performance facts interviewers like: List is a singly linked list — O(1) prepend and head, O(n) random access; Vector is a tree-backed indexed sequence with effectively constant-time access and updates, a better general default for large sequences; Array is the raw JVM array — mutable, fixed-length, fastest for primitives and interop. Pick List for head-recursive processing, Vector as a general-purpose default, Array at performance-critical boundaries. Saying "I default to immutable and justify each mutable use" is the answer interviewers want.

A match expression compares a value against patterns: literals, types, destructured case classes, guards, and alternatives. case Order(id, Some(coupon), amount) if amount > 1000 => binds fields and applies a condition in one line. The interview-grade part is sealed traits: marking a trait sealed restricts implementations to the same file, so the compiler knows every subtype. Combined with match, you get exhaustiveness checking — forget a case and you get a compile-time warning instead of a runtime MatchError. That is how Scala models algebraic data types: sealed trait JobStatus with case object Queued, case class Running(startedAt: Instant), and case class Failed(reason: String) is safer than string statuses, because adding a new state breaks every non-updated match loudly. Mention that matching works through unapply, so you can also write custom extractors.

All three encode "may not have a value" at different information levels. Option[A] says a value may be absent, with no reason — right for lookups and nullable fields. Either[E, A] carries why it failed in the Left; since Scala 2.12 it is right-biased, so map/flatMap operate on the success side and you can chain validations in a for-comprehension, short-circuiting on the first Left. Try[A] (Success/Failure) wraps code that throws — it catches non-fatal exceptions, making it the bridge between exception-throwing Java or legacy APIs and functional code: Try(dateFormat.parse(s)). Rules of thumb: Option for plain absence, Either for domain errors you want typed and named, Try at the boundary with exception-throwing code — converting outward with .toEither or .toOption as data moves into your core logic.

Both can hold abstract and concrete members. Differences: a class can mix in many traits but extend only one abstract class; in Scala 2 traits cannot take constructor parameters (Scala 3 allows them); abstract classes map cleanly onto Java for interop. Traits are Scala's tool for composing behavior — small capabilities like Logging or JsonSupport stacked onto a class. When multiple traits define the same method, Scala resolves it by linearization: the class and its traits are flattened into one ordered chain (roughly right-to-left through the extends ... with ... clause), and super calls walk that chain. This enables "stackable modifications" with abstract override, where each trait wraps the previous implementation. Close with practical guidance: use an abstract class for constructor state or Java interop; otherwise default to traits for composability.

map applies A => B inside a context, keeping the shape: Option[A] => Option[B], List[A] => List[B]. flatMap applies A => F[B] and flattens — what you need when the function itself returns a wrapped value; mapping instead would give Option[Option[B]] or List[List[B]]. Chained lookups make it concrete: getUser(id).flatMap(getAccount).map(_.balance). A for-comprehension is compiler sugar over exactly these calls — for { u <- getUser(id); a <- getAccount(u) } yield a.balance desugars into flatMap/map (plus withFilter for guards). That is why for-comprehensions work uniformly across Option, Either, Try, Future, and collections: anything with map and flatMap qualifies. In interviews, being able to hand-desugar a small for-comprehension is a reliable signal you understand the mechanics, not just the syntax.

Contrast four evaluation strategies. val evaluates once, immediately, at definition. def evaluates on every call. lazy val evaluates once, on first access, and memoizes — with thread-safe initialization. A by-name parameter (x: => T) passes an unevaluated expression that re-evaluates each time it is used inside the function — this is how getOrElse avoids computing its default unless needed, and how you write constructs like retry(3) { fetchPage() }. Use lazy val for expensive resources you may never touch (config parsing, connections) and for breaking initialization-order problems between vals. For collections, .view and LazyList defer element computation, so hugeList.view.map(expensive).take(5) computes only five elements. Worth one closing sentence: Spark transformations are also declared lazily and only execute at an action — the same principle applied at cluster scale.

An object is a singleton; a companion object is one with the same name as a class, in the same file. The pair can access each other's private members. Companions hold what Java calls statics: constants, factory methods, typeclass instances. Two special methods matter. apply lets the object be called like a function — Person("Asha") is Person.apply("Asha") — which is why case classes need no new, and why you can hide constructors behind validating factories that return Option or Either. unapply runs in reverse: it deconstructs a value for pattern matching; case classes generate it automatically, and writing your own gives you a custom extractor. A strong concrete example: a companion Email.fromString(s): Either[String, Email] guaranteeing every Email in the system was validated at construction — smart constructors in one sentence.

reduce combines elements pairwise using the elements themselves — no initial value, so the result type must match the element type and it throws on an empty collection. foldLeft(z)(f) starts from an explicit initial accumulator, so it is total on empty input and the accumulator can be a different type — folding rows into a Map[String, Int], for instance. foldLeft is strictly sequential, left to right, which is fine on one machine. The distributed angle earns marks: parallel and distributed folds (like Spark's reduce and aggregate) combine partial results from partitions in unspecified order, so the operation must be associative and the initial value a true identity — summing works; subtraction or order-dependent string building silently breaks. Rule: foldLeft for type-changing sequential accumulation with a safe empty case; reduce for associative same-type combination on known non-empty data.

Several language properties made Scala the natural host. First, the JVM: Spark needed to interoperate with the Hadoop ecosystem (HDFS, YARN, Java libraries) and benefit from mature JIT performance. Second, Scala's immutable collections API — map, filter, flatMap, reduce over immutable data — was the direct template for RDDs: an RDD presents as an immutable distributed collection with the same combinators, so the language's idioms became the engine's API. Third, immutability plus deterministic transformations suits Spark's fault-tolerance model, where lost partitions are recomputed from lineage rather than replicated. Fourth, closures make it natural to write a function inline and ship it to executors. Finally, static typing enabled the typed Dataset API. Practical consequence: Scala is Spark's first-class API — internals and some features, like typed Datasets, live on the JVM side.

For DataFrame and SQL workloads the honest answer is: largely equivalent. Both front-ends compile to the same Catalyst plans, so a filter-aggregate-join pipeline performs similarly in either language. Differences live at the edges. Python UDFs move rows between JVM executors and Python worker processes with serialization both ways — historically expensive; pandas UDFs on Arrow narrow but do not erase the gap. Scala UDFs run in-process on the JVM. The typed Dataset API, custom data sources, and most extension points are JVM-only, and compile-time typing catches schema drift earlier. Python wins on the surrounding ecosystem — pandas, ML tooling — and hiring breadth. A strong answer picks by context: PySpark for DataFrame-centric analytics teams; Scala where UDF-heavy transforms, connector development, or an existing JVM codebase dominate — and prefer built-in column expressions over UDFs in either language.

Variance defines how subtyping of type parameters relates to subtyping of the container. List[+A] is covariant: because lists are immutable, a List[String] can safely be used wherever List[Any] is expected. Array[A] is invariant: it is mutable, and covariant mutable containers are unsound — Java made arrays covariant and pays with runtime ArrayStoreException; Scala rules it out at compile time. Contravariance (-A) flips the direction and applies to consumers: Function1[-A, +R] means an Any => Int can stand in where a String => Int is required — a function handling any input certainly handles strings. Rule of thumb: producers of A can be covariant, consumers contravariant, and anything that both reads and writes A must be invariant. The compiler enforces this by rejecting covariant types in method-parameter position. This question reliably separates syntax readers from people who understand why.

When a Scala closure references anything from its enclosing scope, the compiler captures those references into the function object. Spark serializes that function and ships it to executors — if it captured something non-serializable, the job dies at submission with "Task not serializable". The classic trap: rdd.map(x => x * this.factor) does not capture just factor; using a field or method captures this, dragging the entire enclosing class — connections, loggers, SparkContext — into the closure. Fixes, in order of preference: copy needed fields into local vals before the closure (val f = factor; rdd.map(_ * f)); move helper logic into a top-level object, which serializes as a singleton reference; mark non-serializable members @transient lazy val so each executor rebuilds them locally; only as a last resort make the enclosing class Serializable. This is pure Scala closure semantics — the distributed runtime just makes the capture visible and expensive.