Pandas Interview Questions: 16 Real Questions with Expert Answers

loc selects by label and iloc by integer position. The classic gotcha: loc slicing is inclusive of the end label (df.loc['a':'c'] includes 'c'), while iloc follows Python semantics and excludes the end (df.iloc[0:3] returns rows 0-2). loc also accepts boolean masks, which is the idiomatic way to filter with assignment: df.loc[mask, 'col'] = value.

pd.merge is a SQL-style join on one or more columns (or indexes) with how='inner'/'left'/'right'/'outer'. df.join is a convenience method that joins on the index by default, and pd.concat stacks DataFrames along an axis — axis=0 appends rows, axis=1 aligns on the index and adds columns. Worth adding: concat with mismatched columns produces a NaN-filled union unless you pass join='inner'.

Detect with isna()/notna(), drop with dropna(subset=..., how=...), fill with fillna() using a scalar, a per-column dict, or ffill/bfill, and use interpolate() for numeric gaps. A detail interviewers like: np.nan is a float, so an integer column with missing values silently becomes float64 unless you use nullable dtypes like Int64 with pd.NA.

df.drop_duplicates(subset=['col1','col2'], keep='first') deduplicates on chosen columns; keep='last' or keep=False change which rows survive, and df.duplicated() returns the boolean mask if you want to inspect duplicates first. For 'latest record per key', sort by timestamp then drop_duplicates(subset='key', keep='last').

Boolean masking is the standard: df[(df['salary'] > 500000) & (df['city'] == 'Bangalore')] — note the parentheses and & / | instead of and/or, because masks are element-wise arrays. df.query("salary > 500000 and city == 'Bangalore'") is a readable alternative, and isin() handles membership checks against a list of values.

agg reduces each group to one row (df.groupby('dept')['sal'].agg(['mean','max'])); transform returns output aligned to the original DataFrame's shape, letting you broadcast group statistics back — df['dept_avg'] = df.groupby('dept')['sal'].transform('mean'); filter keeps or drops entire groups on a group-level condition. Transform is the expected answer to 'add each employee's deviation from department average without a join'.

apply with axis=1 calls a Python function once per row, losing NumPy's C-level vectorization and paying interpreter overhead millions of times. Replace it with vectorized column arithmetic, np.where/np.select for conditionals, .str methods for strings, and map/replace for lookups. A typical result: a row-wise apply taking seconds on 1M rows drops to milliseconds as np.where(df['a'] > df['b'], 'x', 'y').

It appears with chained indexing like df[df['x'] > 0]['y'] = 1, where pandas cannot guarantee whether you are writing to a view or a temporary copy — the assignment may silently do nothing. The fix is a single .loc call: df.loc[df['x'] > 0, 'y'] = 1. Copy-on-Write in pandas 2.x (default in 3.0) removes the ambiguity entirely: chained assignment simply never modifies the original.

pivot reshapes long to wide but raises ValueError if an index/column pair has duplicates; pivot_table handles duplicates by aggregating them (aggfunc='mean' by default, with margins=True for totals). melt is the reverse — it unpivots wide columns into long id_vars/value_vars format, which is what you want before a groupby. Also know stack/unstack for MultiIndex-based reshaping.

Measure with df.memory_usage(deep=True), then downcast numerics (astype('int32'), pd.to_numeric(downcast=...)), convert low-cardinality strings to category — a city column with 50 unique values over 10M rows can shrink 10-20x — and pass dtype= and usecols= to read_csv so waste never loads at all. In pandas 2.x, the pyarrow-backed string dtype is far lighter than object.

resample is a time-based groupby over a DatetimeIndex: df.resample('D')['sales'].sum() buckets rows into fixed calendar bins (D, W, ME, 15min) and aggregates each bin, changing the output frequency. rolling computes a sliding-window statistic over consecutive rows or a time offset (rolling('7D').mean()) while keeping the original frequency. asfreq changes frequency without aggregating — it only selects and fills, which is why asfreq('D') on sparse data introduces NaNs.

Duplicate keys on either side cause a many-to-many explosion — a 1M-row merge can silently become 50M rows. Guard with validate='one_to_many' (raises MergeError on violation), use indicator=True to audit left_only/right_only rows after outer joins, and set suffixes explicitly so colliding columns don't become ambiguous _x/_y. Also verify key dtypes match — merging int64 keys against object keys matches nothing.

First cut columns and types at read time — read_csv(usecols=..., dtype={...}, parse_dates=[...]) often shrinks the working set 5-10x. If it still doesn't fit, stream with chunksize=1_000_000, aggregate each chunk, and combine partial results, which works for any distributive computation like sums, counts, or groupby partials. For repeated access, convert once to partitioned Parquet — and be honest that beyond a point the right tool is DuckDB, Polars, or Spark rather than pandas.

Historically, whether indexing returned a view or a copy was unpredictable — it depended on internal block layout — which made chained assignment treacherous and forced SettingWithCopyWarning to exist. With CoW (pd.options.mode.copy_on_write = True; default in pandas 3.0), every derived object behaves as an independent copy, but data is only physically copied when you actually write, giving copy semantics at near-view performance. Practically: chained assignment never mutates the parent, and defensive .copy() calls mostly disappear.

pd.merge_asof performs an ordered nearest-key join: for each left row it picks the latest right-side row at or before its timestamp (direction='backward' by default), with tolerance= to cap the allowed gap and by= for per-key alignment such as symbol-wise quotes-to-trades matching. Both inputs must be sorted on the join key. It is the standard answer for trades vs quotes, sensor readings vs events, or attaching the most recent FX rate to transactions.

The fastest pandas idiom is sort_values('ts').drop_duplicates('user_id', keep='last') — one sort plus a linear pass. df.loc[df.groupby('user_id')['ts'].idxmax()] reads cleaner but computes an index per group and is slower on high-cardinality keys. At 100M rows, also say you would push it to the warehouse with ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY ts DESC) and pull only the result — interviewers reward knowing when not to use pandas.