Pandas Interview Questions: 16 Real Questions with Expert Answers

pd.merge is a SQL-style join on one or more columns (or indexes) with how='inner'/'left'/'right'/'outer'. df.join is a convenience method that joins on the index by default, and pd.concat stacks DataFrames along an axis — axis=0 appends rows, axis=1 aligns on the index and adds columns. Worth adding: concat with mismatched columns produces a NaN-filled union unless you pass join='inner'.

Detect with isna()/notna(), drop with dropna(subset=..., how=...), fill with fillna() using a scalar, a per-column dict, or ffill/bfill, and use interpolate() for numeric gaps. A detail interviewers like: np.nan is a float, so an integer column with missing values silently becomes float64 unless you use nullable dtypes like Int64 with pd.NA.

df.drop_duplicates(subset=['col1','col2'], keep='first') deduplicates on chosen columns; keep='last' or keep=False change which rows survive, and df.duplicated() returns the boolean mask if you want to inspect duplicates first. For 'latest record per key', sort by timestamp then drop_duplicates(subset='key', keep='last').

Boolean masking is the standard: df[(df['salary'] > 500000) & (df['city'] == 'Bangalore')] — note the parentheses and & / | instead of and/or, because masks are element-wise arrays. df.query("salary > 500000 and city == 'Bangalore'") is a readable alternative, and isin() handles membership checks against a list of values.

apply with axis=1 calls a Python function once per row, losing NumPy's C-level vectorization and paying interpreter overhead millions of times. Replace it with vectorized column arithmetic, np.where/np.select for conditionals, .str methods for strings, and map/replace for lookups. A typical result: a row-wise apply taking seconds on 1M rows drops to milliseconds as np.where(df['a'] > df['b'], 'x', 'y').

pivot reshapes long to wide but raises ValueError if an index/column pair has duplicates; pivot_table handles duplicates by aggregating them (aggfunc='mean' by default, with margins=True for totals). melt is the reverse — it unpivots wide columns into long id_vars/value_vars format, which is what you want before a groupby. Also know stack/unstack for MultiIndex-based reshaping.

Measure with df.memory_usage(deep=True), then downcast numerics (astype('int32'), pd.to_numeric(downcast=...)), convert low-cardinality strings to category — a city column with 50 unique values over 10M rows can shrink 10-20x — and pass dtype= and usecols= to read_csv so waste never loads at all. In pandas 2.x, the pyarrow-backed string dtype is far lighter than object.

First cut columns and types at read time — read_csv(usecols=..., dtype={...}, parse_dates=[...]) often shrinks the working set 5-10x. If it still doesn't fit, stream with chunksize=1_000_000, aggregate each chunk, and combine partial results, which works for any distributive computation like sums, counts, or groupby partials. For repeated access, convert once to partitioned Parquet — and be honest that beyond a point the right tool is DuckDB, Polars, or Spark rather than pandas.

pd.merge_asof performs an ordered nearest-key join: for each left row it picks the latest right-side row at or before its timestamp (direction='backward' by default), with tolerance= to cap the allowed gap and by= for per-key alignment such as symbol-wise quotes-to-trades matching. Both inputs must be sorted on the join key. It is the standard answer for trades vs quotes, sensor readings vs events, or attaching the most recent FX rate to transactions.

The fastest pandas idiom is one sort plus a linear pass:

df.sort_values('ts').drop_duplicates('user_id', keep='last')

This reads cleaner but computes an index per group, and is slower on high-cardinality keys:

df.loc[df.groupby('user_id')['ts'].idxmax()]

At 100M rows, also say you would push it to the warehouse with ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY ts DESC) and pull only the result — interviewers reward knowing when not to use pandas.