Pay Equity: the gap that mostly isn't (raw vs. adjusted)

The data in this project is synthetic — generated locally by data/make_data.py, with no external source. That is deliberate: only a known generative model tells you which pay gaps are genuine and which are pure confounding, so you can score the method. No signup, no downloads, no GPU.

Before you start

New to confounding / adjusted comparisons? Start here.

Compare the average pay of two groups and you get a raw gap. But averages hide why. If group B is concentrated in lower-paying job levels, group B’s average will be lower even if every person is paid exactly the market rate for their level. The raw gap then measures where people sit, not unequal pay for equal work.

The fix is an adjusted comparison: hold the legitimate pay factors (job level, tenure) equal and ask whether a gap remains. A regression does this — the coefficient on “is in group B,” with level and tenure in the model, is the gap that survives the controls. Often the scary raw gap shrinks dramatically once adjusted. Sometimes it even flips sign — that’s Simpson’s paradox.

A tiny example:

                    group A     group B
avg pay             $95k        $80k      -> raw gap looks like -16%
but... % at senior  60%         20%       -> B is mostly in junior roles
same-level pay      equal       equal     -> adjusted gap ~ 0

What you’ll learn: measure a raw gap, expose the confounder, estimate the adjusted gap with a regression, and then decide — per department — which gaps are genuine and which are explained away. Section 3 lists exactly what you’ll be able to do.

New to Python too? See Start Here — it starts from zero, and shows how to run this in your browser with nothing to install.

1. The Brief

You’re handed HR compensation data and the question every People team dreads getting wrong: do we have a pay-equity problem, and where? You’ll show why the raw gap overstates it, adjust for the legitimate factors, and pinpoint the departments with a genuine residual gap — instead of raising a false alarm across the whole company or missing the real ones.

This is a high-stakes, high-visibility analysis, and it is routinely botched in both directions: a raw gap reported as discrimination (ignoring role mix), or a real gap buried because “on average we’re fine.” Regulators, boards, and employees all ask the question; the credible answer is the adjusted one, computed transparently. Getting raw-vs-adjusted right is the analytical core of the whole field.

2. Difficulty & time

Difficulty 5 / 10. It needs a regression and the judgment to read its coefficient as an adjusted effect — a real modelling step, plus the confounding insight that a raw average is not a like-for-like comparison. Calibrated against the ledger (G10): a peer of accuracy-is-lying and analytics-with-duckdb (both 5) — one genuine analytical pipeline with a conceptual trap. It sits above maverick-spend (4), which applies fixed rules with no model, and below recommend-from-scratch (7), which derives an algorithm from scratch. The regression here is a few lines of scikit-learn; the difficulty is in interpreting it, not implementing it.

Time is separate from difficulty: Part A is about 75 minutes; Part B is 2–3 hours of your own work.

3. What you’ll be able to do after

Measure a raw gap in log-salary (so a difference reads as an approximate percentage) and explain why it isn’t a like-for-like comparison.
Expose a confounder — show how group composition across job levels drives most of a raw gap — with a simple crosstab.
Estimate an adjusted gap by regressing log-salary on group plus controls, and read the group coefficient correctly.
Decide per department which gaps are genuine residual penalties and which are explained away — the difference between a real finding and a false alarm.

The finished result

By the end of Part A you’ll have watched a scary raw gap shrink to almost nothing once you control for job level — and pinned the gaps that survive to the exact departments:

raw gap (B vs A):                    -15.4%     <- alarming, on the plain average
mean job level:   A = 3.40   vs   B = 2.29      <- but group B sits in the lower levels
adjusted gap (same level & tenure):   -2.2%     <- most of the raw gap was WHERE people sit
per department (15 total, 5 truly unequal):
    flag by RAW gap:       15 depts   (over-flags -- the confounder)
    flag by ADJUSTED gap:   5 depts   (exactly the 5 real ones)

Most of a −15% headline was role mix, not unequal pay for equal work. In Part B you’ll make that same adjusted call across 40 departments and clear an F1 floor, where the raw-gap rule flags nearly all of them and fails.

4. Prereqs & time box

You can write a function and use a pandas DataFrame; you’ve seen a linear regression once (or are willing to treat LinearRegression().fit(X, y).coef_ as a black box that returns the per-feature effect). No prior pay-equity or econometrics background needed.

Where to write and run code

Two ways to run this project — pick either:

In your browser (no install). If the project page shows a ▶ Run button, click it to run Part A right here — nothing to set up. Part A uses pandas, numpy, and scikit-learn, which run in the browser.
On your computer. The one-time setup — installing Python, downloading the code from the course site (no GitHub, no git), opening a terminal, creating a virtual environment, installing the requirements, and running Part A step by step — is covered once in Start Here. Come back here once python -c "import sklearn" runs without error. Part B (writing your own solution) needs this local setup.

Time box

Part A — Guided Build: 60–90 minutes. Hard cap.
Part B — Your Turn: 2–4 hours. Hard cap.

5. Setup & data

The data is synthetic and generated locally — see the banner above and SOURCES.md for full provenance. One command creates both files:

python data/make_data.py

That writes two CSVs into data/ (git-ignored; regenerate anytime, byte-for-byte identical because the seed is fixed):

Part A — employees_a.csv (1,088 employees, 15 departments).
Part B — employees_b.csv (2,811 employees, 40 departments).

Each row is emp_id, department, group, level, tenure_years, salary. group is the protected attribute (A/B); level (1–5) and tenure_years are the legitimate pay factors. Which departments carry a genuine residual gap is not in the file — the grader recovers it from the seed.

6. Part A — Guided Build

You’ll analyze the 1,088-employee employees_a.csv. Run the whole thing, or (better) one # %% cell at a time — see Start Here:

python part_a/measure_gap.py

Prefer the browser? If this page has a ▶ Run button, click it instead of the command above — Part A runs in your browser with nothing to install. The STEP output and every checkpoint below are identical either way.

The full, commented script is in part_a/measure_gap.py. The spine:

Step 1 — The raw gap. Mean log-salary difference, B vs A.

Checkpoint: raw gap ≈ −15%. Alarming — but hold on.

Step 2 — The confounder. Crosstab of group by job level.

Checkpoint: group B is concentrated in the lower levels (mean level ~2.3 vs ~3.4 for A). Lower levels pay less, so much of the raw gap is where people sit.

Step 3 — The adjusted gap. Regress log-salary on group + level + tenure; read the group coefficient.

Checkpoint: adjusted gap ≈ −2%, far smaller than the raw −15%. Most of the raw gap was the level confounder — the whole lesson in one comparison.

Step 4 — Per department. Compute the adjusted gap for each department; flag the genuine ones, and compare to flagging by the raw gap.

Checkpoint: 15 departments, 5 with a genuine gap. The raw rule flags all 15 (confounding); the adjusted rule flags exactly the 5 real ones.

Step 5 — Save the per-department verdict to part_a/gap_by_department.csv.

Why this and not that

Why log-salary, not raw dollars. Pay is multiplicative (a “10% gap” means more in senior roles). Differences in log-salary read as approximate percentages and keep the regression well-behaved across pay scales.
Why control for level and tenure, but not for everything. You control for legitimate pay factors to isolate the unexplained gap. Throwing in a variable that is itself affected by the group (a “bad control”) can hide the very gap you’re testing for — adjustment is a judgment, not a reflex.
Why per-department, not one company number. A single adjusted gap can average a real problem in a few departments into an innocent-looking whole. Disaggregating is where the actionable finding lives.

Functions you’ll meet (and where to read more)

Function	What it does here	Docs
`np.log(series)`	Work in log-salary so gaps read as percentages	numpy.log
`pd.crosstab(a, b, normalize=)`	Show the group × level composition (the confounder)	crosstab
`LinearRegression().fit(X, y)`	Estimate the adjusted gap (the group coefficient)	LinearRegression
`np.column_stack([...])`	Build the feature matrix (group, level, tenure)	column_stack
`df.groupby("department")`	Fit the adjusted gap one department at a time	groupby

If something looks off

The adjusted gap is much smaller than the raw gap — did I break something? No — that shrink is the finding. The raw −15% mixes in the fact that group B sits in lower-paying job levels; controlling for level and tenure (the legitimate pay factors) strips that out and leaves the ~−2% that is actually unequal pay for equal work. A gap that collapses once you adjust is the expected result here, not an error — only what survives the controls is a like-for-like comparison.
A checkpoint number doesn’t match. The data is seeded, so the numbers are exact. Re-run python data/make_data.py (you may have skipped or edited it), then re-run Part A.
Setup problems (Python not found, pip, FileNotFoundError, re-activating the venv): see the troubleshooting list in Start Here.

7. Part B — Your Turn

Part B is the optional “your turn” half — a guided fill-in-the-blank, not a blank page. You get a working starter (part_b/starter.py) that already handles the easy version; your job is to finish the # TODO it marks out. More open than Part A’s step-by-step, but you are never staring at an empty file. If you finished Part A and want to prove the skill, this is where it happens. New to Python? It’s completely fine to stop after Part A and come back later.

Same core skill, more departments, and now scored. For each of the 40 departments in employees_b.csv, decide whether it carries a genuine residual gap. The starter (part_b/starter.py) flags by the raw gap — Part A’s Step 1 — and scores ~0.49, below the floor.

The transfer: the raw gap is negative in almost every department (group B sits lower), so a raw-gap rule flags nearly all of them. Only the adjusted gap, computed per department, separates the genuine penalties (~13 of 40) from the explained-away ones.

Acceptance criteria (checkable)

Your solution writes into part_b/out/:

flagged.csv — one row per department: department, flagged_gap (1 if genuine gap).
report.json — with keys departments, flagged, achieved_metric (you can leave achieved_metric null; the grader scores you).

Then:

python part_b/starter.py
pytest part_b/test_solution.py -q

If pytest reports everything as skipped, it just means the output files don’t exist yet — run python part_b/starter.py first (once your solution fills in the TODOs), then re-run.

You’re done when the tests pass: your per-department verdict reaches F1 ≥ 0.85 against the true set (the reference reaches ~0.96). Flagging by the raw gap scores ~0.49 and fails — the adjusted regression is what clears the floor.

Constraints — what must be true of your solution, not how to get there

Judge each department by an adjusted gap (control for level and tenure), not the raw average.
Emit a verdict for every department, genuine or not.
Don’t over-flag: your flagged count should be near the true ~13, not near 40.

Hints

Hint 1 — reuse Step 3, per department

You already have the adjusted-gap regression from Part A. Run it inside a groupby("department") loop and keep the coefficient on group == "B" for each department.

Hint 2 — the decision rule

Flag a department whose adjusted group-B coefficient is below a small negative threshold (e.g. −0.03 in log-points, ≈ −3%). Genuine gaps sit well below it; explained-away ones sit near zero.

Hint 3 — sanity-check the count (still not the code)

Compare your adjusted-flag count to the raw-flag count. If they’re nearly equal, you’re still flagging by the raw gap — the whole point is that the adjusted rule flags far fewer.

8. Self-check

You don’t need the answer key. You’re done when all of these hold:

pytest reports F1 ≥ 0.85 (aim for ~0.96 — the adjusted rule separates the classes cleanly).
Your flagged count is far below the raw-gap count (you adjusted for the confounder).
Re-running from python data/make_data.py reproduces the same numbers — the data is seeded.

And the tell-tale: if you flag almost every department, you’re reading the confounder as a penalty — exactly the mistake this project exists to prevent.

9. Stretch

Confidence, not just a point. A coefficient without a standard error is half an answer. Compute the group coefficient’s standard error (or bootstrap it) and flag only departments where the gap is significantly negative — a more honest rule than a fixed threshold.
Which control matters? Re-run the adjusted gap dropping level, then dropping tenure. Show how much of the raw gap each control explains.
(Genuinely hard) A bad control. Add a variable that is itself influenced by group (say, “assigned to a high-visibility project”) and show how controlling for it hides a real gap — the cautionary tale of over-adjustment.

10. Ship it

Put this in a portfolio repo and it counts. In that repo’s README:

State the finding in one sentence a non-analyst gets: “the −15% raw gap is mostly role mix; the real, unexplained gap is in these ~13 departments.”
Show the raw → adjusted collapse and the per-department split (raw flags everything; adjusted flags the real ones).
Include the adjusted-gap function and one paragraph on confounding and the risk of a bad control. That paragraph is the part that signals you understand adjustment, not just .fit().

Keep make_data.py and the tests in the repo so anyone can reproduce your numbers from zero.

11. Sources

Rendered from SOURCES.md.

Field	Value
Dataset	Synthetic HR compensation — employees, levels, pay (Part A 1,088, Part B 2,811 employees)
Type	Synthetic
Generated by	`data/make_data.py` (this repo), seeds `20260718` (A) and `20260719` (B)
Source URL	Not applicable — generated, not fetched
License	Not applicable — synthetic, no third-party rights
Access / generation date	2026-07-18
Redistribution	Permitted — synthetic; output regenerated from the seeded script, never committed
robots.txt checked	Not applicable — no scraping
ToS URL	Not applicable — no external source

Synthetic data teaches the mechanic; it does not prove a finding about the real world. The gaps and the confounding were injected on purpose — recovering them demonstrates the technique, not a claim about any real workforce. Real pay-equity work needs domain and legal care beyond this exercise.

Next up

Finished this one? Continue the AI & Machine Learning track:

Attrition Leakage: the model that’s too good to be true → · Browse all courses