Uplift Targeting: spend on the persuadable, not the sure thing

Project at a glance

• Track: AI & Machine Learning
• Difficulty: Advanced · 8/10
• Time: ~1.5 h for Part A · ~4.0 h more for the optional Part B
• You’ll use: pandas, numpy, scikit-learn, pytest
• 💻 Local setup needed (no in-browser ▶ Run yet) — a one-time, free Python install, then you just run files. Start Here walks you through it.
• Heads up — this is an advanced project. It assumes you’re comfortable with pandas, numpy, scikit-learn, pytest and the concepts behind them. New to the platform mechanics only? See Start Here.

The data in this project is synthetic — generated locally by data/make_data.py, with no external source. That is deliberate: uplift is a counterfactual (you never see both the treated and untreated outcome for the same person), so only a generative model gives you the true per-customer effect to score against. No signup, no downloads, no GPU.

Before you start

New to uplift / treatment effects? Start here — it’s a genuine shift in what you predict.

Most targeting models predict response: who is likely to convert. But a marketing offer costs money, so what you actually want is uplift (a.k.a. incremental effect): who converts because of the offer. Those are different people. Kevin Hillstrom’s four segments make it concrete:

• Persuadables — convert only if treated. This is who you want to spend on.
• Sure things — convert either way. Treating them wastes money (they’d have converted free).
• Lost causes — never convert. Wasted money.
• Sleeping dogs — convert less when treated (the offer annoys them). Treating them is actively harmful — negative uplift.

A response model loves sure-things (high conversion probability) and can’t even see that sleeping dogs exist. An uplift model estimates the difference the treatment makes for each person — which needs a randomized experiment (some treated, some held out as control) so you can compare like with like.

What you’ll learn: measure the average treatment effect, see why targeting by response misfires, estimate per-customer uplift with a two-model T-learner (treated model minus control model), and target the top slice by uplift. Section 3 lists exactly what you’ll be able to do.

New to Python too? See Start Here — it starts from zero, and shows how to run this in your browser with nothing to install.

1. The Brief

You ran a randomized campaign — some customers got the offer, some didn’t — and now you must decide who to target next time on a limited budget. You’ll show why “target the likely converters” quietly wastes most of the spend, then build an uplift model that finds the persuadables and measure how much more incremental conversion it captures for the same budget.

Uplift modelling is the difference between a campaign that looks busy and one that actually causes sales. Retailers, banks, and subscription businesses run exactly this analysis to decide who gets a discount or a retention call — and getting it wrong means paying people who’d have bought anyway (or nagging the ones you’re about to lose). It’s also a clean, portable introduction to causal thinking: predicting an effect, not an outcome.

2. Difficulty & time

Difficulty 8 / 10. This is the batch’s hardest: it requires causal, counterfactual thinking (you can never observe an individual’s uplift directly), a two-model estimator built correctly off the treatment/control split, and an uplift-specific evaluation (captured uplift vs an oracle) that ordinary accuracy/AUC can’t express. Calibrated against the ledger (G10): it sits above recommend-from-scratch (7) — that one is demanding implementation (gradient descent by hand) on a familiar objective, whereas this one is demanding conceptually (response ≠ uplift, sleeping dogs, targeting an unobservable effect) even though the code is scikit-learn. It’s well above the difficulty-5/6 modelling projects (pay-equity-gap, attrition-leakage), which predict observable outcomes. The implementation is short; the reasoning is the hard part.

Time is separate from difficulty: Part A is about 90 minutes; Part B is 3–4 hours of your own work.

3. What you’ll be able to do after

• Measure the average treatment effect (ATE) from a randomized split, and explain why the average hides who to target.
• Explain why response targeting misfires — it spends on sure-things (zero uplift) and can even target sleeping-dogs (negative uplift).
• Estimate per-customer uplift with a T-learner — a model on the treated, a model on the control, and their difference.
• Evaluate targeting honestly — measure the fraction of achievable uplift a top-slice campaign captures, and compare strategies on the same budget.

The finished result

By the end of Part A you’ll have targeted the persuadables instead of the sure-things, and measured how much more incremental conversion that captures on the same 30% budget:

average treatment effect (ATE):  +2.39%   -- one average; it can't say who to target
response targeting, top 30%:  captures 0.17 of the achievable uplift
uplift targeting (T-learner), top 30%:  captures 0.87   -- about 5x, same budget
targeted group's mean true uplift:  uplift +0.359  vs  response +0.068

Same budget, roughly 5x the incremental conversions — you paid to change minds, not to reward the already-decided.

4. Prereqs & time box

You can train a scikit-learn classifier and use a pandas DataFrame; you’ve met the idea of a probability prediction. No causal-inference background is assumed — the segment framework above is the mental model you need.

Where to write and run code

Two ways to run this project — pick either:

• In your browser (no install). If the project page shows a ▶ Run button, click it to run Part A right here — nothing to set up. Part A uses pandas, numpy, and scikit-learn, which run in the browser.
• On your computer. The one-time setup — installing Python, downloading the code from the course site (no GitHub, no git), opening a terminal, creating a virtual environment, installing the requirements, and running Part A step by step — is covered once in Start Here. Come back here once python -c "import sklearn" runs without error. Part B (writing your own solution) needs this local setup.

Time box

• Part A — Guided Build: 60–90 minutes. Hard cap.
• Part B — Your Turn: 2–4 hours. Hard cap.

5. Setup & data

The data is synthetic and generated locally — see the banner above and SOURCES.md for full provenance. One command creates both files:

python data/make_data.py

That writes two CSVs into data/ (git-ignored; regenerate anytime, byte-for-byte identical because the seed is fixed):

• Part A — customers_a.csv (4,000 customers) and labels_a.csv (the true per-customer uplift — Part A may check its work against it).
• Part B — customers_b.csv (10,000 customers). No truth file — the grader has it.

Each row is customer_id, x1, x2, x3, treatment (1 = got the offer), converted (0/1). The treatment was assigned at random, which is what makes uplift estimable.

6. Part A — Guided Build

You’ll work customers_a.csv. Run the whole thing, or (better) one # %% cell at a time — see Start Here:

python part_a/target_uplift.py

Prefer the browser? If this page has a ▶ Run button, click it instead of the command above — Part A runs in your browser with nothing to install. The STEP output and every checkpoint below are identical either way.

The full, commented script is in part_a/target_uplift.py. The spine:

Step 1 — The ATE. Treated conversion minus control conversion.

Checkpoint: a small positive ATE (~+2%). It’s one average across everyone — it can’t say who to target, and it hides that some customers are put off by the offer.

Step 2 — Response targeting (the trap). Model who converts; target the top 30% by response.

Checkpoint: captured-uplift fraction ≈ 0.17 — most of the achievable uplift left on the table. Response targeting funds sure-things who’d convert anyway.

Step 3 — Estimate uplift (T-learner). Fit a model on the treated rows and one on the control rows; per-customer uplift is the difference of their predicted conversion probabilities.

Checkpoint: predicted uplift ranges from strongly positive to negative — the model can see sleeping-dogs that a response model can’t.

Step 4 — Uplift targeting. Target the top 30% by uplift.

Checkpoint: captured-uplift fraction ≈ 0.87 — about 5× the response rule, same budget.

Step 5 — Who gets targeted. Compare the two strategies’ targeted groups.

Checkpoint: the uplift group’s mean true uplift (+0.36) dwarfs the response group’s (+0.07). Writes part_a/targeting_a.csv.

Why this and not that

• Why a randomized split makes uplift estimable. With random treatment, treated and control customers are comparable, so “treated conversion − control conversion” for similar x is a fair estimate of the offer’s effect. Without randomization you’d be measuring who chose the offer, not what it did.
• Why a T-learner (two models), not one model with treatment as a feature. A single model can bury a small treatment effect under the much larger differences between customers. Fitting the treated and control worlds separately lets each capture its own response surface; the difference isolates the effect. (An S-learner is the one-model alternative — a good thing to compare in the stretch.)
• Why captured-uplift, not accuracy or AUC. You never observe an individual’s uplift, so you can’t score it like a label. What you can score is the decision: of the uplift actually available in a top-30% budget, how much did your ranking capture? That’s the business metric.

Functions you’ll meet (and where to read more)

Function	What it does here	Docs
boolean mask X[t == 1]	Split into treated vs control to fit two models	numpy indexing
DecisionTreeClassifier().fit	Model conversion within the treated / control worlds	DecisionTree
predict_proba(X)[:, 1]	Predicted conversion probability (for the difference)	predict_proba
np.argsort(-score)[:k]	Take the top-k customers by score	argsort
np.sort(truth)[::-1]	Build the oracle’s best-possible top-k (for the fraction)	sort

If something looks off

• Response targeting captured only ~0.17 — did I do something wrong? No — that low number is the lesson. Targeting the likeliest converters funds “sure things” (high response, zero uplift) and can’t even see “sleeping dogs” the offer drives away. Score by uplift — treated-minus-control, Step 4 — and the same 30% budget jumps to ~0.87. Targeting by predicted response looks reasonable and is exactly the trap this project is about.
• A checkpoint number doesn’t match. The data is seeded, so the numbers are exact. Re-run python data/make_data.py (you may have skipped or edited it), then re-run Part A.
• Setup problems (Python not found, pip, FileNotFoundError, re-activating the venv): see the troubleshooting list in Start Here.

7. Part B — Your Turn

Part B is the optional “your turn” half — a guided fill-in-the-blank, not a blank page. You get a working starter (part_b/starter.py) that already handles the easy version; your job is to finish the # TODO it marks out. More open than Part A’s step-by-step, but you are never staring at an empty file. If you finished Part A and want to prove the skill, this is where it happens. New to Python? It’s completely fine to stop after Part A and come back later.

Same core skill, more customers, and now blind and scored. Score every one of the 10,000 customers in customers_b.csv by predicted uplift so a top-30% campaign captures the most incremental conversions. There’s no truth file — the grader has it. The starter (part_b/starter.py) scores by response and captures ~0.10, below the floor.

The transfer: in Part A you could check both strategies against the true uplift. Here you can’t peek — you have to trust the T-learner and hand in scores. (Watch the scale, too: recovering the truth to grade regenerates the dataset, so if you ever call the ground-truth helper, call it once, not per row.)

Acceptance criteria (checkable)

Your solution writes into part_b/out/:

• scores.csv — one row per customer: customer_id, uplift_score (higher = target first).
• report.json — with keys customers, achieved_metric (you can leave achieved_metric null; the grader scores you).

Then:

python part_b/starter.py
pytest part_b/test_solution.py -q

If pytest reports everything as skipped, it just means the output files don’t exist yet — run python part_b/starter.py first (once your solution fills in the TODOs), then re-run.

You’re done when the tests pass: your top-30% captures ≥ 0.65 of the achievable uplift (the reference reaches ~0.91). Response targeting captures ~0.10 and fails.

Constraints — what must be true of your solution, not how to get there

• Use the treatment/control split — score by an estimated effect, not by predicted conversion.
• Score every customer.
• Don’t target by response probability, and don’t ignore that some uplift is negative (targeting a sleeping-dog is worse than skipping them).

Hints

Hint 1 — split the world in two

Fit one model on the treated rows (treatment == 1) and one on the control rows (treatment == 0). Each predicts conversion probability in its world.

Hint 2 — uplift is the difference

For every customer, predict with BOTH models and subtract: P(convert | treated) − P(convert | control). That difference is the uplift score.

Hint 3 — sanity-check the ranking (still not the code)

Your top-scored customers should be persuadables, not the highest-converting customers. If your ranking matches a plain response model’s, you haven’t taken the difference.

8. Self-check

You don’t need the answer key. You’re done when all of these hold:

• pytest reports captured-uplift ≥ 0.65 (aim for ~0.9 with the T-learner).
• Your top-scored customers are not the same as a response model’s top scorers.
• Re-running from python data/make_data.py reproduces the same numbers — the data is seeded.

And the tell-tale: if your uplift ranking is basically your response ranking, you scored the level, not the effect — and you’re about to pay the people who’d have bought anyway.

9. Stretch

• S-learner vs T-learner. Fit a single model with treatment as a feature and take predict(treated=1) − predict(treated=0). Compare its captured-uplift to the T-learner’s.
• The Qini curve. Plot captured uplift as you increase the targeted fraction from 0 to 100%, for uplift vs response vs random. The area between the curves is the Qini coefficient — a fuller picture than one 30% number.
• (Genuinely hard) Confounded data. Break randomization (make treatment depend on x1) and show the naive T-learner now estimates uplift with bias — then correct it with propensity weighting. This is the leap from “A/B test” to observational causal inference.

10. Ship it

Put this in a portfolio repo and it counts. In that repo’s README:

• State the finding in one sentence a CMO gets: “targeting likely converters wasted ~80% of the budget’s potential; targeting by uplift captured ~5× the incremental conversions.”
• Show the captured-uplift of response vs uplift targeting (and, if you did the stretch, the Qini curve).
• Include the T-learner code and one paragraph on the four segments — especially sleeping-dogs. That paragraph is what signals you understand causal targeting, not just classification.

Keep make_data.py and the tests in the repo so anyone can reproduce your numbers from zero.

11. Sources

Rendered from SOURCES.md.

Field	Value
Dataset	Synthetic randomized-campaign data with known per-customer uplift (Part A 4,000, Part B 10,000 customers)
Type	Synthetic
Generated by	data/make_data.py (this repo), seeds 20260718 (A) and 20260719 (B)
Source URL	Not applicable — generated, not fetched
License	Not applicable — synthetic, no third-party rights
Access / generation date	2026-07-18
Redistribution	Permitted — synthetic; output regenerated from the seeded script, never committed
robots.txt checked	Not applicable — no scraping
ToS URL	Not applicable — no external source

Synthetic data teaches the mechanic; it does not prove a finding about the real world. The segments and effects were set on purpose — recovering them demonstrates the technique, not a claim about any real campaign. Real uplift work needs a genuine experiment and fuller validation (see SOURCES.md).

Next up

That’s the top of the AI & Machine Learning track. Browse all courses → to pick your next build.