Problem
Fairness is currently only a prompt instruction ("NEVER let name, gender, race… affect any score"). Instructed LLMs still exhibit measurable demographic score deltas, and we feed the model exactly the signals that carry those cues: the candidate's name (ResumeParsed.name, GitHubSignal.name), location (GitHubSignal.location), and company. _github_summary puts name= straight into the dossier. Telling the model to ignore what we hand it is not a control.
Proposal
Implement blind screening: strip personal-identity fields from a copy of the profile before building the LLM dossier. The original profile is untouched and still drives the report's Candidate: header — only the prompt is redacted.
Redact (demographic cues, not needed to score skill):
ResumeParsed.name → blank
GitHubSignal.name / .location / .company / .twitter_username → None
Preserve (professional signal, not demographic):
GitHubSignal.username, .blog, work-history company names, discovered.github, discovered.email.
Implementation notes
- New
_redact_for_prompt(profile) -> CandidateProfile in evaluator.py using model_copy(update=...).
- Call it as the first line of
_user_prompt.
- Note:
resume.text still contains the name/email inline; fully scrubbing free text is out of scope here (removes signal, high false-positive). This issue targets the structured fields, which is where _github_summary/_resume_structured_summary surface identity most cleanly. A follow-up can consider free-text redaction.
Acceptance criteria
Files
src/hiregauge/evaluator.py
tests/test_evaluation.py
Blocks: counterfactual fairness tests (separate issue). Related: fairness prompt in evaluator._FAIRNESS.
Problem
Fairness is currently only a prompt instruction ("NEVER let name, gender, race… affect any score"). Instructed LLMs still exhibit measurable demographic score deltas, and we feed the model exactly the signals that carry those cues: the candidate's name (
ResumeParsed.name,GitHubSignal.name), location (GitHubSignal.location), and company._github_summaryputsname=straight into the dossier. Telling the model to ignore what we hand it is not a control.Proposal
Implement blind screening: strip personal-identity fields from a copy of the profile before building the LLM dossier. The original profile is untouched and still drives the report's
Candidate:header — only the prompt is redacted.Redact (demographic cues, not needed to score skill):
ResumeParsed.name→ blankGitHubSignal.name/.location/.company/.twitter_username→NonePreserve (professional signal, not demographic):
GitHubSignal.username,.blog, work-history company names,discovered.github,discovered.email.Implementation notes
_redact_for_prompt(profile) -> CandidateProfileinevaluator.pyusingmodel_copy(update=...)._user_prompt.resume.textstill contains the name/email inline; fully scrubbing free text is out of scope here (removes signal, high false-positive). This issue targets the structured fields, which is where_github_summary/_resume_structured_summarysurface identity most cleanly. A follow-up can consider free-text redaction.Acceptance criteria
tests/test_evaluation.pyasserting the redaction.Files
src/hiregauge/evaluator.pytests/test_evaluation.pyBlocks: counterfactual fairness tests (separate issue). Related: fairness prompt in
evaluator._FAIRNESS.