Skip to content

GenderExtractor: compute the second-highest pronoun count order-independently - #859

Open
ManasBagul23 wants to merge 1 commit into
dbpedia:masterfrom
ManasBagul23:fix-genderextractor-order-dependent-secondcount
Open

ManasBagul23 wants to merge 1 commit into
dbpedia:masterfrom
ManasBagul23:fix-genderextractor-order-dependent-secondcount

Conversation

@ManasBagul23

@ManasBagul23 ManasBagul23 commented Oct 20, 2026 •

Copy link
Copy Markdown

The max/second-max tracking loop in GenderExtractor only updates secondCount when the running
max is dethroned by a later, larger count:

if (count > maxCount)
{
  secondCount = maxCount.toDouble
  maxCount = count
  maxGender = gender
}

So secondCount stays at its initial 0.0 whenever the eventual overall max happens to be the
first entry Scala's Map iteration produces for a page's pronoun counts — even when a genuinely
close second-place count exists. The confidence check maxCount/secondCount > minDifference then
divides by 0.0, evaluates to Infinity, and always passes, exactly when it should be guarding
against a near-tie.

Confirmed with the same counts in both iteration orders: male=10 processed first gives
secondCount=0.0 (ratio Infinity, always "confident"); female=9 processed first gives the
correct secondCount=9.0 (ratio 1.11). Same data, different outcome, purely from unspecified
map iteration order — this is a real, data-dependent nondeterminism, not a hypothetical.

Fix: sort the counts descending and read off the top two, so the result no longer depends on
iteration order.

Summary by CodeRabbit

  • Bug Fixes
    • Improved gender extraction accuracy by consistently selecting the most frequent gender and evaluating the runner-up count correctly.
    • Prevented confidence checks from passing incorrectly when a second gender count was not properly determined.
    • Empty gender data now reliably produces no gender result with a zero count.
    • Existing output conditions remain unchanged.

…endently

The max/second-max tracking loop only updated secondCount when the
running max was dethroned by a later, larger count:

    if (count > maxCount) {
      secondCount = maxCount.toDouble
      maxCount = count
      maxGender = gender
    }

So secondCount stays at its initial 0.0 whenever the eventual overall
max happens to be the first entry Scala's Map iteration produces for
this page's pronoun counts, even if a genuinely close second-place
count exists. The confidence check maxCount/secondCount > minDifference
then divides by 0.0, evaluates to Infinity, and always passes, exactly
when it should be guarding against a near-tie.

Confirmed with the same counts in both iteration orders: male=10
processed first gives secondCount=0.0 (ratio Infinity, always
"confident"), female=9 processed first gives the correct secondCount=9.0
(ratio 1.11). Same data, different outcome, purely from unspecified map
iteration order.

Sort the counts descending and read off the top two instead, so the
result no longer depends on iteration order.
Copilot AI balanced review requested due to automatic review settings October 20, 2026 06:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c7f5258e-b63d-4eef-b536-92e176eee86e
📥 Commits

Reviewing files that changed from the base of the PR and between b66cb8b and e1e3886.

📒 Files selected for processing: 1
  • core/src/main/scala/org/dbpedia/extraction/mappings/GenderExtractor.scala

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

📝 Walkthrough

GenderExtractor now sorts gender counts in descending order and uses the first two results to determine the top gender and confidence counts. When counts are empty, it uses no gender and a count of zero. Existing output conditions remain unchanged.

Changes: Gender count selection

Layer / File(s) Summary
Select top gender counts
core/src/main/scala/org/dbpedia/extraction/mappings/GenderExtractor.scala
The sorted results provide the top gender and count, and the second-highest count. If no second result exists, its count defaults to zero.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to e1e38

No actionable merge-blocking risk remains in this change.

Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: computing the second-highest pronoun count independently of iteration order.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches: 💡 1
🧪 Generate unit tests (beta)
  • Create a new PR
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants