Skip to content

[CRITICAL] Fix metadata extraction inconsistencies in scraped problems #54

Description

@AVPthegreat

Problem: Problem metadata (titles, descriptions, constraints, I/O samples) is inconsistent across scrapers. Some problems have complete data while others are missing critical fields, leading to incomplete or unusable problem bundles.\n\nImpact: Users receive ZIP files with partial or missing problem data, making the tool unreliable for actual use.\n\nRoot Cause:\n- Scraper selectors may be outdated or fragile\n- HTML structure variations not handled\n- Missing fallback logic for alternative layouts\n- No validation before including problems in output\n\nAcceptance Criteria:\n- ✅ Add metadata completeness validator before including problems in ZIP\n- ✅ Filter out problems missing title, statement, or I/O samples\n- ✅ Log which problems were skipped and why\n- ✅ Update scraper selectors for all platforms to handle common layout variants\n- ✅ Ensure at least 80% of scraped problems have complete metadata\n- ✅ Add unit tests validating metadata completeness for each scraper\n\nPriority: P0 - Blocks core functionality

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Critical prioritybugSomething isn't workingscraperPlatform scrapers

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions