Skip to content

ge-hch.5.15.15: add risk scorer tests #166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SorraTheOrc merged 2 commits into from
Jan 18, 2026
Merged

ge-hch.5.15.15: add risk scorer tests #166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
032d761
ge-hch.5.15.15: add risk scorer tests
SorraTheOrc Jan 18, 2026
b9e7220
bd sync: 2026-01-17 19:58:00
SorraTheOrc Jan 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .beads/issues.jsonl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@
{"id":"ge-hch.5.15.12","title":"Implement: Return-Path Checker","description":"Implement checkReturnPath function in director.js.\n\n## Acceptance Criteria\n- [ ] checkReturnPath(returnPath, story) returns { feasible, reason, confidence }\n- [ ] Extracts knot names from story.mainContentContainer._namedContent\n- [ ] Returns feasible=true, confidence=0.9 if knot exists\n- [ ] Returns feasible=false, confidence=0.0 if knot missing\n- [ ] Completes in under 50ms\n\n## Related Feature\nge-hch.5.15.2 (Return-Path Feasibility Checker)","status":"closed","priority":1,"issue_type":"task","assignee":"@Patch","created_at":"2026-01-16T15:03:27.53497903-08:00","created_by":"rgardler","updated_at":"2026-01-17T15:43:03.438701328-08:00","closed_at":"2026-01-17T15:43:03.438701328-08:00","close_reason":"PR #163 merged","external_ref":"https://github.com/TheWizardsCode/GEngine/pull/163","labels":["Status: PR Created"],"dependencies":[{"issue_id":"ge-hch.5.15.12","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:27.536217967-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.13","title":"Tests: Return-Path Checker","description":"Unit tests for return-path feasibility checking.\n\n## Acceptance Criteria\n- [ ] Test: return_path campfire passes (exists in demo.ink)\n- [ ] Test: return_path nonexistent_knot_xyz fails\n- [ ] Test: Director rejects proposal with invalid return_path\n- [ ] Test: completion time under 50ms\n\n## Related Feature\nge-hch.5.15.2 (Return-Path Feasibility Checker)","status":"closed","priority":1,"issue_type":"task","assignee":"@Patch","created_at":"2026-01-16T15:03:27.632238424-08:00","created_by":"rgardler","updated_at":"2026-01-17T15:49:07.647812072-08:00","closed_at":"2026-01-17T15:49:07.647812072-08:00","close_reason":"PR merged","external_ref":"https://github.com/TheWizardsCode/GEngine/pull/164","labels":["Status: PR Created"],"dependencies":[{"issue_id":"ge-hch.5.15.13","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:27.633077458-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.14","title":"Implement: Risk Scorer","description":"Implement computeRiskScore function with 3 active + 3 placeholder metrics.\n\n## Acceptance Criteria\n- [ ] computeRiskScore(proposal, context, config) returns weighted score 0.0-1.0\n- [ ] Implements proposal_confidence_risk: 1.0 - confidence_score\n- [ ] Implements narrative_pacing_risk: based on branch length vs phase\n- [ ] Implements return_path_confidence_risk: from checker\n- [ ] Placeholder metrics return 0.3 default\n- [ ] Configurable weights via config object\n\n## Related Feature\nge-hch.5.15.3 (Risk Scorer)","status":"closed","priority":1,"issue_type":"task","assignee":"@Patch","created_at":"2026-01-16T15:03:35.345472464-08:00","created_by":"rgardler","updated_at":"2026-01-17T16:03:05.124860848-08:00","closed_at":"2026-01-17T16:03:05.124860848-08:00","close_reason":"PR #165 merged","external_ref":"https://github.com/TheWizardsCode/GEngine/pull/165","labels":["Status: PR Created"],"dependencies":[{"issue_id":"ge-hch.5.15.14","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:35.346620476-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.15","title":"Tests: Risk Scorer","description":"Unit tests for risk scoring.\n\n## Acceptance Criteria\n- [ ] Test: high-confidence proposal (0.9) yields low risk (\u003c0.3)\n- [ ] Test: low-confidence proposal (0.3) yields high risk (\u003e0.5)\n- [ ] Test: long branch in exposition yields elevated pacing risk\n- [ ] Determinism: 10 calls with same input produce identical score\n\n## Related Feature\nge-hch.5.15.3 (Risk Scorer)","status":"open","priority":1,"issue_type":"task","assignee":"Probe","created_at":"2026-01-16T15:03:35.389561441-08:00","created_by":"rgardler","updated_at":"2026-01-16T15:03:35.389561441-08:00","dependencies":[{"issue_id":"ge-hch.5.15.15","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:35.390300568-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.15","title":"Tests: Risk Scorer","description":"Unit tests for risk scoring.\n\n## Acceptance Criteria\n- [ ] Test: high-confidence proposal (0.9) yields low risk (\u003c0.3)\n- [ ] Test: low-confidence proposal (0.3) yields high risk (\u003e0.5)\n- [ ] Test: long branch in exposition yields elevated pacing risk\n- [ ] Determinism: 10 calls with same input produce identical score\n\n## Related Feature\nge-hch.5.15.3 (Risk Scorer)","status":"in_progress","priority":1,"issue_type":"task","assignee":"@Patch","created_at":"2026-01-16T15:03:35.389561441-08:00","created_by":"rgardler","updated_at":"2026-01-17T18:55:36.904731352-08:00","dependencies":[{"issue_id":"ge-hch.5.15.15","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:35.390300568-08:00","created_by":"rgardler"}],"comments":[{"id":205,"issue_id":"ge-hch.5.15.15","author":"rgardler","text":"Added risk scorer test coverage: pacing risk elevated for long exposition, 10-run determinism, confidence-based thresholds align with acceptance; targeted test run via 'npm test --silent -- tests/unit/director.test.js' (fails globally due to start-server-and-test argument requirement, but direct jest run passes).","created_at":"2026-01-18T02:56:32Z"},{"id":206,"issue_id":"ge-hch.5.15.15","author":"rgardler","text":"Opened PR https://github.com/TheWizardsCode/GEngine/pull/166 (ge-hch.5.15.15: add risk scorer tests). Branch feature/ge-hch.5.15.15-risk-scorer-tests pushed. Tests: npm test --silent -- tests/unit/director.test.js (director suite passes; overall script exits with start-server-and-test arg error); npx jest tests/unit/director.test.js --runInBand (pass).","created_at":"2026-01-18T02:57:38Z"}]}
{"id":"ge-hch.5.15.16","title":"Implement: Embedding Service","description":"Create web/demo/js/embedding-service.js with transformers.js.\n\n## Acceptance Criteria\n- [ ] Loads Xenova/all-MiniLM-L6-v2 model via transformers.js\n- [ ] WebWorker wrapper for non-blocking inference\n- [ ] embed(text) returns embedding vector\n- [ ] similarity(vec1, vec2) returns cosine similarity\n- [ ] Lazy loading on first use\n- [ ] Graceful fallback if model fails\n\n## Related Feature\nge-hch.5.15.4 (Embedding Service)","status":"open","priority":2,"issue_type":"task","assignee":"Patch","created_at":"2026-01-16T15:03:41.761163209-08:00","created_by":"rgardler","updated_at":"2026-01-16T15:03:41.761163209-08:00","dependencies":[{"issue_id":"ge-hch.5.15.16","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:41.761957697-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.17","title":"Tests: Embedding Service","description":"Unit tests for embedding service.\n\n## Acceptance Criteria\n- [ ] Test: similarity(happy, joyful) \u003e 0.7\n- [ ] Test: similarity(happy, database) \u003c 0.4\n- [ ] Test: embed(null) returns null gracefully\n- [ ] Performance: first embed \u003c 3s, subsequent \u003c 100ms\n\n## Related Feature\nge-hch.5.15.4 (Embedding Service)","status":"open","priority":2,"issue_type":"task","assignee":"Probe","created_at":"2026-01-16T15:03:41.806727691-08:00","created_by":"rgardler","updated_at":"2026-01-16T15:03:41.806727691-08:00","dependencies":[{"issue_id":"ge-hch.5.15.17","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:41.807448395-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.18","title":"Implement: Player Preference Tracker","description":"Create web/demo/js/player-preference.js for tracking preferences.\n\n## Acceptance Criteria\n- [ ] Records { branchType, accepted, timestamp } events\n- [ ] Computes preference score per branch type (0.0-1.0)\n- [ ] Persists in localStorage key ge-hch.ai-preferences\n- [ ] Cold-start returns 0.5 for all types\n- [ ] getPreference(branchType) and recordOutcome(branchType, accepted) APIs\n\n## Related Feature\nge-hch.5.15.5 (Player Preference Tracker)","status":"open","priority":2,"issue_type":"task","assignee":"Patch","created_at":"2026-01-16T15:03:51.748963075-08:00","created_by":"rgardler","updated_at":"2026-01-16T15:03:51.748963075-08:00","dependencies":[{"issue_id":"ge-hch.5.15.18","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:51.750476216-08:00","created_by":"rgardler"}]}
Expand Down
21 changes: 11 additions & 10 deletions tests/unit/director.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,20 +133,21 @@ describe('Director core', () => {
expect(score).toBeGreaterThan(0.5);
});

it('scales pacing risk by phase expectations', () => {
const longText = 'long '.repeat(200); // > 500 chars
const proposal = { metadata: { confidence_score: 0.6 }, content: { text: longText } };
const ctxDefault = { returnPathCheck: { confidence: 0.9 }, phase: 'exposition' };
const defaultScore = Director.computeRiskScore(proposal, ctxDefault, {});
it('elevates pacing risk for long exposition branches', () => {
const shortProposal = { metadata: { confidence_score: 0.6 }, content: { text: 'concise text' } };
const longProposal = { metadata: { confidence_score: 0.6 }, content: { text: 'long '.repeat(220) } }; // ~1100 chars
const expositionCtx = { returnPathCheck: { confidence: 0.9 }, phase: 'exposition' };

const ctxClimax = { returnPathCheck: { confidence: 0.9 }, phase: 'climax' };
const climaxScore = Director.computeRiskScore(proposal, ctxClimax, {});
const shortScore = Director.computeRiskScore(shortProposal, expositionCtx, {});
const longScore = Director.computeRiskScore(longProposal, expositionCtx, {});
const climaxScore = Director.computeRiskScore(longProposal, { ...expositionCtx, phase: 'climax' }, {});

expect(defaultScore).toBeGreaterThan(climaxScore);
expect(defaultScore).toBeGreaterThan(0.35);
expect(longScore).toBeGreaterThan(shortScore);
expect(longScore).toBeGreaterThan(climaxScore);
expect(longScore).toBeGreaterThan(0.35);
});

it('is deterministic across repeated calls', () => {
it('is deterministic across repeated calls (10 runs)', () => {
const proposal = { metadata: { confidence_score: 0.7 }, content: { text: 'stable content' } };
const context = { returnPathCheck: { confidence: 0.9 } };
const scores = Array.from({ length: 10 }, () => Director.computeRiskScore(proposal, context, {}));
Expand Down