Skip to content

ge-hch.5.15.20: integrate director evaluation in inkrunner #167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SorraTheOrc merged 2 commits into from
Jan 18, 2026
Merged

ge-hch.5.15.20: integrate director evaluation in inkrunner #167

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0b14acf
ge-hch.5.15.20: integrate director evaluation in inkrunner
SorraTheOrc Jan 18, 2026
af519b7
bd sync: 2026-01-17 19:04:14
SorraTheOrc Jan 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .beads/issues.jsonl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@
{"id":"ge-hch.5.15.18","title":"Implement: Player Preference Tracker","description":"Create web/demo/js/player-preference.js for tracking preferences.\n\n## Acceptance Criteria\n- [ ] Records { branchType, accepted, timestamp } events\n- [ ] Computes preference score per branch type (0.0-1.0)\n- [ ] Persists in localStorage key ge-hch.ai-preferences\n- [ ] Cold-start returns 0.5 for all types\n- [ ] getPreference(branchType) and recordOutcome(branchType, accepted) APIs\n\n## Related Feature\nge-hch.5.15.5 (Player Preference Tracker)","status":"open","priority":2,"issue_type":"task","assignee":"Patch","created_at":"2026-01-16T15:03:51.748963075-08:00","created_by":"rgardler","updated_at":"2026-01-16T15:03:51.748963075-08:00","dependencies":[{"issue_id":"ge-hch.5.15.18","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:51.750476216-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.19","title":"Tests: Player Preference Tracker","description":"Unit tests for player preference tracking.\n\n## Acceptance Criteria\n- [ ] Test: 3 accepts + 1 reject of dialogue yields preference \u003e 0.6\n- [ ] Test: 0 history yields preference = 0.5\n- [ ] Test: 100+ events still performant (\u003c10ms)\n- [ ] Test: localStorage persistence works\n\n## Related Feature\nge-hch.5.15.5 (Player Preference Tracker)","status":"open","priority":2,"issue_type":"task","assignee":"Probe","created_at":"2026-01-16T15:03:51.807524607-08:00","created_by":"rgardler","updated_at":"2026-01-16T15:03:51.807524607-08:00","dependencies":[{"issue_id":"ge-hch.5.15.19","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:51.808421437-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.2","title":"Return-Path Feasibility Checker","description":"Validate that the AI's proposed return knot exists in the story to prevent dead-ends.\n\n## Player Experience Change\nPlayers will never be stranded in an AI branch with no way back. If the AI proposes a non-existent return path, the choice is silently rejected.\n\n## Acceptance Criteria\n- [ ] Returns `{ feasible: boolean, reason: string, confidence: number }`\n- [ ] `feasible=true` if `return_path` knot exists in story (confidence=0.9)\n- [ ] `feasible=false` if knot does not exist (confidence=0.0, reason='Return path knot does not exist')\n- [ ] Completes in \u003c50ms\n- [ ] Unit test: `return_path: 'campfire'` passes (knot exists in demo.ink)\n- [ ] Unit test: `return_path: 'nonexistent_knot_xyz'` fails\n- [ ] Integration test: Director rejects proposal with invalid return_path\n\n## Minimal Implementation\n- Create `checkReturnPath(returnPath, story)` function\n- Extract knot names from `story.mainContentContainer._namedContent`\n- Simple existence check\n\n## Dependencies\n- ge-hch.5.15.1 (Decision Flow Engine)\n\n## Deliverables\n- Return-path checker in director.js\n- Unit tests with valid/invalid return paths","status":"closed","priority":1,"issue_type":"feature","created_at":"2026-01-16T15:01:40.467783504-08:00","created_by":"rgardler","updated_at":"2026-01-17T10:51:48.6478971-08:00","closed_at":"2026-01-17T10:51:48.6478971-08:00","close_reason":"Return-path checker implemented, tested and integrated into Director","dependencies":[{"issue_id":"ge-hch.5.15.2","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:01:40.469157452-08:00","created_by":"rgardler"},{"issue_id":"ge-hch.5.15.2","depends_on_id":"ge-hch.5.15.1","type":"blocks","created_at":"2026-01-16T15:04:32.206416228-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.20","title":"Implement: Director Integration","description":"Modify inkrunner.js to use Director for AI choice injection.\n\n## Acceptance Criteria\n- [ ] generateAIChoice() calls director.evaluate() before injecting\n- [ ] AI choice injected only if decision === approve\n- [ ] Silent skip on reject (no error, no AI choice)\n- [ ] Loading indicator shows Evaluating AI choice during evaluation\n- [ ] Logs rejection reasons to console\n\n## Implementation Notes\n- Modify generateAIChoice() in web/demo/js/inkrunner.js\n- Import director.js module\n- Handle both sync and async evaluation\n\n## Related Feature\nge-hch.5.15.6 (Director Integration)","status":"open","priority":1,"issue_type":"task","assignee":"Patch","created_at":"2026-01-16T15:03:59.856948737-08:00","created_by":"rgardler","updated_at":"2026-01-16T15:03:59.856948737-08:00","dependencies":[{"issue_id":"ge-hch.5.15.20","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:59.857773656-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.20","title":"Implement: Director Integration","description":"Modify inkrunner.js to use Director for AI choice injection.\n\n## Acceptance Criteria\n- [ ] generateAIChoice() calls director.evaluate() before injecting\n- [ ] AI choice injected only if decision === approve\n- [ ] Silent skip on reject (no error, no AI choice)\n- [ ] Loading indicator shows Evaluating AI choice during evaluation\n- [ ] Logs rejection reasons to console\n\n## Implementation Notes\n- Modify generateAIChoice() in web/demo/js/inkrunner.js\n- Import director.js module\n- Handle both sync and async evaluation\n\n## Related Feature\nge-hch.5.15.6 (Director Integration)","status":"in_progress","priority":1,"issue_type":"task","assignee":"@Patch","created_at":"2026-01-16T15:03:59.856948737-08:00","created_by":"rgardler","updated_at":"2026-01-17T19:04:10.502918195-08:00","external_ref":"https://github.com/TheWizardsCode/GEngine/pull/167","labels":["Status: PR Created"],"dependencies":[{"issue_id":"ge-hch.5.15.20","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:59.857773656-08:00","created_by":"rgardler"}],"comments":[{"id":207,"issue_id":"ge-hch.5.15.20","author":"rgardler","text":"Integrated Director evaluate into addAIChoice with sync/async support; added require fallback for Director in Node. Loading indicator now shows 'Evaluating AI choice...' during Director evaluation. Rejects are silent with console reason; approvals inject AI choice. Tests updated for sync evaluate; targeted jest run: npx jest tests/unit/inkrunner.test.js tests/unit/director.test.js --runInBand (pass).","created_at":"2026-01-18T03:03:41Z"}]}
{"id":"ge-hch.5.15.21","title":"Tests: Director Integration","description":"Integration tests for Director-governed injection.\n\n## Acceptance Criteria\n- [ ] Playthrough: complete demo.ink with mix of accepted/rejected\n- [ ] Playthrough: no runtime errors when all branches rejected\n- [ ] Test: mocked Director approve leads to AI choice shown\n- [ ] Test: mocked Director reject leads to no AI choice\n\n## Related Feature\nge-hch.5.15.6 (Director Integration)","status":"open","priority":1,"issue_type":"task","assignee":"Probe","created_at":"2026-01-16T15:03:59.901304347-08:00","created_by":"rgardler","updated_at":"2026-01-16T15:03:59.901304347-08:00","dependencies":[{"issue_id":"ge-hch.5.15.21","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:03:59.902099293-08:00","created_by":"rgardler"}]}
{"id":"ge-hch.5.15.22","title":"Implement: Director Config UI","description":"Extend AI Settings modal with Director configuration.\n\n## Acceptance Criteria\n- [ ] Risk threshold slider (0.1-0.8, default 0.4) in settings\n- [ ] Enable Director checkbox (default checked)\n- [ ] Settings persist in localStorage\n- [ ] Changes take effect on next choice (no reload)\n- [ ] Invalid values clamped to valid range\n\n## Implementation Notes\n- Extend renderSettingsPanel() in api-key-manager.js\n- Add Director Settings section\n- Bind to settings.directorRiskThreshold and settings.directorEnabled\n\n## Related Feature\nge-hch.5.15.7 (Director Configuration UI)","status":"closed","priority":2,"issue_type":"task","assignee":"@OpenCode","created_at":"2026-01-16T15:04:07.947028051-08:00","created_by":"rgardler","updated_at":"2026-01-16T22:07:33.585947557-08:00","closed_at":"2026-01-16T22:07:33.585947557-08:00","close_reason":"Completed","dependencies":[{"issue_id":"ge-hch.5.15.22","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:04:07.948288344-08:00","created_by":"rgardler"}],"comments":[{"id":195,"issue_id":"ge-hch.5.15.22","author":"rgardler","text":"Added Director controls to AI Settings (enable toggle + risk threshold slider with clamping + persistence). Settings feed inkrunner.js to govern Director usage. No UI yet for telemetry, deferred to ge-hch.5.15.24.","created_at":"2026-01-17T06:07:17Z"},{"id":196,"issue_id":"ge-hch.5.15.22","author":"rgardler","text":"Settings panel now hides the entire AI config when AI choices are disabled, plus Director controls collapse when either AI or Director toggles are off. Keeps UI compact and avoids misleading controls.","created_at":"2026-01-17T06:11:54Z"}]}
{"id":"ge-hch.5.15.23","title":"Tests: Director Config UI","description":"UI tests for Director configuration.\n\n## Acceptance Criteria\n- [ ] Test: changing threshold updates getSettings().directorRiskThreshold\n- [ ] Test: invalid threshold (2.0) clamped to valid range\n- [ ] Test: high threshold (0.8) accepts more proposals than low (0.2)\n- [ ] Test: disabling Director falls back to naive injection\n\n## Related Feature\nge-hch.5.15.7 (Director Configuration UI)","status":"closed","priority":2,"issue_type":"task","assignee":"@OpenCode","created_at":"2026-01-16T15:04:07.991961562-08:00","created_by":"rgardler","updated_at":"2026-01-17T01:40:45.906548983-08:00","closed_at":"2026-01-17T01:40:45.906582258-08:00","dependencies":[{"issue_id":"ge-hch.5.15.23","depends_on_id":"ge-hch.5.15","type":"parent-child","created_at":"2026-01-16T15:04:07.992789597-08:00","created_by":"rgardler"}],"comments":[{"id":197,"issue_id":"ge-hch.5.15.23","author":"rgardler","text":"Added deterministic mock proposal hook to inkrunner and updated Playwright tests to use mock proposals for Director acceptance tests. This avoids hitting external LLM endpoints and makes approval counts deterministic. Files changed: web/demo/js/inkrunner.js, tests/demo.telemetry.spec.ts. (Assignee: @OpenCode)","created_at":"2026-01-17T07:29:10Z"},{"id":198,"issue_id":"ge-hch.5.15.23","author":"rgardler","text":"Completed Director UI tests and deterministic mock hooks. Added/updated: web/demo/js/inkrunner.js, web/demo/js/director.js, tests/demo.telemetry.spec.ts, tests/unit/director.test.js. Ran unit tests (npm run test:unit) and Playwright demo tests locally; both passed. PR https://github.com/TheWizardsCode/GEngine/pull/156 merged. Deleting local branch feature/ge-hch.5.15-director and remote counterpart after merge. Closing per acceptance criteria: threshold updates, clamping, high/low threshold behavior, and Director disable fallback are covered by tests. (Assignee: @OpenCode)","created_at":"2026-01-17T09:40:44Z"}]}
Expand Down
4 changes: 2 additions & 2 deletions tests/unit/inkrunner.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -380,7 +380,7 @@ describe('inkrunner AI integration', () => {
metadata: { confidence_score: 0.9 }
};

const evaluateMock = jest.fn(async (_proposal, _ctx, options) => {
const evaluateMock = jest.fn((_proposal, _ctx, options) => {
expect(options).toHaveProperty('riskThreshold', 0.4);
return { decision: 'approve', reason: 'ok', riskScore: 0.2, latencyMs: 15 };
});
Expand Down Expand Up @@ -425,7 +425,7 @@ describe('inkrunner AI integration', () => {
};

window.Director = {
evaluate: jest.fn(async () => ({ decision: 'reject', reason: 'too risky', latencyMs: 7, riskScore: 0.9 }))
evaluate: jest.fn(() => ({ decision: 'reject', reason: 'too risky', latencyMs: 7, riskScore: 0.9 }))
};

const result = await inkrunner.addAIChoice({ forceDirectorEnabled: true, mockProposalOverride: proposal });
Expand Down
116 changes: 66 additions & 50 deletions web/demo/js/inkrunner.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,17 @@
* @type {HTMLElement|null}
*/
let loadingIndicator = null;

function getDirector() {
if (typeof window !== 'undefined' && window.Director) {
return window.Director;
}
try {
return require('./director.js');
} catch (e) {
return null;
}
}

/**
* Creates and returns the loading indicator element
Expand Down Expand Up @@ -559,58 +570,63 @@

const writerStart = (typeof performance !== 'undefined' && performance.now) ? performance.now() : Date.now();

try {
const proposal = mockProposalOverride
? normalizeMockProposal(mockProposalOverride)
: (useMockProposal ? getMockProposalIfAvailable() : await generateAIProposal());
try {
const proposal = mockProposalOverride
? normalizeMockProposal(mockProposalOverride)
: (useMockProposal ? getMockProposalIfAvailable() : await generateAIProposal());

hideLoadingIndicator();

hideLoadingIndicator();


if (!proposal) {
return 'no_proposal';
}

const writerMs = Math.max(
0,
((typeof performance !== 'undefined' && performance.now) ? performance.now() : Date.now()) - writerStart
);

currentAIProposal = proposal;

let directorResult = null;
const director = directorEnabled ? getDirector() : null;
if (director && typeof director.evaluate === 'function') {
const indicator = createLoadingIndicator();
const textEl = indicator.querySelector('.ai-loading-text');
if (textEl) textEl.textContent = 'Evaluating AI choice...';
indicator.style.display = 'flex';
choicesEl.appendChild(indicator);

try {
const maybePromise = director.evaluate(proposal, { story }, { riskThreshold });
directorResult = (maybePromise && typeof maybePromise.then === 'function')
? await maybePromise
: maybePromise;
} catch (e) {
console.warn('[inkrunner] Director evaluation failed, skipping AI choice', e);
hideLoadingIndicator();
return;
}

hideLoadingIndicator();

const directorMs = (directorResult && typeof directorResult.latencyMs === 'number') ? directorResult.latencyMs : 0;
const totalMs = writerMs + directorMs;
logTelemetry('ai_evaluation', {
proposal_id: proposal.id,
decision: directorResult && directorResult.decision,
writerMs,
directorMs,
totalMs
});

if (!directorResult || directorResult.decision !== 'approve') {
console.log('[inkrunner] Director rejected AI proposal:', directorResult && directorResult.reason);
return 'rejected';
}
}

if (!proposal) {
return 'no_proposal';
}

const writerMs = Math.max(
0,
((typeof performance !== 'undefined' && performance.now) ? performance.now() : Date.now()) - writerStart
);

currentAIProposal = proposal;

let directorResult = null;
if (directorEnabled && window.Director && typeof window.Director.evaluate === 'function') {
const indicator = createLoadingIndicator();
const textEl = indicator.querySelector('.ai-loading-text');
if (textEl) textEl.textContent = 'Evaluating AI choice...';
indicator.style.display = 'flex';
choicesEl.appendChild(indicator);

try {
directorResult = await window.Director.evaluate(proposal, { story }, { riskThreshold });
} catch (e) {
console.warn('[inkrunner] Director evaluation failed, skipping AI choice', e);
hideLoadingIndicator();
return;
}

hideLoadingIndicator();

const directorMs = (directorResult && typeof directorResult.latencyMs === 'number') ? directorResult.latencyMs : 0;
const totalMs = writerMs + directorMs;
logTelemetry('ai_evaluation', {
proposal_id: proposal.id,
decision: directorResult.decision,
writerMs,
directorMs,
totalMs
});

if (directorResult.decision !== 'approve') {
console.log('[inkrunner] Director rejected AI proposal:', directorResult.reason);
return 'rejected';
}
}

const btn = document.createElement('button');
const styleClass = settings.aiChoiceStyle === 'normal' ? 'ai-choice-normal' : 'ai-choice';
Expand Down