Skip to content

Update detector spec tests to match the #129 threshold tuning #151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
InfinitePortaldev wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Update detector spec tests to match the #129 threshold tuning #151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions src/__tests__/reasoning/detectors.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ function withFiller(claims: FixtureClaim[], edges: FixtureEdge[], padTo: number
// ── Load-bearing vibes ──────────────────────────────────────────────────────

describe('detectLoadBearingVibes', () => {
it('fires when vibes claim has ≥3 downstream', () => {
it('fires when vibes claim has ≥2 downstream (threshold tuned in #129)', () => {
const g = withFiller([
{ id: 'v1', basis: 'vibes', text: 'we need auth' },
{ id: 'd1', basis: 'deduction' },
Expand All @@ -80,10 +80,8 @@ describe('detectLoadBearingVibes', () => {
const g = withFiller([
{ id: 'v1', basis: 'vibes' },
{ id: 'd1', basis: 'deduction' },
{ id: 'd2', basis: 'deduction' },
], [
{ from: 'd1', to: 'v1', type: 'depends_on' },
{ from: 'd2', to: 'v1', type: 'supports' },
]);
expect(detectLoadBearingVibes(g)).toHaveLength(0);
});
Expand Down Expand Up @@ -152,7 +150,10 @@ describe('detectUnchallengedChain', () => {
// chainHasChallenge inspects edges where both endpoints are in the chain,
// so a challenge from outside doesn't count. To properly challenge, the
// question edge needs both endpoints in the chain.
// Reconfigure: have c2 questions c1 within the chain.
// Note the detector walks a chain from EVERY node, so a sub-chain that
// starts below the challenged pair (here c2→c3→c4) is itself a candidate
// once it meets the length minimum. Challenge each link so no qualifying
// sub-chain is left unchallenged.
const g2 = withFiller([
{ id: 'c1', basis: 'assumption' },
{ id: 'c2', basis: 'deduction' },
Expand All @@ -163,6 +164,7 @@ describe('detectUnchallengedChain', () => {
{ from: 'c2', to: 'c3', type: 'depends_on' },
{ from: 'c3', to: 'c4', type: 'depends_on' },
{ from: 'c2', to: 'c1', type: 'questions' },
{ from: 'c3', to: 'c2', type: 'questions' },
]);
expect(detectUnchallengedChain(g2)).toHaveLength(0);
});
Expand All @@ -171,10 +173,8 @@ describe('detectUnchallengedChain', () => {
const g = withFiller([
{ id: 'c1', basis: 'assumption' },
{ id: 'c2', basis: 'deduction' },
{ id: 'c3', basis: 'deduction' },
], [
{ from: 'c1', to: 'c2', type: 'depends_on' },
{ from: 'c2', to: 'c3', type: 'depends_on' },
]);
expect(detectUnchallengedChain(g)).toHaveLength(0);
});
Expand Down Expand Up @@ -238,7 +238,7 @@ describe('detectEchoChamber', () => {
// ── Bright: well-sourced load-bearer ────────────────────────────────────────

describe('detectWellSourcedLoadBearer', () => {
it('fires on research/empirical/deduction basis with ≥3 downstream', () => {
it('fires on research/empirical/deduction basis with ≥2 downstream (threshold tuned in #129)', () => {
const g = withFiller([
{ id: 'r1', basis: 'research', text: 'OWASP ranks XSS #3' },
{ id: 'd1', basis: 'deduction' }, { id: 'd2', basis: 'deduction' }, { id: 'd3', basis: 'deduction' },
Expand Down
14 changes: 11 additions & 3 deletions src/__tests__/reasoning/pipeline-per-detector.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,13 @@ describe('pipeline integration — all 6 detectors end-to-end', () => {
expect(t).toBe('unchallenged_chain');
});

it('fires echo_chamber', () => {
it('echo_chamber graphs currently surface as load_bearing_vibes', () => {
// Since the #129 threshold tuning, any graph that qualifies for
// echo_chamber (user vibes claim, ≥2 assistant supports) also qualifies
// for load_bearing_vibes (≥2 downstream, same edges), and selection has
// no per-type priority — so load_bearing_vibes wins every time. This
// test pins the current behavior; whether echo_chamber should be able
// to surface at all again is an open product question.
const t = seedAndRun({
claims: [
{ text: 'im sure', basis: 'vibes', speaker: 'user', confidence: 'medium', external_id: 'u' },
Expand All @@ -96,7 +102,7 @@ describe('pipeline integration — all 6 detectors end-to-end', () => {
{ from: 'a2', to: 'u', type: 'supports' },
],
});
expect(t).toBe('echo_chamber');
expect(t).toBe('load_bearing_vibes');
});

it('fires well_sourced_load_bearer', () => {
Expand Down Expand Up @@ -139,6 +145,9 @@ describe('pipeline integration — all 6 detectors end-to-end', () => {
});

it('fires grounded_premise_adopted', () => {
// Exactly one assistant support: enough for grounded_premise_adopted
// (min 1 since #129) while staying below well_sourced_load_bearer's
// downstream minimum, so the grounded finding is the one that surfaces.
const t = seedAndRun({
claims: [
{ text: 'OWASP XSS #3', basis: 'research', speaker: 'user', confidence: 'high', external_id: 'u' },
Expand All @@ -150,7 +159,6 @@ describe('pipeline integration — all 6 detectors end-to-end', () => {
],
edges: [
{ from: 'a', to: 'u', type: 'supports' },
{ from: 'b', to: 'u', type: 'depends_on' },
],
});
expect(t).toBe('grounded_premise_adopted');
Expand Down