Skip to content

feat: stamp the current UTC date and a no-denial rule into every delegate prompt - #205

Merged
antonbabenko merged 3 commits into
masterfrom
feat/date-grounding
Sep 24, 2026
Merged

antonbabenko merged 3 commits into
masterfrom
feat/date-grounding

Conversation

@antonbabenko

@antonbabenko antonbabenko commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

Summary

Delegates trained months ago call newer model, tool, or version names "hallucinated" and file them as critical issues. That blocks consensus rounds with false positives. #204 proposed a standalone "mandatory temporal grounding" skill; this PR does the same job in code instead.

  • New core/grounding.js: groundingNote() returns today's UTC date plus one rule. A delegate must not call something non-existent only because it does not recognize it; it marks the claim [unverified] and says what would confirm it.
  • Added in all 4 prompt builders: Codex runOnce, Gemini buildAgyArgs, Grok and OpenRouter buildInitialTurns. Every path gets it: unified server tools, consensus peers, arbiter passes, and the standalone /ask-* bridges. core/orchestrate.js is unchanged.
  • Host side: looking up facts stays with the orchestrator. AGENTS.md gets a "Time-sensitive questions" paragraph (regenerated into POWER.md and SKILL.md). Every /ask-* command, /consensus, and the CONTEXT section of rules/delegation-format.md get a "Time-sensitive facts" step. The host verifies with its own retrieval and puts the facts in the prompt with an as-of date and source.
  • Unchanged: the expert personas, no new skill, no config key, no vendor tool names.

Why code and not prose: Grok and OpenRouter run with no tools, so "run date -u / search" can never be followed. Persona text is also paid on every call, times panel size, times rounds. The design came from /ask-all, where all 5 models agreed.

How to verify (before / after)

The test fact is public: "OpenAI GPT-6 Astra is now generally available on Amazon Bedrock", posted Sep 8, 2026. That is after most delegates' training cutoff, which makes it the typical false "hallucinated" finding.

  • Before: the installed plugin (v3.16.1 or older).
  • After: this branch, claude --plugin-dir <checkout of feat/date-grounding>, or the released version after /plugin marketplace update antonbabenko + /reload-plugins.

Replies vary between runs; judge them by the pattern, not the exact words.

1. The delegate sees today's date (code change)

/deliberation:ask-grok Forward this question verbatim and add no context of your own: "What is today's date? Reply with the date and where you got it."
  • Before: Grok guesses a date near its training cutoff, or says it cannot know.
  • After: today's UTC date, taken from its instructions.

To check all providers at once, run the same line with /deliberation:ask-all.

2. The delegate does not deny a model it does not know (code change)

/deliberation:ask-all Forward this question verbatim and add no context or facts of your own: "Our Terraform config pins the Amazon Bedrock model 'OpenAI GPT-6 Astra'. Review that choice from your own knowledge only: is this a real, available Bedrock model?"
  • Before: one or more delegates call the model fictional, hallucinated, or non-existent, and treat it as a blocking issue.
  • After: delegates say they cannot confirm it, mark it [unverified], and name what would confirm it (for example the Bedrock model catalog or AWS What's New). None of them claims it does not exist.

3. The host looks the fact up first (prose change)

/deliberation:ask-all Our Terraform config pins the Amazon Bedrock model 'OpenAI GPT-6 Astra'. Is this a real, available Bedrock model?
  • After: Claude checks it with its own tools (web or AWS docs) before dispatching, and the delegation prompt carries the fact with its source and date: the announcement above, Sep 8, 2026. The delegates then answer from that fact.
  • Before: Claude may forward the question bare, which gives the test 2 behavior.

Behavior changes

  • The OpenRouter bridge now always sends a system turn, even without developer instructions. Expert calls already sent one, because they carry a persona.
  • The Codex prompt now always contains the --- separator; the note sits above it.
  • The Gemini prompt order is instructions, note, read-only guard, question.
  • *-reply continuations keep the date from the start of the thread.

Tests

  • New test/core-grounding.test.js:
    • GR1: UTC date for a fixed clock.
    • GR2-GR5: each builder includes the note.
    • GR6: codex's echoed, stamped prompt plus a 429 still classifies as rate-limit.
  • Updated exact-shape tests grok.test.js G1/G10/GF7 and bridge.test.js S1/S2. The date is pinned by regex, so they cannot flake at midnight.
  • npm run check: 881/881.

Pre-PR review

/consensus (GPT, Gemini, Grok, DeepSeek, Kimi; code-reviewer) converged in 2 rounds.

Accepted: the note said "your own tools", which contradicted Grok's no-tools note. It now says "tools you actually have (if any)".

Dismissed, with reasons:

  • gemini-reply is not re-stamped. It resumes an agy conversation whose first prompt already carries the note, so there is one date per thread, the same as grok-reply.
  • OpenRouter always sending a system role could break some models. Expert calls already sent a persona system turn; only bare standalone calls change.
  • The always-present --- could confuse codex echo consumption. classifyCodex consumes the sent prompt line by line, not by separator, and GR6 covers it.
  • Arbiter or other paths may skip the note. Every peer and arbiter call goes through an adapter's ask(), and the 4 builders are the only places a prompt is assembled.
  • Token cost and cache interaction. The note is about 70 tokens by design, and the cache effect is documented in TECHNICAL.md.

Deferred: an eval tracking whether each model actually uses the [unverified] tag. That needs live models, outside the unit suite.

Refs #204.

…gate prompt

Delegates trained months ago call newer model or tool names hallucinated and
file false critical issues. core/grounding.js adds today's UTC date plus a
one-sentence rule (unrecognized means [unverified], never non-existent) in all
four prompt builders, so every path gets it, standalone bridges included.
Looking facts up stays with the host: AGENTS.md and the ask-all/consensus
commands tell it to verify and inline them with an as-of date and source.

Replaces the temporal-grounding skill proposed in #204.
@antonbabenko

Copy link
Copy Markdown
Owner Author

Before / after check

Same two prompts, same day (2026-09-24), same models: codex (default), gemini-3.8-flash-high, grok-4.7, deepseek-v4.1-flash, kimi-k3.
Before = installed v3.16.1 (47cab9a). After = this branch.

1. "What is today's date? Reply with the date and where you got it." (/ask-grok, no added context)

Grok reply
Before "I don't have today's date. Nothing in this message supplies a current date, and I have no clock, tools, or external source to check."
After "2026-09-24 (UTC). I got it from the runtime context in this message: 'Current date (UTC): 2026-09-24.'"

The quoted line exists only in core/grounding.js on this branch, so the after run went through the new code.

2. "Our Terraform config pins the Amazon Bedrock model 'OpenAI GPT-6 Astra'. Is this a real, available Bedrock model?" (/ask-all, researcher)

The model went GA on Bedrock in September 2026 (AWS What's New).

Verdict
Before 1 of 5 said real (codex, the only delegate that searched). gemini, grok, deepseek and kimi said "not real" / "fabricated", tagging every claim [unverified] and still denying.
After 5 of 5 said real and GA, none called it fabricated.

Before, the existing [unverified] rule did not stop a denial: the four tool-less delegates tagged their claims and still concluded "not real". In a /consensus review that is four false critical issues against a correct config.

Caveat: the after run also used the new host step (the orchestrator looked the fact up and put it in the prompt with an as-of date and source), so run 2 shows the combined effect of both halves, not the server-side no-denial rule alone.

@antonbabenko
antonbabenko merged commit 7e9262a into master Sep 24, 2026
1 check passed
@antonbabenko
antonbabenko deleted the feat/date-grounding branch September 24, 2026 10:26
ivankorn added a commit to ivankorn/deliberation that referenced this pull request Sep 27, 2026
- Rebase onto origin/master and drop temporal grounding (addressed in antonbabenko#205)
- Drop host skills (ask-all, consensus) to narrow PR scope
- Keep built-in Gemini named 'gemini' and report model in model field (B1)
- Restore null guard in resolveModels for invalid records (B2)
- Drop google from models; built-in Gemini configured via providers.gemini (B3, B4, B15)
- Report invalid local records regardless of OpenRouter enabled state (B5)
- Enforce active panel dispatch in ask-one without matchModel fallback (B6)
- Exclude local models from auto-arbiter preference (B7)
- Key configLevers strictly by alias and identify local model levers in analyze (B8)
- Wire provider timeout defaults to delegate calls when unset (B9)
- Key configured delegate names as <provider>:<alias> to prevent slug collisions (B16)
- Guard against sending OPENROUTER_API_KEY to keyless local/custom endpoints (B13a, B13b, B14)
- Add entries for ollama and lmstudio to config.default.json and drop llmstudio alias
- Add comprehensive regression test suite in test/local-providers.test.js (B12)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant