Skip to content

README.md

Latest commit

 

History

History
169 lines (132 loc) · 4.86 KB

README.md

File metadata and controls

169 lines (132 loc) · 4.86 KB

nl-deterministic-context-budgeting

An educational local-first C# project showing how deterministic context budgeting keeps prompts within a fixed token budget while preserving required business context.

This version uses an e-commerce support scenario (late + damaged delivery refund resolution).

Overview

This repository demonstrates a practical deterministic prompt-packing flow:

  1. Reserve output tokens.
  2. Reserve fixed prompt overhead.
  3. Compute remaining context token budget.
  4. Sort candidate context blocks deterministically:
    • required blocks first
    • higher priority first
    • newer observation timestamp first
    • block ID as final tie-break
  5. Include blocks while budget remains.
  6. Exclude overflow blocks with explicit reasons.
  7. Compose the final prompt and optionally invoke a local OpenAI-compatible model.

If any required block cannot fit, the app marks the result as CanProceed = false and exits before model invocation.

What This Project Demonstrates

  • Deterministic context selection independent of input order
  • Explicit budget accounting (context window - reserved output - fixed prompt)
  • Required vs optional block handling with typed exclusion reasons
  • Heuristic token estimation for practical local budgeting
  • Prompt composition that preserves packed block order
  • Optional local model invocation via Semantic Kernel + OpenAI-compatible endpoint
  • Unit tests for packing stability, budget behavior, and prompt ordering

Prerequisites

  • .NET 10 SDK or later
  • Optional for model calls:
    • LM Studio OpenAI-compatible server (http://localhost:1234/v1)
    • or Ollama OpenAI-compatible endpoint (http://localhost:11434/v1)

Quick Start

From the project root:

dotnet test DeterministicContextBudgeting.slnx

Run budgeting + prompt preview only (no model call):

set DCB_App__EnableModelCall=false
dotnet run --project DeterministicContextBudgeting

Run with model invocation enabled:

set DCB_App__EnableModelCall=true
dotnet run --project DeterministicContextBudgeting

Configuration

Default settings are in DeterministicContextBudgeting/appsettings.json under App.

Example:

{
  "App": {
    "ModelContextWindowTokens": 850,
    "ReservedOutputTokens": 300,
    "FixedPromptTokens": 220,
    "Provider": "lmstudio",
    "BaseUrl": "http://localhost:1234/v1",
    "ApiKey": "not-needed",
    "ModelId": "deepseek/deepseek-r1-0528-qwen3-8b",
    "Temperature": 0.0,
    "EnableModelCall": true
  }
}

Environment variable overrides use prefix DCB_:

  • DCB_App__ModelContextWindowTokens
  • DCB_App__ReservedOutputTokens
  • DCB_App__FixedPromptTokens
  • DCB_App__Provider
  • DCB_App__BaseUrl
  • DCB_App__ApiKey
  • DCB_App__ModelId
  • DCB_App__Temperature
  • DCB_App__EnableModelCall

Ollama example:

set DCB_App__Provider=ollama
set DCB_App__BaseUrl=http://localhost:11434/v1
set DCB_App__ApiKey=ollama
set DCB_App__ModelId=mistral:7b
set DCB_App__EnableModelCall=true
dotnet run --project DeterministicContextBudgeting

How It Works

  1. EcommerceSupportScenario creates realistic support context blocks.
  2. ContextBudgeter estimates token cost and packs deterministically.
  3. ContextBudgetingResult records included/excluded blocks and proceed gating.
  4. PromptComposer builds the final prompt from included blocks.
  5. Program.cs prints budget summary and optionally calls the configured local model.

Project Structure

.
+-- DeterministicContextBudgeting.slnx
+-- DeterministicContextBudgeting/
|   +-- DeterministicContextBudgeting.csproj
|   +-- Program.cs
|   +-- appsettings.json
|   +-- App/
|   |   +-- AppConfig.cs
|   |   +-- EcommerceSupportScenario.cs
|   +-- Budgeting/
|   |   +-- ContextBudgeter.cs
|   |   +-- HeuristicTokenEstimator.cs
|   |   +-- ITokenEstimator.cs
|   +-- Domain/
|   |   +-- ContextBlock.cs
|   |   +-- PromptBudgetRequest.cs
|   |   +-- ContextBudgetingResult.cs
|   |   +-- PackedContextBlock.cs
|   |   +-- ExcludedContextBlock.cs
|   +-- Prompting/
|       +-- PromptComposer.cs
+-- DeterministicContextBudgeting.Tests/
|   +-- DeterministicContextBudgeting.Tests.csproj
|   +-- ContextBudgeterTests.cs
|   +-- PromptComposerTests.cs
+-- LICENSE
+-- README.md

Deterministic Guarantees

  • Stable inclusion order for equivalent candidates
  • Required overflow is explicit and blocks proceed (CanProceed = false)
  • Budget invariants are validated (used + remaining = available)
  • Prompt block order exactly follows packed block order

License

See the LICENSE file for details.

Contributing

Contributions are welcome for improvements within current project scope.

Suggested areas:

  • Better token estimation strategies with model-specific calibration
  • Additional business scenarios beyond e-commerce support
  • More deterministic tie-break and overflow policy variants
  • Expanded tests for edge cases and larger context sets