[Feature] Configurable model for pruning operations (distill/compress)

[well0nez]

Problem

DCP currently relies on the main conversation model to make all pruning decisions and generate distillation/compression content. The tools (distill, compress, prune) are called autonomously by the primary model through injected system instructions.

When running an expensive model like Claude Opus 4 (claude-opus-4-6), this means every distillation and compression summary is generated at Opus-tier pricing. In practice, this has caused significant additional cost on my end — context management operations that could be handled by a smaller, cheaper model (e.g. Haiku or Sonnet) are instead burning through the most expensive model available.

The irony: a plugin designed to optimize token usage ends up inflating costs because the pruning work itself is done by a top-tier model.

Proposed Solution

Add a pruningModel configuration option to dcp.jsonc that allows users to specify a different (cheaper) model for pruning operations:

{
  "pruningModel": "anthropic/claude-sonnet-4-20250514"
}

Technical Considerations

After analyzing the codebase I want to be transparent that this is not a trivial change. Key challenges I identified:

1. No separation between decision and execution: Currently, the main model decides what to prune AND generates the distillation/summary text in a single tool call. These would need to be decoupled.

2. session.prompt() is not a lightweight LLM call: The OpenCode SDK's session.prompt() accepts a model parameter, but it routes through the full plugin pipeline (all hooks fire, messages are persisted, DCP itself would run on the child session). This is not a simple completion API call.

3. State isolation: DCP tools operate on session-specific state (toolParameters, toolIdList, prune.tools). A child session spawned via session.prompt() would have empty state — tool IDs referenced in the pruning prompt wouldn't resolve.

4. Architectural options:

   • Pre-turn pruning in messages.transform hook: Before the main model runs, DCP detects pruning is needed, sends the <prunable-tools> list to a cheaper model via a child session (text-only, no DCP tools), parses the structured response, and applies decisions to the main session state. The main model then sees an already-pruned context. DCP already skips sub-agents (state.isSubAgent), so the child session wouldn't trigger recursive pruning.
   • Direct LLM API call: Bypass OpenCode entirely and call the LLM provider SDK directly. Cleaner separation, but requires a separate API key in the DCP config, which breaks the seamless auth experience.

Impact

For users running expensive frontier models as their primary, this would be a significant cost optimization — pruning decisions and summary generation don't require frontier-level intelligence.

Thanks for the excellent plugin — DCP has been genuinely useful for keeping long sessions manageable. This would make it even better for users who run expensive models.

[Tarquinen]

So you're basically saying create a tool that broadcasts its now time to prune/distill/compress whatever, and then when that is received switch models to a cheaper model and inject some kind of prompt that "you should use the distill" tool, and let the cheaper model process the distillation? This would mean you would lose all cache read for the tool processing call as you're now on a different model, and the benefit is the output tokens (which will be hugely less in number than the input tokens, maybe something like 100x on average) will be cheaper (and worse quality)? Cache read for opus is half the cost of normal input tokens on haiku.

What am I missing here? This sounds more expensive, worse performance, and a much less seamless process requiring 2 tool calls instead of 1..

[well0nez]

You're right on all counts — I hadn't considered the cache read dynamics at all. The whole premise of "use a cheaper model" falls apart when the expensive model already has the context cached and the cheap model would need to cold-read everything. That's just more expensive for worse results. My bad.

To give some context on what actually led me here: I'm on a max 200 plan. My 5-hour window freed up and within 30 minutes the entire window was burned through. Next window, same thing. I disabled the plugin as a test — the excessive consumption stopped immediately. So I went looking for ways to reduce the cost, landed on "use a cheaper model for pruning", and clearly didn't understand the mechanics well enough to realize that's not where the problem is.

Thing is, it was running smoothly for a while. Then at some point it got very aggressive with context management and started eating through tokens rapidly. I can't pinpoint exactly what changed or what triggered it — I don't have enough visibility into what was happening under the hood.

Apologies if this caused any frustration. The feature request was based on a flawed understanding. But if you have any idea what might cause that kind of sudden spike in token consumption, I'd appreciate any pointers — because the plugin was genuinely great before that happened.

[Tarquinen]

yea that's really annoying I agree, this happens because anthropic caching is really terrible (no other providers behave this way), any kind of modifications invalidate the entire cache other than the system prompt, also their cache read tokens are "free" on subscription, so losing any cache is really bad. There's some information specific for the anthropic sub in the readme, although I should probably add more. Because of this I added manual mode, which doesn't have any injections so I think the cache should build normally. I haven't been able to test this properly because my claude pro trial ran out, but I think it should work.

Here is an example of a claude sub session with DCP, as you can see the cache hit rate is terrible

and here's what it looks like normally or with other models

[Tarquinen]

If you could test out the opencode-session-timeline script in the scripts directory and confirm, that would be really helpful. You might need to change some paths to match your own filesystem. Unfortunately I can't maintain every subscription to test all the models everyone uses...

[well0nez]

So I ran the timeline script on my session - apologies in advance for the size of that screenshot, I honestly didn't realize how massive that session had gotten. That's not my usual session size, but that's kind of the point. DCP was so good at keeping things manageable that I rarely needed to start a new one. Important context was preserved and I could just keep going.

Anyway, here's what I'm seeing:

The first ~166 steps look fine, cache hit rate is solid, 60-100%, everything behaving as expected. Then around step 167 it just falls off a cliff. Cache drops to 0% and never recovers. From there on it's hundreds of steps at 100k-200k uncached input tokens each.

The tricky part is that the summary at the bottom says 'Overall cache hit rate: 122.9%' and 'Provider caching appears to be working' - which looks totally fine at a glance. But that's because the good first half pulls the average up, and the >100% values from negative input deltas inflate it further. The summary doesn't really surface the fact that the session splits into two completely different phases — one where caching works and one where it's completely dead.

Around step 919 is where I disabled the plugin. After that, no more DCP tool calls.

Hope this helps narrow things down. Happy to run any other diagnostics if useful.

[Tarquinen]

yea the analysis at the bottom isn't that useful, this was just some quick vibe coded thing so I could see what each step of the session looks like. This is interesting though, any idea what might have changed at steps 166 and 206 where caching died completely? Did your session compact, did you switch models/providers, anything like that? Also, do you think you could try out the plugin again with manual mode on, and then check this script again to see if you get healthy values like you got here in the first 166? To turn manual mode on you just need to add

"manualMode": {
    "enabled": false,
},

to your dcp.jsonc. It shouldn't use any prune tools automatically in this mode, but you can still trigger them with /dcp distill or /dcp prune, and you can give them guidance like /dcp distill the information from the last search