fix(serve): install bundle modules on cold start; declare all 4 providers#70
Merged
Conversation
…ders
Root cause
----------
'amplifier-agent serve chat-completions' lifespan calls list_provider_models()
BEFORE any AmplifierSession is created. On a fresh install the provider Python
packages are not in the tool venv because:
1. bundle.prepare(install_deps=True) processes only session.orchestrator,
session.context, and top-level providers/tools/hooks -- NOT session.provider.
So the session.provider: anthropic-provider stub in bundle.md never triggered
a cold-prepare install despite the comment claiming otherwise.
2. run works because it calls inject_provider + Engine.boot() → create_session()
→ session.initialize() → resolver.async_resolve('provider-anthropic', source)
→ ModuleActivator.activate() → uv pip install. The lifespan never calls
create_session(), so that lazy-install path never fires for serve.
Changes
-------
src/amplifier_agent_lib/bundle/bundle.md
Add a top-level providers: section with all 4 CATALOG providers
(anthropic, openai, azure-openai, ollama) as install-only stubs (module +
source, no credentials). foundation's bundle.prepare() processes top-level
providers:, so cold-prep and the post-install hook now install all 4.
Update session.provider comment to reflect the actual mechanics.
src/amplifier_agent_http/app.py
Import PROVIDER_CATALOG. In lifespan, after prepare_bundle_for_session() and
before the providers loop, call prepared.resolver.async_resolve(module_id,
source) for each CATALOG entry. This is the same install function run uses
(via create_session → session.initialize → resolver.async_resolve). Idempotent:
fast path when already installed, lazy activation otherwise. Activation
failures are logged as warnings and let the providers loop report the failure.
src/amplifier_agent_cli/modes/single_turn.py
Clear prepared.mount_plan['providers'] = [] before inject_provider. Now that
bundle.md populates providers: with 4 stubs, inject_provider's 'no-op if
providers already present' guard would fire and skip injecting the runtime
provider with env-var credentials. Clearing first mirrors the pattern that
_session_runner.run_chat_turn already uses for exactly this reason.
tests/http/test_lifespan_module_install.py
New: 3 tests verifying the wire-up.
- test_lifespan_calls_install_trigger: resolver.async_resolve called for
every PROVIDER_CATALOG entry before the providers loop.
- test_lifespan_install_failure_still_fails_loud: activation failure is
non-fatal (warning logged); providers loop still runs and collects the
ProviderModuleNotInstalledError → exit 2.
- test_lifespan_install_idempotent_warm_cache: fast async_resolve (warm
cache) does not prevent lifespan completing successfully.
CHANGELOG.md: Updated [Unreleased] Added + Changed sections.
Verified
--------
- ruff check + ruff format --check: clean
- pyright src/: 1 pre-existing error (single_turn.py:720, unrelated)
- uv run pytest tests/config/: 47/47 passed
- uv run pytest tests/http/: 16/16 passed (13 existing + 3 new)
- uv run pytest tests/cli/: 237/238 passed (1 pre-existing env-var failure)
- uv run pytest tests/bundle/ tests/test_bundle_*.py: 28/28 passed
Pre-existing failures on main (confirmed by git stash test):
tests/cli/test_provider_sources.py::test_build_provider_entry_missing_env_var
(ANTHROPIC_API_KEY set in env; test deletes it but credentials file wins)
tests/test_bundle_hook_streaming.py::test_mount_registers_ten_handlers
(expects 10 handlers, gets 11; hook_streaming.py added a handler upstream)
🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)
Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two coupled changes that close the "fresh install →
serve chat-completionsfails because providers aren't installed" gap:bundle.mdnow declares all 4 default providers so cold-prep installs all of them.serve chat-completionslifespan triggers the same install-on-first-use path thatrunalready uses.Evidence the gap exists
Confirmed in a DTU smoke test on a fresh Ubuntu container:
The install machinery works correctly;
servejust doesn't trigger it.Investigation findings
The install trigger is
prepared.resolver.async_resolve(module_id, source_hint)inamplifier_foundation.bundle._prepared.BundleModuleResolver. It installs viaModuleActivator.activate()(→uv pip install --editable).How
runreaches it:_execute_turn→inject_provider→Engine.boot(bundle_override=prepared)→prepared.create_session()→session.initialize()→ kernel callsresolver.async_resolve("provider-anthropic", source)→ package installed.Why
servemisses it:The lifespan never calls
create_session()(that's per-request in_session_runner.run_chat_turn).list_provider_models()triesimportlib.import_module("amplifier_module_provider_anthropic")before any session is created — and fails withProviderModuleNotInstalledError.Why cold-prep doesn't help alone:
bundle.prepare(install_deps=True)processessession.orchestrator,session.context, and top-levelproviders:/tools:/hooks:— but NOTsession.provider:. The comment in bundle.md claiming "declaring it here ensures the cold-prepare step installs the provider module" was wrong.Idempotency:
async_resolvehas a fast path (if module_id in self._paths) that returns immediately on warm cache.ModuleActivatorskips reinstall when already present.Why this is the right fix
A previous discussion considered a wrapper-side bootstrap (have
amplifier-app-opencoderunamplifier-agent runonce before spawning the server). That's wrapper-specific — every consumer ofamplifier-agent serve(Slack bots, web UIs, custom integrations) would have to reimplement the same bootstrap.Fixing it in
amplifier-agentitself makes the install-on-cold-start behavior universal.File changes
src/amplifier_agent_lib/bundle/bundle.mdproviders:section with all 4 CATALOG providers as install-only stubs (module + source, no credentials). Updatesession.providercomment to reflect actual mechanics.src/amplifier_agent_http/app.pyPROVIDER_CATALOG. Callprepared.resolver.async_resolve(module_id, source)for every CATALOG entry in lifespan before the providers loop. Failures are logged as warnings; the providers loop then reports them as structured errors.src/amplifier_agent_cli/modes/single_turn.pyprepared.mount_plan["providers"] = []beforeinject_provider. Now that bundle.md populates top-levelproviders:with 4 stubs, inject_provider's "no-op if already present" guard would skip injecting the runtime provider. Mirrors the pattern_session_runner.run_chat_turnalready uses.tests/http/test_lifespan_module_install.pyCHANGELOG.md[Unreleased]Added / Changed.Compatibility
amplifier-agent run) is unchanged in behavior — same code path, same runtime result.host_config.providersschema (from feat(http)!: require host_config.providers; fail loudly on misconfig #69) is unchanged.Backlog impact
Resolves the install gap surfaced by the DTU smoke test. Server mode now works from a fresh
uv tool installwith zero manual prep steps.Companion update
amplifier-app-opencode users no longer need to know about bundle prep or post-install hooks —
amplifier-opencode launchjust works after a fresh install.