Skip to content

fix(serve): install bundle modules on cold start; declare all 4 providers#70

Merged
manojp99 merged 1 commit into
mainfrom
feat/serve-installs-providers
Jun 22, 2026
Merged

fix(serve): install bundle modules on cold start; declare all 4 providers#70
manojp99 merged 1 commit into
mainfrom
feat/serve-installs-providers

Conversation

@manojp99

Copy link
Copy Markdown
Collaborator

Summary

Two coupled changes that close the "fresh install → serve chat-completions fails because providers aren't installed" gap:

  1. bundle.md now declares all 4 default providers so cold-prep installs all of them.
  2. serve chat-completions lifespan triggers the same install-on-first-use path that run already uses.

Evidence the gap exists

Confirmed in a DTU smoke test on a fresh Ubuntu container:

# Fresh `uv tool install amplifier-agent`
amplifier-agent run -y "say hi"
  → succeeds; venv now has _editable_impl_amplifier_module_provider_anthropic.pth + 9 others

# Same fresh install
amplifier-agent serve chat-completions --config <providers-block>
  → ProviderModuleNotInstalledError: provider 'anthropic': module 'anthropic' not installed
  → venv has ZERO amplifier_module_* packages

The install machinery works correctly; serve just doesn't trigger it.

Investigation findings

The install trigger is prepared.resolver.async_resolve(module_id, source_hint) in amplifier_foundation.bundle._prepared.BundleModuleResolver. It installs via ModuleActivator.activate() (→ uv pip install --editable).

How run reaches it:
_execute_turninject_providerEngine.boot(bundle_override=prepared)prepared.create_session()session.initialize() → kernel calls resolver.async_resolve("provider-anthropic", source) → package installed.

Why serve misses it:
The lifespan never calls create_session() (that's per-request in _session_runner.run_chat_turn). list_provider_models() tries importlib.import_module("amplifier_module_provider_anthropic") before any session is created — and fails with ProviderModuleNotInstalledError.

Why cold-prep doesn't help alone:
bundle.prepare(install_deps=True) processes session.orchestrator, session.context, and top-level providers:/tools:/hooks: — but NOT session.provider:. The comment in bundle.md claiming "declaring it here ensures the cold-prepare step installs the provider module" was wrong.

Idempotency: async_resolve has a fast path (if module_id in self._paths) that returns immediately on warm cache. ModuleActivator skips reinstall when already present.

Why this is the right fix

A previous discussion considered a wrapper-side bootstrap (have amplifier-app-opencode run amplifier-agent run once before spawning the server). That's wrapper-specific — every consumer of amplifier-agent serve (Slack bots, web UIs, custom integrations) would have to reimplement the same bootstrap.

Fixing it in amplifier-agent itself makes the install-on-cold-start behavior universal.

File changes

File Change
src/amplifier_agent_lib/bundle/bundle.md Add top-level providers: section with all 4 CATALOG providers as install-only stubs (module + source, no credentials). Update session.provider comment to reflect actual mechanics.
src/amplifier_agent_http/app.py Import PROVIDER_CATALOG. Call prepared.resolver.async_resolve(module_id, source) for every CATALOG entry in lifespan before the providers loop. Failures are logged as warnings; the providers loop then reports them as structured errors.
src/amplifier_agent_cli/modes/single_turn.py Clear prepared.mount_plan["providers"] = [] before inject_provider. Now that bundle.md populates top-level providers: with 4 stubs, inject_provider's "no-op if already present" guard would skip injecting the runtime provider. Mirrors the pattern _session_runner.run_chat_turn already uses.
tests/http/test_lifespan_module_install.py New: 3 tests verifying the wire-up (trigger called, failure is non-fatal → exit 2, idempotent on warm cache).
CHANGELOG.md Add to [Unreleased] Added / Changed.

Compatibility

Backlog impact

Resolves the install gap surfaced by the DTU smoke test. Server mode now works from a fresh uv tool install with zero manual prep steps.

Companion update

amplifier-app-opencode users no longer need to know about bundle prep or post-install hooks — amplifier-opencode launch just works after a fresh install.

…ders

Root cause
----------
'amplifier-agent serve chat-completions' lifespan calls list_provider_models()
BEFORE any AmplifierSession is created. On a fresh install the provider Python
packages are not in the tool venv because:

1. bundle.prepare(install_deps=True) processes only session.orchestrator,
   session.context, and top-level providers/tools/hooks -- NOT session.provider.
   So the session.provider: anthropic-provider stub in bundle.md never triggered
   a cold-prepare install despite the comment claiming otherwise.

2. run works because it calls inject_provider + Engine.boot() → create_session()
   → session.initialize() → resolver.async_resolve('provider-anthropic', source)
   → ModuleActivator.activate() → uv pip install. The lifespan never calls
   create_session(), so that lazy-install path never fires for serve.

Changes
-------
src/amplifier_agent_lib/bundle/bundle.md
  Add a top-level providers: section with all 4 CATALOG providers
  (anthropic, openai, azure-openai, ollama) as install-only stubs (module +
  source, no credentials). foundation's bundle.prepare() processes top-level
  providers:, so cold-prep and the post-install hook now install all 4.
  Update session.provider comment to reflect the actual mechanics.

src/amplifier_agent_http/app.py
  Import PROVIDER_CATALOG. In lifespan, after prepare_bundle_for_session() and
  before the providers loop, call prepared.resolver.async_resolve(module_id,
  source) for each CATALOG entry. This is the same install function run uses
  (via create_session → session.initialize → resolver.async_resolve). Idempotent:
  fast path when already installed, lazy activation otherwise. Activation
  failures are logged as warnings and let the providers loop report the failure.

src/amplifier_agent_cli/modes/single_turn.py
  Clear prepared.mount_plan['providers'] = [] before inject_provider. Now that
  bundle.md populates providers: with 4 stubs, inject_provider's 'no-op if
  providers already present' guard would fire and skip injecting the runtime
  provider with env-var credentials. Clearing first mirrors the pattern that
  _session_runner.run_chat_turn already uses for exactly this reason.

tests/http/test_lifespan_module_install.py
  New: 3 tests verifying the wire-up.
  - test_lifespan_calls_install_trigger: resolver.async_resolve called for
    every PROVIDER_CATALOG entry before the providers loop.
  - test_lifespan_install_failure_still_fails_loud: activation failure is
    non-fatal (warning logged); providers loop still runs and collects the
    ProviderModuleNotInstalledError → exit 2.
  - test_lifespan_install_idempotent_warm_cache: fast async_resolve (warm
    cache) does not prevent lifespan completing successfully.

CHANGELOG.md: Updated [Unreleased] Added + Changed sections.

Verified
--------
- ruff check + ruff format --check: clean
- pyright src/: 1 pre-existing error (single_turn.py:720, unrelated)
- uv run pytest tests/config/: 47/47 passed
- uv run pytest tests/http/: 16/16 passed (13 existing + 3 new)
- uv run pytest tests/cli/: 237/238 passed (1 pre-existing env-var failure)
- uv run pytest tests/bundle/ tests/test_bundle_*.py: 28/28 passed

Pre-existing failures on main (confirmed by git stash test):
  tests/cli/test_provider_sources.py::test_build_provider_entry_missing_env_var
    (ANTHROPIC_API_KEY set in env; test deletes it but credentials file wins)
  tests/test_bundle_hook_streaming.py::test_mount_registers_ten_handlers
    (expects 10 handlers, gets 11; hook_streaming.py added a handler upstream)

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@manojp99 manojp99 merged commit 485d089 into main Jun 22, 2026
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant