Skip to content

Give codavox its own environment; fix two bugs that made it unreachable - #65

Merged
miharp merged 3 commits into
productionfrom
fix/codavox-package-source
Jul 26, 2026
Merged

Give codavox its own environment; fix two bugs that made it unreachable#65
miharp merged 3 commits into
productionfrom
fix/codavox-package-source

Conversation

@miharp

@miharp miharp commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Found by vagrant destroy && vagrant up from scratch. Two real bugs, plus a design change so codavox never touches the dev environment.

The design: codavox gets its own environment

production is the Vagrant synced checkout of this repo — the dev environment. An edit there must appear on the next agent run with nothing in between: no r10k, no codavox reseal. So codavox now publishes codavox_test out of codavox-basedir/, which holds one subdirectory per environment and deliberately excludes production.

environment served from changes appear
production the synced checkout, straight from environmentpath immediately, no deploy step
codavox_test sealed from codavox-basedir/, distributed by codavox on the next reseal + poll

Because codavox-basedir/ is itself inside the synced folder, editing the test environment on the host is still immediate on the primary — it just needs a reseal to reach a compiler, which is the part being tested. Measured: a host edit reached the compiler ~5s after systemctl reload codavox-publish. Meanwhile puppet lookup message on the primary still reflects a host edit to production with no deploy step at all.

codavox_test carries a module with a file resource, not just a notify — so applying it exercises code-content, the half of the versioned-code contract a notify never touches.

This also fixes a permanent block

Pointing the publisher at the working tree didn't merely conflate concerns, it failed outright. .onceover/ holds rspec-puppet fixture symlinks whose targets are absolute macOS host paths:

refusing absolute symlink target "/Users/michaelharp/projects/control-repo/site-modules/profile"
  at .onceover/…/spec/fixtures/modules/profile

codavox was right to refuse — that's the extractor's escapesTree check — and the agent then failed loudly and left the environment untouched rather than deploying a partial tree. Correct behavior, but a hard block, and it would have returned the first time anyone ran onceover.

Bug 1: role::compiler had never run on any node

manifests/site.pp had no definition for compiler.example.com, so the compiler fell into node default and got profile::base only. Dead since the VM was added in #63, and nothing noticed — puppet agent -t exits 0, because applying a smaller catalog successfully is still success.

Onceover could not have caught it. It compiles role::compiler against the compiler's factset and passes, which proves the role is valid, not that any node is classified into it. Two similar-looking questions; only one was being asked.

Bug 2: codavox had no installable package

Without codavox::package_source, the module emits package { 'codavox': ensure => installed }, which needs a repository — codavox has none yet (miharp/codavox#42). Set as a Hiera literal pointing at the v0.5.0 release asset, written out rather than assembled from a version and an arch fact, since a URL a profile builds is a URL nobody can grep for.

Verified on the VMs

==> codavox package version
  ok   puppet: codavox 0.5.0          ok   compiler: codavox 0.5.0
==> config uses basedir, not the old staging key
  ok   puppet: basedir: ".../production/codavox-basedir"
==> services
  ok   puppet: codavox-publish active  ok   compiler: codavox-agent active
==> the compiler converged on codavox_test
  ok   compiler serves c74c0bee5ba83d60400b4f1b021d57451e35d0ba2c702e0c20ad392f33465dd5
==> the publisher's fleet view agrees with the compiler
       COMPILER              ENVIRONMENT   CODE_ID       LAST POLL
       compiler.example.com  codavox_test  c74c0bee5ba8  8s ago
  ok   publisher and compiler agree
==> staged adoption: wire_server false leaves catalog compilation alone
  ok   compiler environmentpath untouched   ok   no versioned-code.conf yet
==> agent runs are idempotent
  ok   puppet / compiler / agent01 / agent02: exit 0, no changes
ALL CHECKS PASSED

Onceover passes all five roles.

Still outstanding (not in this PR)

The primary's own static catalogs still come from profile::static_catalogs, whose code_id.sh returns git rev-parse HEAD. Because puppetserver serves the working tree through the synced folder, uncommitted edits change the served content but not the code_id — two catalogs can carry the same id over different content. Worth fixing separately; it is the failure codavox exists to prevent, currently live on the primary.

🤖 Generated with Claude Code

miharp and others added 3 commits July 26, 2026 09:04
codavox has no package repository (miharp/codavox#42), so
package { 'codavox': ensure => installed } has nothing to resolve against and the
Puppet run fails on both codavox nodes. Setting codavox::package_source points
the rpm provider at a release asset instead.

Written as a literal rather than assembled from a version and an architecture
fact: a URL a profile builds is a URL nobody can grep for. Both codavox nodes
here are aarch64, and a node on another architecture overrides the key in
data/nodes/<certname>.yaml.
manifests/site.pp had no definition for compiler.example.com, so the compiler
fell into `node default` and got profile::base only. role::compiler has therefore
never run on it since the VM was added, and nothing noticed: `puppet agent -t`
exits 0 because applying a smaller catalog successfully is still success.

Onceover did not catch it either, and could not have. It compiles role::compiler
against the compiler's factset and passes, because compiling a role proves the
role is valid — not that any node is classified into it. The two questions look
alike and only one of them was being asked.

Found by bringing the VMs up from scratch and looking for codavox on the
compiler, where there was none: no package, no agent, no code_id.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The publisher was pointed at /etc/puppetlabs/code/environments, which holds
production — the Vagrant synced checkout of this repo. That is the dev
environment: an edit here has to show up on the next agent run with nothing in
between, so it must not be made to depend on r10k or on a codavox reseal.

It also did not work. A working tree carries build artifacts, and .onceover/
holds rspec-puppet fixture symlinks whose targets are absolute macOS host paths.
codavox refused to unpack them, which is the extractor's escapesTree check doing
its job — and the agent then failed loudly and left the environment untouched
rather than deploying a partial tree. Correct behavior, but a permanent block,
and it would have come back the first time anyone ran onceover.

So codavox gets codavox_test, in codavox-basedir/. That directory holds one
subdirectory per environment and deliberately excludes production, so sealing
never touches the dev tree.

Because codavox-basedir/ is itself inside the synced folder, editing the test
environment on the host is still immediate on the primary — it just needs a
reseal to reach a compiler, which is the part being tested. Measured: a host edit
reached the compiler about five seconds after `systemctl reload codavox-publish`.
Meanwhile `puppet lookup message` on the primary still reflects a host edit to
production with no deploy step at all.

codavox_test carries a module with a file resource rather than only a notify,
so applying it exercises code-content — the second half of the versioned-code
contract, which a notify alone never touches.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@miharp
miharp merged commit de5dd0b into production Jul 26, 2026
3 checks passed
@miharp
miharp deleted the fix/codavox-package-source branch July 26, 2026 13:50
miharp added a commit that referenced this pull request Jul 26, 2026
)

Reverts #63, #64, and #65: the compiler VM, the pp_role csr_attributes, and all
the codavox wiring. The tree is now identical to 0a7c5f8 apart from one Puppetfile
line.

This repo is a dev environment — an edit to production has to appear on the next
agent run with no deploy step. codavox distributes sealed, immutable,
content-addressed versions. The two models are structurally opposed, and
everything added to reconcile them was paying for the mismatch: a directory of
environments nested inside an environment so the publisher had something other
than the working tree to seal; a working tree that cannot be sealed at all,
because .onceover/ holds rspec-puppet fixture symlinks pointing at absolute macOS
host paths; a compiler that could never serve production, since codavox replaces
environmentpath rather than adding to it; and static catalogs turned off on the
primary, a real capability lost purely to the conflict.

The compiler VM goes too. Without codavox it had no code — no synced mount, and a
stock empty production skeleton — so it installed openvox-server on a node that
could only compile empty catalogs. codavox was the only thing that ever fed it.

codavox validation moves to a control repo built for it, where r10k deploys a
clean tree and static catalogs stay on.

puppetlabs/yumrepo_core is kept because it is the one fix here that had nothing
to do with codavox. yumrepo left Puppet core, and while
site-modules/profile/.fixtures.yml already declared it for the module's own specs,
the control repo never shipped it — so onceover could not compile any RedHat role,
failing with "Unknown resource type: 'yumrepo'". Real nodes were unaffected
because openvox-agent bundles it, which is why it went unnoticed.

Verified: onceover passes all four remaining roles, and the profile suite is back
to 96 examples with no failures.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant