Skip to content

Distribute code to the compiler with codavox - #64

Merged
miharp merged 2 commits into
productionfrom
feat/codavox
Jul 26, 2026
Merged

Distribute code to the compiler with codavox#64
miharp merged 2 commits into
productionfrom
feat/codavox

Conversation

@miharp

@miharp miharp commented Jul 26, 2026

Copy link
Copy Markdown
Owner

The compiler has had no way to get resolved Puppet code. OpenVox Server ships without PE's Code Manager and file sync, so nothing puts a versioned tree on a compiler or lets it answer which version it is serving. This wires codavox in — publisher on the primary where the code already lives, agent on the compiler.

Staged on purpose

profile::codavox::agent::wire_server defaults to false.

codavox has no fallback by design: once OpenVox Server is pointed at it, catalog compilation depends on the agent having deployed code there. A compiler wired before its agent converges fails every catalog compile — loudly, which is correct, but it fails.

So stage 1 is what this PR does: the agent converges the compiler and keeps it current, without touching how catalogs are compiled. Confirm with codavox code-id production on the compiler and codavox compilers on the primary, then flip wire_server to true — a restart, not a wait.

Shape

node role what it runs
puppet.example.com role::puppet_master profile::codavox::publisher — seals the codedir, serves on 8150
compiler.example.com role::compiler profile::codavox::agent — polls, verifies, swaps

Settings live in Hiera under codavox::* rather than as profile parameters, so a node that ends up running both the publisher and an agent still resolves one consistent codavox class instead of declaring it twice with conflicting data.

staging is r10k's basedir — here the codedir the control repo is already synced into. codavox needs no staging area of its own, and pointing it one level higher gives a publisher that starts cleanly and advertises nothing.

The compiler is authorized by the pp_role in its certificate, written into csr_attributes.yaml before enrolment (#63). An estate whose compiler certificates predate codavox would use codavox::publish_allow_certnames instead, since pp_role cannot be added to an already-issued certificate.

puppet-codavox is pinned to v0.1.0 rather than floating on a branch: codavox exists to make code versions deterministic, and resolving its own module non-deterministically would undercut the point.

Drive-by fix: onceover was broken for every RedHat role

Adding puppetlabs/yumrepo_core to the Puppetfile. yumrepo left Puppet core, and while site-modules/profile/.fixtures.yml already declared it for the module's own specs, the control repo never shipped it — so onceover failed to compile role::puppet_master, role::compiler, role::database_server, and role::webserver with Unknown resource type: 'yumrepo'.

Real nodes were unaffected because openvox-agent bundles it, which is why this went unnoticed. I confirmed it was pre-existing by stashing this branch and reproducing the identical four failures on a clean tree. Fixing it is what makes onceover able to verify any of the above.

Two things I did not change, worth a look

  1. The primary still compiles static catalogs with profile::static_catalogs, whose hand-rolled code_id.sh ends in:

    # Last resort: use timestamp (not ideal for static catalogs)
    echo "warning: no git or r10k signature available, using timestamp" >&2
    date +%s

    On this primary it doesn't reach the timestamp — production is the synced control repo, so .git exists and it returns git rev-parse HEAD. But HEAD describes the control repo, not resolved module content, and does not change when the host working tree does. It also writes to stderr while exiting 0, which OpenVox Server logs at ERROR on every compile.

    The practical consequence: a catalog compiled by the primary and one compiled by the compiler now carry code_ids from two different schemes, which are not comparable. Moving the primary onto codavox too (an agent polling its own publisher) is the natural stage 3.

  2. The eyaml private key travels in the artifact. keys/private_key.pkcs7.pem lives in the synced production directory, so the publisher seals it and ships it to compilers. A compiler needs it to decrypt hiera-eyaml data during compilation, so this is functionally required — but it is worth knowing that the key is in the distributed tree rather than provisioned separately.

Verification

  • onceover: all five roles compile against real factsets with the actual Hiera data, including role::puppet_master and role::compiler with codavox wired. Confirmed codavox is fetched at the pinned tag (miharp-codavox 0.1.0).
  • rspec: 118 examples, 0 failures (96 existing + 22 new across both OSes), covering wire_server off and on, the firewall rule at default and custom ports, and that codavox::server is absent by default.
  • rake validate lint check clean; puppet parser validate and cspell clean on all new files.

Not yet run on the Vagrant VMs — happy to vagrant destroy && up to prove a clean install if you want that before merging.

miharp and others added 2 commits July 26, 2026 07:34
The compiler has had no way to get resolved Puppet code. OpenVox Server ships
without Code Manager and file sync, so nothing puts a versioned tree on a
compiler or lets it answer which version it is serving. This wires codavox in:
the publisher on the primary where the code already lives, the agent on the
compiler.

Adoption is staged, because codavox has no fallback by design. Once OpenVox
Server is pointed at it, catalog compilation depends on the agent having
deployed code there, so a compiler wired before its agent converges fails every
compile — loudly, which is correct, but it fails. So
profile::codavox::agent::wire_server defaults to false: the agent converges this
node and keeps it current, without touching how catalogs are compiled. Flipping
it later is a restart rather than a wait.

Settings live in Hiera under codavox::* rather than as profile parameters, so a
node that ends up running both the publisher and an agent still resolves one
consistent codavox class instead of declaring it twice with different data.

The publisher's staging directory is r10k's basedir, which here is the codedir
the control repo is already synced into. codavox needs no staging area of its
own, and pointing it one level higher gives a publisher that starts cleanly and
advertises nothing.

The compiler is authorized by the pp_role in its certificate, which
csr_attributes.yaml writes before enrolment. That works because the compiler VM
was built that way; an estate whose compiler certificates predate codavox would
use codavox::publish_allow_certnames instead, since pp_role cannot be added to an
already-issued certificate.

puppet-codavox is pinned by tag rather than floating on a branch. codavox exists
to make code versions deterministic, and resolving its own module
non-deterministically would undercut the point.

Also adds puppetlabs/yumrepo_core to the Puppetfile. yumrepo left Puppet core, and
while site-modules/profile/.fixtures.yml already declared it for the module's own
specs, the control repo never shipped it — so onceover failed to compile every
RedHat role with "Unknown resource type: 'yumrepo'". Real nodes were unaffected
because openvox-agent bundles it, which is why this went unnoticed. Fixing it is
what makes onceover able to verify any of the above.
codavox 0.5.0 renamed the staging setting to basedir with no alias, so the Hiera
key becomes codavox::basedir and the module pin moves to puppet-codavox v0.2.0,
which emits the new key. Leaving either behind would produce a publisher that
refuses to start, since codavox rejects unknown config keys rather than ignoring
them.

The rename is worth having: nothing is staged. The publisher only reads that
directory and writes nothing to it — the comment on this node's setting now says
so, because the old name implied codavox kept a copy somewhere and it does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@miharp
miharp merged commit 7961ba0 into production Jul 26, 2026
3 checks passed
@miharp
miharp deleted the feat/codavox branch July 26, 2026 12:55
miharp added a commit that referenced this pull request Jul 26, 2026
)

Reverts #63, #64, and #65: the compiler VM, the pp_role csr_attributes, and all
the codavox wiring. The tree is now identical to 0a7c5f8 apart from one Puppetfile
line.

This repo is a dev environment — an edit to production has to appear on the next
agent run with no deploy step. codavox distributes sealed, immutable,
content-addressed versions. The two models are structurally opposed, and
everything added to reconcile them was paying for the mismatch: a directory of
environments nested inside an environment so the publisher had something other
than the working tree to seal; a working tree that cannot be sealed at all,
because .onceover/ holds rspec-puppet fixture symlinks pointing at absolute macOS
host paths; a compiler that could never serve production, since codavox replaces
environmentpath rather than adding to it; and static catalogs turned off on the
primary, a real capability lost purely to the conflict.

The compiler VM goes too. Without codavox it had no code — no synced mount, and a
stock empty production skeleton — so it installed openvox-server on a node that
could only compile empty catalogs. codavox was the only thing that ever fed it.

codavox validation moves to a control repo built for it, where r10k deploys a
clean tree and static catalogs stay on.

puppetlabs/yumrepo_core is kept because it is the one fix here that had nothing
to do with codavox. yumrepo left Puppet core, and while
site-modules/profile/.fixtures.yml already declared it for the module's own specs,
the control repo never shipped it — so onceover could not compile any RedHat role,
failing with "Unknown resource type: 'yumrepo'". Real nodes were unaffected
because openvox-agent bundles it, which is why it went unnoticed.

Verified: onceover passes all four remaining roles, and the profile suite is back
to 96 examples with no failures.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant