Skip to content

feat(docs): add AVM documentation parity utility - #67

Open
Jared Holgate (jaredfholgate) wants to merge 11 commits into
mainfrom
jaredfholgate-avm-docs-parity-template
Open

feat(docs): add AVM documentation parity utility#67
Jared Holgate (jaredfholgate) wants to merge 11 commits into
mainfrom
jaredfholgate-avm-docs-parity-template

Conversation

@jaredfholgate

@jaredfholgate Jared Holgate (jaredfholgate) commented Aug 14, 2026

Copy link
Copy Markdown
Member

Summary

  • add reusable AVM Scriban templates for Bicep module documentation
  • add root bicepconfig.json documentation settings, including all three multi-scope example reassignments
  • add a single-module verifier and parallel all-module runner
  • use bicep docs generate <file> --stdout for semantic output and standard config discovery
  • compare generated READMEs byte-for-byte while retaining structured parameter, example, exported-type, and exported-variable reports
  • report genuine Bicep compilation failures without generating a README

Final parity baseline

Against Azure/bicep-registry-modules commit 55c62d45eaf6675c09bf663616c3e7fdd8c4560f, using Bicep commit 583ede7ae0617361fa66e08feae8d5d457de3017, the definitive run completed in 00:13:36.2321022:

  • 573 modules compared
  • 572 READMEs generated; all 572 matched byte-for-byte
  • 1 genuine compiler failure: avm/ptn/app/container-job-toolkit (BCP426, BCP104, BCP287, BCP036); no README was generated
  • 0 example-model mismatches
  • 9 parameter-model differences, all additional typed paths unavailable to the legacy ARM-JSON generator; no expected paths are missing
  • published executable SHA256: 30F50C158D1C6715E1A28C1FDF8F9691EE243226FE24AC82970AADD610BCCB18
  • full evidence: https://gist.github.com/jaredfholgate/71babd2c996edf08473dcdb10a58aa12

Validation

  • ./build.ps1 pre-commit (28 component tests)
  • strict JSON validation for bicepconfig.json
  • PowerShell syntax parsing for both parity scripts
  • static checks for the final CLI/config migration
  • full 573-module comparison against the exact Bicep PR head

Related work

Add the validated Scriban README template, semantic model index, and parity verifier for Bicep AVM module documentation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-code-quality

github-code-quality Bot commented Aug 14, 2026

Copy link
Copy Markdown

Code Coverage Overview

Languages: PowerShell

PowerShell / code-coverage-agent

The overall coverage in commit faa31a8 in the jaredfholgate-avm-do... branch remains at 89%, unchanged from commit 5d15776 in the main branch.


Updated August 18, 2026 11:56 UTC

@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown

Test results

   12 files  +  6    579 suites  +30   13m 50s ⏱️ + 8m 2s
1 013 tests + 21  1 011 ✅ +19   2 💤 +2  0 ❌ ±0 
3 084 runs  +102  3 055 ✅ +96  29 💤 +6  0 ❌ ±0 

Results for commit faa31a8. ± Comparison against base commit 5d15776.

♻️ This comment has been updated with latest results.

Add all-module byte-parity reporting and retain genuine compiler failures and semantic model mismatches in detailed reports.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c1d476aa-095e-4b89-8a06-66f8642720ad
Include captured compiler errors in the aggregate parity validation report.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c1d476aa-095e-4b89-8a06-66f8642720ad
Record the Bicep version and executable SHA256 in aggregate parity reports.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c1d476aa-095e-4b89-8a06-66f8642720ad
Use the docs configuration for every module, reassign multi-scope examples, and compare structured discriminator parameters in the semantic index.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 5b6b551b-dfe1-461c-8caa-e9d250df1026
Use the finalized Bicep docs config name, schema, and module input selection.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c1d476aa-095e-4b89-8a06-66f8642720ad
Use standard bicepconfig discovery, docs generate stdout output, and export-aware semantic templates.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c1d476aa-095e-4b89-8a06-66f8642720ad
Keep the standard Bicep documentation configuration valid strict JSON.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c1d476aa-095e-4b89-8a06-66f8642720ad
Jared Holgate (jaredfholgate) added a commit to Azure/bicep that referenced this pull request Aug 18, 2026
## Description

Adds the experimental `bicep docs generate` command, which renders
module documentation from Bicep's semantic model plus discovered usage
examples through [Scriban](https://github.com/scriban/scriban).

Implements [REP 0025: Module documentation generation
commands](https://github.com/Azure/bicep-reps/blob/main/active/0025-module-documentation-generation.md)
and supersedes
[#19694](#19694).

The validated AVM templates, root configuration, semantic verifier, and
full-repository runner are in [Azure/azure-verified-modules-tools PR
#67](Azure/azure-verified-modules-tools#67).

## Example usage

```powershell
# Generate README.md beside one module
bicep docs generate .\main.bicep

# Render one module to stdout
bicep docs generate .\main.bicep --stdout

# Generate documentation for matched modules
bicep docs generate --pattern '.\modules\**\main.bicep'
```

`--stdout` follows the existing Bicep output-option contract and cannot
be combined with `--pattern`, `--outdir`, or `--outfile`.

## What's included

**CLI**
- one `bicep docs generate` command with explicit `.bicep` or
`--pattern` input
- standard `--stdout`, `--outdir`, and `--outfile` behavior
- built-in Markdown, custom Scriban templates, includes, and ordered
inline/JSON-file custom values
- one SARIF document per invocation for bulk compilation, rendering, and
write failures
- shared compilation state across bulk generation

**Core engine**
- reusable deterministic typed model and renderer in `Bicep.Core`
- resource types, parameters, outputs, exported types, exported
variables, exported functions, referenced modules, and usage examples
- recursive type expansion with cycle detection, a node budget,
truncation metadata, and cancellation
- generic example sources and parent-to-child reassignment without
AVM-specific hard-coding

**Configuration**
- typed `documentation` section in `bicepconfig.json`
- built-in defaults and merged `bicepconfig.schema.json`
validation/IntelliSense
- config-relative template paths anchored to the resolved config file
- per-source standard nearest-file configuration resolution

**JSON-RPC**
- `bicep/generateDocs` and `bicep/outputDocs` remain directly on
`IBicepClient`
- each path uses its own resolved `bicepconfig.json`
- shared compilation state for bulk requests
- request cancellation reaches model construction and rendering

## Configuration

```json
{
  "documentation": {
    "output": {
      "file": "README.md"
    },
    "template": {
      "file": "docs/templates/readme.scriban",
      "includeRoot": "docs/templates",
      "values": {
        "owner": "Platform Team"
      }
    },
    "examples": {
      "sources": [
        {
          "path": "examples",
          "include": ["*.bicep", "**/main.bicep"],
          "exclude": ["**/dependencies*.bicep"]
        }
      ],
      "reassignments": []
    }
  }
}
```

Configured relative template paths are anchored to the resolved config
file's directory. CLI template paths remain current-directory-relative.

`bicepconfig.json` remains nearest-file-wins with no ancestor merging. A
module with its own config does not inherit repository-level
documentation settings and receives built-in defaults for omitted
values. Repositories requiring uniform settings should prevent nested
configs until configuration inheritance is designed separately.

Full experimental reference:
[`docs/experimental/docs-commands.md`](https://github.com/Azure/bicep/blob/jaredfholgate-bicep-docs-generation/docs/experimental/docs-commands.md).

## Impact on existing commands

`InputOutputArgumentsResolver` supports a fixed output filename resolver
so docs can use `README.md` rather than extension replacement. Existing
callers omit it and retain their existing behavior. Common
filesystem/path exception predicates now live in `Bicep.IO`; CLI-wide
central exception handling is intentionally not changed in this PR.

Documentation uses the standard non-atomic
`OutputWriter.WriteToFileAsync` path used by other commands. Compilation
and rendering complete before the write, so those failures do not
overwrite an existing output.

## Validation

- `Bicep.Core.UnitTests`: 7,001 passed
- `Bicep.Cli.IntegrationTests`: 1,137 passed
- `Bicep.Cli.UnitTests`: 72 passed
- `Bicep.IO.UnitTests`: 145 passed
- `Bicep.RpcClient.Tests`: 96 passed, 1 skipped
- Bicep solution build: 0 warnings, 0 errors
- VS Code UI and extension builds passed
- all newly introduced and directly modified documentation/configuration
production paths reached 100% executable line and branch coverage

**Trimmed single-file publish** — Release `win-x64`, commit
`583ede7ae0617361fa66e08feae8d5d457de3017`:
- built-in stdout output matched the comprehensive golden exactly
- custom Scriban rendering with `include` matched exactly
- config-relative template/include resolution from `bicepconfig.json`
matched exactly
- executable SHA256:
`30F50C158D1C6715E1A28C1FDF8F9691EE243226FE24AC82970AADD610BCCB18`

**Full `Azure/bicep-registry-modules` validation** against the same
Bicep commit:
- 573 modules compared in `00:13:36.2321022`
- 572 READMEs generated and all 572 matched byte-for-byte
- one module was not written because `avm/ptn/app/container-job-toolkit`
has genuine compilation errors (`BCP426`, `BCP104`, `BCP287`, and
`BCP036`)
- example-model mismatches: 0
- parameter-model differences: 9 modules exposing additional typed paths
unavailable to the legacy ARM-JSON generator; no expected paths are
missing
- full evidence:
https://gist.github.com/jaredfholgate/71babd2c996edf08473dcdb10a58aa12

## Checklist

- [x] I have read and adhere to the [contribution
guide](https://github.com/Azure/bicep/blob/main/CONTRIBUTING.md).

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5b6b551b-dfe1-461c-8caa-e9d250df1026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant