Skip to content

Commit ebd7f9f

Browse files
EMaherCopilot
EMaher
and
Copilot
committed
feat: add dry-run validation step to publish pipeline templates
The publish pipelines generated by �piops init now include a dry-run validation gate before the actual publish. If the dry-run fails (e.g., connectivity issues, invalid resources, permission errors), the pipeline halts immediately — preventing partial failures from leaving APIM in an inconsistent state. Changes: - GitHub Actions workflow template: added dry-run validation steps - Azure DevOps pipeline template: added dry-run validation tasks - Unit tests for both templates - Updated CI/CD and dry-run documentation Closes #116 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 18bc2be commit ebd7f9f

7 files changed

Lines changed: 227 additions & 6 deletions

File tree

‎docs/ci-cd/azure-devops.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,8 @@ Each stage:
146146
3. **Loads per-environment variables** — Each stage uses its own variable group (`apim-dev`, `apim-prod`)
147147
4. **Authenticates per-environment** — Uses environment-specific service connections (`AZURE_SERVICE_CONNECTION_DEV`, `AZURE_SERVICE_CONNECTION_PROD`)
148148
5. **Substitutes tokens** — Replaces `{#[TOKEN_NAME]#}` placeholders in `configuration.<env>.yaml` with secret variable values before publishing
149-
6. **Applies overrides** — Passes `--overrides configuration.{env}.yaml` to apply [environment-specific overrides](../guides/environment-overrides.md)
149+
6. **Runs a dry-run validation** — Executes `apiops publish --dry-run` to verify the publish would succeed. If this step fails, the pipeline halts and the real publish is never attempted, preventing partial failures from leaving APIM in an inconsistent state.
150+
7. **Applies overrides** — Passes `--overrides configuration.{env}.yaml` to apply [environment-specific overrides](../guides/environment-overrides.md)
150151

151152
### Publish Pipeline Walkthrough
152153

‎docs/ci-cd/github-actions.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,8 @@ The workflow runs automatically when changes are pushed to `main` in these paths
101101
2. **Checks out the repository** with `fetch-depth: 2` (needed for git diff).
102102
3. **Authenticates with Azure** using OIDC federated credentials.
103103
4. **Substitutes tokens** — replaces `{#[TOKEN_NAME]#}` placeholders in `configuration.<env>.yaml` with pipeline secret values.
104-
5. **Runs `apiops publish`** in one of two modes:
104+
5. **Runs a dry-run validation** — executes `apiops publish --dry-run` to verify the publish would succeed. If this fails, the workflow halts and the real publish is never attempted. This prevents partial failures from leaving APIM in an inconsistent state.
105+
6. **Runs `apiops publish`** in one of two modes:
105106
- **Incremental** (default): uses `--commit-id` to publish only changed files.
106107
- **Full**: publishes all artifacts in the repository (useful for recovery or initial setup).
107108

‎docs/guides/dry-run-workflow.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -237,6 +237,33 @@ This lets reviewers see _"this PR will create 2 APIs and update 1 backend"_ dire
237237

238238
---
239239

240+
## Built-in Pipeline Validation Gate
241+
242+
The publish pipelines generated by `apiops init` include an **automatic dry-run validation step** before every publish. This prevents partial failures from leaving your APIM instance in an inconsistent state.
243+
244+
### How it works
245+
246+
1. The pipeline runs `apiops publish --dry-run` with the same arguments as the real publish
247+
2. If the dry-run exits with a non-zero code (connectivity issues, invalid resources, permission errors), the pipeline **halts immediately**
248+
3. The real publish step only executes if the dry-run succeeds
249+
250+
This means APIM is never modified unless the full operation is validated first.
251+
252+
### What triggers a dry-run failure?
253+
254+
| Scenario | Result |
255+
|----------|--------|
256+
| Resource group or APIM service doesn't exist | Pre-flight validation fails |
257+
| Invalid credentials or expired token | Authentication error |
258+
| Malformed artifact files | Parse error during resource discovery |
259+
| Network connectivity issues | API call timeout/failure |
260+
261+
### Opting out
262+
263+
If you want to skip the dry-run gate (e.g., for faster iteration in dev), remove the "Dry-run validation" step from your generated pipeline file. The publish step works independently.
264+
265+
---
266+
240267
## Related
241268

242269
- [apiops publish](../commands/publish.md) — Full command reference

‎src/templates/azure-devops/publish-pipeline.ts‎

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,9 +70,42 @@ export function generatePublishPipeline(config: PublishPipelineConfig): string {
7070
fi
7171
displayName: 'Validate token substitution (${env})'
7272
73+
- task: AzureCLI@2
74+
displayName: 'Dry-run validation (${env}, incremental)'
75+
condition: and(succeeded(), ne('\${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo'))
76+
inputs:
77+
azureSubscription: 'AZURE_SERVICE_CONNECTION_${envUpper}'
78+
scriptType: 'bash'
79+
scriptLocation: 'inlineScript'
80+
inlineScript: |
81+
npx apiops publish \\
82+
--resource-group $(APIM_RESOURCE_GROUP_${envUpper}) \\
83+
--service-name $(APIM_SERVICE_NAME_${envUpper}) \\
84+
--source ${config.artifactDir} \\
85+
--overrides configuration.${env}.yaml \\
86+
--commit-id $(Build.SourceVersion) \\
87+
--subscription-id $(AZURE_SUBSCRIPTION_ID) \\
88+
--dry-run
89+
90+
- task: AzureCLI@2
91+
displayName: 'Dry-run validation (${env}, all artifacts)'
92+
condition: and(succeeded(), eq('\${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo'))
93+
inputs:
94+
azureSubscription: 'AZURE_SERVICE_CONNECTION_${envUpper}'
95+
scriptType: 'bash'
96+
scriptLocation: 'inlineScript'
97+
inlineScript: |
98+
npx apiops publish \\
99+
--resource-group $(APIM_RESOURCE_GROUP_${envUpper}) \\
100+
--service-name $(APIM_SERVICE_NAME_${envUpper}) \\
101+
--source ${config.artifactDir} \\
102+
--overrides configuration.${env}.yaml \\
103+
--subscription-id $(AZURE_SUBSCRIPTION_ID) \\
104+
--dry-run
105+
73106
- task: AzureCLI@2
74107
displayName: 'Publish to ${env} (incremental - last commit only)'
75-
condition: ne('\${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo')
108+
condition: and(succeeded(), ne('\${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo'))
76109
inputs:
77110
azureSubscription: 'AZURE_SERVICE_CONNECTION_${envUpper}'
78111
scriptType: 'bash'
@@ -88,7 +121,7 @@ export function generatePublishPipeline(config: PublishPipelineConfig): string {
88121
89122
- task: AzureCLI@2
90123
displayName: 'Publish to ${env} (all artifacts)'
91-
condition: eq('\${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo')
124+
condition: and(succeeded(), eq('\${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo'))
92125
inputs:
93126
azureSubscription: 'AZURE_SERVICE_CONNECTION_${envUpper}'
94127
scriptType: 'bash'

‎src/templates/github-actions/publish-workflow.ts‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,29 @@ ${autoDeployComment}
103103
exit 1
104104
fi
105105
106+
- name: Dry-run validation (${env}, incremental)
107+
if: \${{ github.event.inputs.COMMIT_ID_CHOICE != 'publish-all-artifacts-in-repo' }}
108+
run: |
109+
npx apiops publish \\
110+
--subscription-id \${{ secrets.AZURE_SUBSCRIPTION_ID }} \\
111+
--resource-group \${{ secrets.APIM_RESOURCE_GROUP_${envUpper} }} \\
112+
--service-name \${{ secrets.APIM_SERVICE_NAME_${envUpper} }} \\
113+
--source ${config.artifactDir} \\
114+
--overrides configuration.${env}.yaml \\
115+
--commit-id \${{ needs.get-commit.outputs.commit_id }} \\
116+
--dry-run
117+
118+
- name: Dry-run validation (${env}, all artifacts)
119+
if: \${{ github.event.inputs.COMMIT_ID_CHOICE == 'publish-all-artifacts-in-repo' }}
120+
run: |
121+
npx apiops publish \\
122+
--subscription-id \${{ secrets.AZURE_SUBSCRIPTION_ID }} \\
123+
--resource-group \${{ secrets.APIM_RESOURCE_GROUP_${envUpper} }} \\
124+
--service-name \${{ secrets.APIM_SERVICE_NAME_${envUpper} }} \\
125+
--source ${config.artifactDir} \\
126+
--overrides configuration.${env}.yaml \\
127+
--dry-run
128+
106129
- name: Publish to ${env} (incremental - last commit only)
107130
if: \${{ github.event.inputs.COMMIT_ID_CHOICE != 'publish-all-artifacts-in-repo' }}
108131
run: |

‎tests/unit/templates/azure-devops/publish-pipeline.test.ts‎

Lines changed: 70 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -147,8 +147,8 @@ describe('azure-devops/publish-pipeline', () => {
147147
});
148148
expect(pipeline).toContain('incremental - last commit only');
149149
expect(pipeline).toContain('all artifacts');
150-
expect(pipeline).toContain("ne('${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo')");
151-
expect(pipeline).toContain("eq('${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo')");
150+
expect(pipeline).toContain("and(succeeded(), ne('${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo'))");
151+
expect(pipeline).toContain("and(succeeded(), eq('${{ parameters.COMMIT_ID_CHOICE }}', 'publish-all-artifacts-in-repo'))");
152152
});
153153

154154
it('should pass Build.SourceVersion as commit-id in incremental step', () => {
@@ -255,5 +255,73 @@ describe('azure-devops/publish-pipeline', () => {
255255
expect(tokenIdx).toBeGreaterThan(0);
256256
expect(tokenIdx).toBeLessThan(publishIdx);
257257
});
258+
259+
it('should include dry-run validation steps before publish steps', () => {
260+
const pipeline = generatePublishPipeline({
261+
artifactDir: './apim-artifacts',
262+
environments: ['dev'],
263+
});
264+
expect(pipeline).toContain('Dry-run validation (dev, incremental)');
265+
expect(pipeline).toContain('Dry-run validation (dev, all artifacts)');
266+
});
267+
268+
it('should include --dry-run flag in dry-run validation steps', () => {
269+
const pipeline = generatePublishPipeline({
270+
artifactDir: './apim-artifacts',
271+
environments: ['dev'],
272+
});
273+
const lines = pipeline.split('\n');
274+
const dryRunIncrIdx = lines.findIndex((l) => l.includes('Dry-run validation (dev, incremental)'));
275+
const dryRunSection = lines.slice(dryRunIncrIdx, dryRunIncrIdx + 20).join('\n');
276+
expect(dryRunSection).toContain('--dry-run');
277+
});
278+
279+
it('should place dry-run validation before actual publish steps', () => {
280+
const pipeline = generatePublishPipeline({
281+
artifactDir: './apim-artifacts',
282+
environments: ['dev'],
283+
});
284+
const dryRunIdx = pipeline.indexOf('Dry-run validation (dev, incremental)');
285+
const publishIdx = pipeline.indexOf("Publish to dev (incremental");
286+
expect(dryRunIdx).toBeGreaterThan(0);
287+
expect(dryRunIdx).toBeLessThan(publishIdx);
288+
});
289+
290+
it('should include dry-run validation for each environment', () => {
291+
const pipeline = generatePublishPipeline({
292+
artifactDir: './apim-artifacts',
293+
environments: ['dev', 'prod'],
294+
});
295+
expect(pipeline).toContain('Dry-run validation (dev, incremental)');
296+
expect(pipeline).toContain('Dry-run validation (dev, all artifacts)');
297+
expect(pipeline).toContain('Dry-run validation (prod, incremental)');
298+
expect(pipeline).toContain('Dry-run validation (prod, all artifacts)');
299+
});
300+
301+
it('should pass commit-id in incremental dry-run step', () => {
302+
const pipeline = generatePublishPipeline({
303+
artifactDir: './apim-artifacts',
304+
environments: ['dev'],
305+
});
306+
const lines = pipeline.split('\n');
307+
const dryRunIncrIdx = lines.findIndex((l) => l.includes('Dry-run validation (dev, incremental)'));
308+
const nextTaskIdx = lines.findIndex((l, i) => i > dryRunIncrIdx + 1 && l.includes("- task:"));
309+
const dryRunSection = lines.slice(dryRunIncrIdx, nextTaskIdx).join('\n');
310+
expect(dryRunSection).toContain('--commit-id');
311+
expect(dryRunSection).toContain('--dry-run');
312+
});
313+
314+
it('should not pass commit-id in all-artifacts dry-run step', () => {
315+
const pipeline = generatePublishPipeline({
316+
artifactDir: './apim-artifacts',
317+
environments: ['dev'],
318+
});
319+
const lines = pipeline.split('\n');
320+
const dryRunAllIdx = lines.findIndex((l) => l.includes('Dry-run validation (dev, all artifacts)'));
321+
const nextTaskIdx = lines.findIndex((l, i) => i > dryRunAllIdx + 1 && l.includes("- task:"));
322+
const dryRunSection = lines.slice(dryRunAllIdx, nextTaskIdx).join('\n');
323+
expect(dryRunSection).not.toContain('--commit-id');
324+
expect(dryRunSection).toContain('--dry-run');
325+
});
258326
});
259327
});

‎tests/unit/templates/github-actions/publish-workflow.test.ts‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -265,5 +265,73 @@ describe('github-actions/publish-workflow', () => {
265265
expect(substituteIdx).toBeLessThan(validateSubstitutionIdx);
266266
expect(validateSubstitutionIdx).toBeLessThan(publishIdx);
267267
});
268+
269+
it('should include dry-run validation steps before publish steps', () => {
270+
const workflow = generatePublishWorkflow({
271+
artifactDir: './apim-artifacts',
272+
environments: ['dev'],
273+
});
274+
expect(workflow).toContain('Dry-run validation (dev, incremental)');
275+
expect(workflow).toContain('Dry-run validation (dev, all artifacts)');
276+
});
277+
278+
it('should include --dry-run flag in dry-run validation steps', () => {
279+
const workflow = generatePublishWorkflow({
280+
artifactDir: './apim-artifacts',
281+
environments: ['dev'],
282+
});
283+
const lines = workflow.split('\n');
284+
const dryRunIncrIdx = lines.findIndex((l) => l.includes('Dry-run validation (dev, incremental)'));
285+
const dryRunSection = lines.slice(dryRunIncrIdx, dryRunIncrIdx + 15).join('\n');
286+
expect(dryRunSection).toContain('--dry-run');
287+
});
288+
289+
it('should place dry-run validation before actual publish steps', () => {
290+
const workflow = generatePublishWorkflow({
291+
artifactDir: './apim-artifacts',
292+
environments: ['dev'],
293+
});
294+
const dryRunIdx = workflow.indexOf('Dry-run validation (dev, incremental)');
295+
const publishIdx = workflow.indexOf('Publish to dev (incremental');
296+
expect(dryRunIdx).toBeGreaterThan(0);
297+
expect(dryRunIdx).toBeLessThan(publishIdx);
298+
});
299+
300+
it('should include dry-run validation for each environment', () => {
301+
const workflow = generatePublishWorkflow({
302+
artifactDir: './apim-artifacts',
303+
environments: ['dev', 'prod'],
304+
});
305+
expect(workflow).toContain('Dry-run validation (dev, incremental)');
306+
expect(workflow).toContain('Dry-run validation (dev, all artifacts)');
307+
expect(workflow).toContain('Dry-run validation (prod, incremental)');
308+
expect(workflow).toContain('Dry-run validation (prod, all artifacts)');
309+
});
310+
311+
it('should pass commit-id in incremental dry-run step', () => {
312+
const workflow = generatePublishWorkflow({
313+
artifactDir: './apim-artifacts',
314+
environments: ['dev'],
315+
});
316+
const lines = workflow.split('\n');
317+
const dryRunIncrIdx = lines.findIndex((l) => l.includes('Dry-run validation (dev, incremental)'));
318+
const nextStepIdx = lines.findIndex((l, i) => i > dryRunIncrIdx + 1 && l.includes('- name:'));
319+
const dryRunSection = lines.slice(dryRunIncrIdx, nextStepIdx).join('\n');
320+
expect(dryRunSection).toContain('--commit-id');
321+
expect(dryRunSection).toContain('--dry-run');
322+
});
323+
324+
it('should not pass commit-id in all-artifacts dry-run step', () => {
325+
const workflow = generatePublishWorkflow({
326+
artifactDir: './apim-artifacts',
327+
environments: ['dev'],
328+
});
329+
const lines = workflow.split('\n');
330+
const dryRunAllIdx = lines.findIndex((l) => l.includes('Dry-run validation (dev, all artifacts)'));
331+
const nextStepIdx = lines.findIndex((l, i) => i > dryRunAllIdx + 1 && l.includes('- name:'));
332+
const dryRunSection = lines.slice(dryRunAllIdx, nextStepIdx).join('\n');
333+
expect(dryRunSection).not.toContain('--commit-id');
334+
expect(dryRunSection).toContain('--dry-run');
335+
});
268336
});
269337
});

0 commit comments

Comments
 (0)