Skip to content

Commit eaf76f4

Browse files
authored
feat: address PR #183 review feedback - Copilot prompt refinements and docs
- Add links to prompt template source files for users who skipped `apiops init` in docs/guides/environment-overrides.md and docs/guides/filtering-resources.md - Remove artificial word wrapping from configure-filter-prompt.md and configure-overrides-prompt.md (let editors handle text wrap) - configure-overrides-prompt.md Step 3: inspect pipeline files first to discover existing variables before asking the user - configure-overrides-prompt.md Step 3: run `gh --version` and offer to auto-query `gh variable list`/`gh secret list` (names only) with permission - configure-overrides-prompt.md Step 3: validate user-provided token names against discovered pipeline variables (warn on mismatch) - configure-overrides-prompt.md Step 5: structural YAML schema validation (top-level keys, name+properties shape, obvious misspellings) - configure-overrides-prompt.md Step 5: replace vague "show" with ask-user- to-review-before-checking-in instruction - Prompt naming (apiops- prefix) already implemented; no change needed Closes #195
1 parent 6b7b61c commit eaf76f4

4 files changed

Lines changed: 115 additions & 276 deletions

File tree

docs/guides/environment-overrides.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,8 @@ apiops publish \
2727

2828
If you ran `apiops init`, a Copilot prompt file was generated at `.github/prompts/apiops-configure-overrides.prompt.md`. Open it in VS Code and ask GitHub Copilot to help you configure environment overrides — it will guide you through setting up environment-specific values interactively.
2929

30+
If you didn't run `apiops init`, you can copy the prompt directly from the repository: [`src/templates/copilot/configure-overrides-prompt.md`](https://github.com/Azure/apiops-cli/blob/main/src/templates/copilot/configure-overrides-prompt.md). Save it as `.github/prompts/apiops-configure-overrides.prompt.md` in your repo and open it in VS Code.
31+
3032
## IDE Autocompletion with JSON Schema
3133

3234
A JSON Schema is available for `configuration.{env}.yaml` override files. Add yaml-language-server comment at the top of your override file. Requires yaml language extension in VSCode.

docs/guides/filtering-resources.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -354,3 +354,5 @@ backends:
354354
## Copilot-Assisted Configuration
355355

356356
If you ran `apiops init`, a Copilot prompt file was generated at `.github/prompts/apiops-configure-filter.prompt.md`. Open it in VS Code and ask GitHub Copilot to help you configure your filter — it will walk you through selecting resources interactively.
357+
358+
If you didn't run `apiops init`, you can copy the prompt directly from the repository: [`src/templates/copilot/configure-filter-prompt.md`](https://github.com/Azure/apiops-cli/blob/main/src/templates/copilot/configure-filter-prompt.md). Save it as `.github/prompts/apiops-configure-filter.prompt.md` in your repo and open it in VS Code.

src/templates/copilot/configure-filter-prompt.md

Lines changed: 43 additions & 121 deletions
Original file line numberDiff line numberDiff line change
@@ -5,180 +5,103 @@ description: 'Configure APIOps resource extraction filters'
55

66
# Configure APIOps Extractor Filters
77

8-
> **How to use:** Open this file in VS Code with GitHub Copilot and ask
9-
> Copilot to help you design a `configuration.extractor.yaml` file for your
10-
> repository.
8+
> **How to use:** Open this file in VS Code with GitHub Copilot and ask Copilot to help you design a `configuration.extractor.yaml` file for your repository.
119
1210
## Goal
1311

14-
Create a `configuration.extractor.yaml` file that limits APIOps extraction to
15-
the Azure API Management resources your team wants to manage in source control.
12+
Create a `configuration.extractor.yaml` file that limits APIOps extraction to the Azure API Management resources your team wants to manage in source control.
1613

1714
---
1815

1916
## How Copilot must work through this prompt
2017

2118
These rules apply to **every** step below. Follow them strictly:
2219

23-
1. **Confirm before proceeding.** At the end of every step, summarize what you
24-
learned or propose, then **STOP and wait for the user to confirm** before
25-
moving to the next step. Never chain steps together without an explicit
26-
"yes" / "go ahead" from the user.
27-
- **Hard stop rule:** When you ask for confirmation, end the response there.
28-
Do **not** include the next question, next resource type, or any forward
29-
action in the same message.
30-
- This hard stop applies to **step boundaries** (Step 0, Step 2, Step 3,
31-
Step 4). In Step 1, follow the single-resource-type cadence below.
32-
2. **Never assume or invent names.** Do not invent API, product, backend,
33-
named value, or any other resource names. Use only names that come from the
34-
live APIM instance or that the user explicitly provides. The local artifact
35-
directory is not authoritative — it may be stale or empty. When unsure, ask.
36-
3. **Default is extract-everything.** APIOps extracts **all** resources of a
37-
type when that type is **omitted** from the filter. Only add a type to the
38-
filter when the user wants to narrow it (SOME) or exclude it (NONE). Do not
39-
add a type just to list every resource.
40-
4. **Empty array means exclude all.** Setting a type to `[]` excludes every
41-
resource of that type. Use this only when the user explicitly wants NONE.
42-
5. **The JSON schema is the source of structure.** To determine which resource
43-
types support sub-entries and what those sub-entries are (for example,
44-
`apis``operations`, `diagnostics`, `schemas`, `releases`), consult the
45-
`extractor-config` JSON schema referenced in the file's
46-
`# yaml-language-server: $schema=...` comment (the public schema URL).
20+
1. **Confirm before proceeding.** At the end of every step, summarize what you learned or propose, then **STOP and wait for the user to confirm** before moving to the next step. Never chain steps together without an explicit "yes" / "go ahead" from the user.
21+
- **Hard stop rule:** When you ask for confirmation, end the response there. Do **not** include the next question, next resource type, or any forward action in the same message.
22+
- This hard stop applies to **step boundaries** (Step 0, Step 2, Step 3, Step 4). In Step 1, follow the single-resource-type cadence below.
23+
2. **Never assume or invent names.** Do not invent API, product, backend, named value, or any other resource names. Use only names that come from the live APIM instance or that the user explicitly provides. The local artifact directory is not authoritative — it may be stale or empty. When unsure, ask.
24+
3. **Default is extract-everything.** APIOps extracts **all** resources of a type when that type is **omitted** from the filter. Only add a type to the filter when the user wants to narrow it (SOME) or exclude it (NONE). Do not add a type just to list every resource.
25+
4. **Empty array means exclude all.** Setting a type to `[]` excludes every resource of that type. Use this only when the user explicitly wants NONE.
26+
5. **The JSON schema is the source of structure.** To determine which resource types support sub-entries and what those sub-entries are (for example, `apis``operations`, `diagnostics`, `schemas`, `releases`), consult the `extractor-config` JSON schema referenced in the file's `# yaml-language-server: $schema=...` comment (the public schema URL).
4727

4828
---
4929

5030
## Step 0 — Determine the Authoritative Resource List
5131

52-
The filter runs at **extraction time against the live Azure API Management
53-
instance**. The local artifact directory may be stale, partial, or empty, so it
54-
is **not** an authoritative list of what exists in Azure. Establish the source
55-
of truth first:
56-
57-
1. **Prefer querying the live APIM instance.** Ask the user for (or reuse if
58-
already known) the subscription ID, resource group, and APIM service name,
59-
and whether the Azure CLI is logged in. If Azure is reachable, enumerate the
60-
resource types and names directly from the instance (for example with
61-
`az apim` / `az rest` calls) and use that as the source of truth.
62-
2. **Fallback when Azure cannot be queried.** Do **not** treat the local
63-
artifacts as the definitive list. Instead, in Step 1 ask the user
64-
type-by-type; for SOME, the user must provide the resource (and
65-
sub-resource) names themselves.
66-
3. Check whether `configuration.extractor.yaml` already exists (it may have
67-
been created by `apiops init`). If it exists, note its current contents —
68-
you will update it in place rather than overwriting it.
69-
70-
Tell the user which mode you will use (live-Azure list vs. user-provided
71-
names), and confirm the connection details if querying Azure.
32+
The filter runs at **extraction time against the live Azure API Management instance**. The local artifact directory may be stale, partial, or empty, so it is **not** an authoritative list of what exists in Azure. Establish the source of truth first:
33+
34+
1. **Prefer querying the live APIM instance.** Ask the user for (or reuse if already known) the subscription ID, resource group, and APIM service name, and whether the Azure CLI is logged in. If Azure is reachable, enumerate the resource types and names directly from the instance (for example with `az apim` / `az rest` calls) and use that as the source of truth.
35+
2. **Fallback when Azure cannot be queried.** Do **not** treat the local artifacts as the definitive list. Instead, in Step 1 ask the user type-by-type; for SOME, the user must provide the resource (and sub-resource) names themselves.
36+
3. Check whether `configuration.extractor.yaml` already exists (it may have been created by `apiops init`). If it exists, note its current contents — you will update it in place rather than overwriting it.
37+
38+
Tell the user which mode you will use (live-Azure list vs. user-provided names), and confirm the connection details if querying Azure.
7239

7340
**STOP. Do not proceed until the user confirms the source of truth.**
7441

7542
---
7643

7744
## Step 1 — Decide Scope Per Resource Type (one type at a time)
7845

79-
Walk through the resource types **one type at a time**. For each type, ask the
80-
user which scope they want:
46+
Walk through the resource types **one type at a time**. For each type, ask the user which scope they want:
8147

82-
- **Extract ALL** — include every resource of this type. Leave this type
83-
**out** of the filter (APIOps extracts everything by default).
84-
- **Extract NONE** — exclude all resources of this type. Add the type with an
85-
empty array: `tags: []`.
86-
- **Extract SOME** — include only specific resources. The user provides which
87-
names (or wildcard patterns) to include. Matching is case-insensitive and
88-
supports `*` and `?` wildcards.
48+
- **Extract ALL** — include every resource of this type. Leave this type **out** of the filter (APIOps extracts everything by default).
49+
- **Extract NONE** — exclude all resources of this type. Add the type with an empty array: `tags: []`.
50+
- **Extract SOME** — include only specific resources. The user provides which names (or wildcard patterns) to include. Matching is case-insensitive and supports `*` and `?` wildcards.
8951

9052
**Single-resource-type cadence for Step 1:**
9153

92-
- Ask about exactly **one** resource type at a time. Do not batch multiple
93-
types into one prompt.
94-
- **Ask ALL / NONE / SOME first.** Do **not** enumerate or query any names up
95-
front. For ALL or NONE, record the answer and move on — no enumeration is
96-
needed.
54+
- Ask about exactly **one** resource type at a time. Do not batch multiple types into one prompt.
55+
- **Ask ALL / NONE / SOME first.** Do **not** enumerate or query any names up front. For ALL or NONE, record the answer and move on — no enumeration is needed.
9756
- **Only when the user answers SOME**, then gather names:
98-
- If you can query the live APIM instance, list that type's names from Azure
99-
to help the user choose.
100-
- Otherwise, ask the user to provide the names/patterns. Do **not** invent
101-
names or pull them from the local artifacts.
102-
- When the user answers a type unambiguously, record the decision and move to
103-
the next type.
104-
- **Update `configuration.extractor.yaml` immediately after each decision that
105-
affects the file** (SOME or NONE adds/updates that type's section; ALL means
106-
no change since the type is omitted). Keep the file in sync as you go rather
107-
than waiting until the end.
57+
- If you can query the live APIM instance, list that type's names from Azure to help the user choose.
58+
- Otherwise, ask the user to provide the names/patterns. Do **not** invent names or pull them from the local artifacts.
59+
- When the user answers a type unambiguously, record the decision and move to the next type.
60+
- **Update `configuration.extractor.yaml` immediately after each decision that affects the file** (SOME or NONE adds/updates that type's section; ALL means no change since the type is omitted). Keep the file in sync as you go rather than waiting until the end.
10861
- Only pause for clarification when the answer is ambiguous.
10962

110-
Resource types to consider (ask only about types that exist in the
111-
instance/artifacts or that the user mentions):
63+
Resource types to consider (ask only about types that exist in the instance/artifacts or that the user mentions):
11264

113-
`apis`, `products`, `namedValues`, `backends`, `loggers`, `diagnostics`,
114-
`tags`, `versionSets`, `policyFragments`, `gateways`, `groups`,
115-
`subscriptions`, `schemas`, `policies`, `workspaces`.
65+
`apis`, `products`, `namedValues`, `backends`, `loggers`, `diagnostics`, `tags`, `versionSets`, `policyFragments`, `gateways`, `groups`, `subscriptions`, `schemas`, `policies`, `workspaces`.
11666

117-
> **APIs can be filtered at the sub-resource level.** Whenever `apis` is SOME
118-
> and specific API names are listed in the filter, **ask about each listed
119-
> API's sub-resources**`operations`, `diagnostics`, `schemas`, and
120-
> `releases`. The user may want everything for that API, or only a subset
121-
> (for example, a single revision or release). Omit a sub-filter to include
122-
> all of that sub-type; set it to `[]` to exclude all.
67+
> **APIs can be filtered at the sub-resource level.** Whenever `apis` is SOME and specific API names are listed in the filter, **ask about each listed API's sub-resources**`operations`, `diagnostics`, `schemas`, and `releases`. The user may want everything for that API, or only a subset (for example, a single revision or release). Omit a sub-filter to include all of that sub-type; set it to `[]` to exclude all.
12368
124-
> **Workspaces apply only if the APIM instance uses workspaces.** Skip this
125-
> type entirely if there are no workspaces. When a user wants SOME for
126-
> `workspaces`, each workspace can also be narrowed by its own sub-resources
127-
> (`apis`, `backends`, `diagnostics`, `groups`, `loggers`, `namedValues`,
128-
> `policyFragments`, `products`, `schemas`, `subscriptions`, `tags`,
129-
> `versionSets`). Omit a sub-filter to include all of that sub-type; set it to
130-
> `[]` to exclude all. Only offer this depth if the user wants it.
69+
> **Workspaces apply only if the APIM instance uses workspaces.** Skip this type entirely if there are no workspaces. When a user wants SOME for `workspaces`, each workspace can also be narrowed by its own sub-resources (`apis`, `backends`, `diagnostics`, `groups`, `loggers`, `namedValues`, `policyFragments`, `products`, `schemas`, `subscriptions`, `tags`, `versionSets`). Omit a sub-filter to include all of that sub-type; set it to `[]` to exclude all. Only offer this depth if the user wants it.
13170
132-
> **Service-level `policies` is effectively a single global policy.** For this
133-
> type, ask only **include (ALL)** or **exclude (NONE)** — SOME does not apply.
71+
> **Service-level `policies` is effectively a single global policy.** For this type, ask only **include (ALL)** or **exclude (NONE)** — SOME does not apply.
13472
135-
After all types are decided, summarize the per-type decisions and **STOP for
136-
confirmation** before generating YAML.
73+
After all types are decided, summarize the per-type decisions and **STOP for confirmation** before generating YAML.
13774

13875
---
13976

14077
## Step 2 — Propose a Filter Strategy
14178

14279
Based on the recorded decisions:
14380

144-
1. Recommend the smallest filter that safely captures the intended scope
145-
(remember: omitted types are fully extracted, so only NONE/SOME types
146-
appear in the file).
81+
1. Recommend the smallest filter that safely captures the intended scope (remember: omitted types are fully extracted, so only NONE/SOME types appear in the file).
14782
2. Explain any tradeoffs between broad and narrow filters.
148-
3. Call out any risk of accidentally excluding required dependencies — for
149-
example, excluding a named value or backend that an included API's policy
150-
references.
83+
3. Call out any risk of accidentally excluding required dependencies — for example, excluding a named value or backend that an included API's policy references.
15184

152-
If the user is unsure, recommend a conservative filter that is easy to refine,
153-
then **STOP for confirmation**.
85+
If the user is unsure, recommend a conservative filter that is easy to refine, then **STOP for confirmation**.
15486

15587
---
15688

15789
## Step 3 — Generate `configuration.extractor.yaml`
15890

159-
> **Note:** The file `configuration.extractor.yaml` may already exist if the
160-
> user ran `apiops init`. If it exists, **update it in place** rather than
161-
> overwriting unrelated content.
91+
> **Note:** The file `configuration.extractor.yaml` may already exist if the user ran `apiops init`. If it exists, **update it in place** rather than overwriting unrelated content.
16292
16393
Create the YAML file content reflecting the confirmed decisions.
16494

16595
Requirements:
16696

167-
- **Preserve the existing schema comment.** If the file already has a
168-
`# yaml-language-server: $schema=...` line (as `apiops init` generates), keep
169-
it **exactly as-is** — it already points at the correct schema version. Only
170-
if the file has **no** schema comment, add one referencing the current schema
171-
version:
97+
- **Preserve the existing schema comment.** If the file already has a `# yaml-language-server: $schema=...` line (as `apiops init` generates), keep it **exactly as-is** — it already points at the correct schema version. Only if the file has **no** schema comment, add one referencing the current schema version:
17298
`# yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/apiops-cli/main/schemas/v1/extractor-config.schema.json`
17399
- Output valid YAML.
174-
- Only include resource types the user chose to narrow (SOME) or exclude
175-
(NONE). Leave ALL types out of the file.
176-
- Use only names/patterns that exist in the artifacts or that the user
177-
provided — do not invent names.
100+
- Only include resource types the user chose to narrow (SOME) or exclude (NONE). Leave ALL types out of the file.
101+
- Use only names/patterns that exist in the artifacts or that the user provided — do not invent names.
178102
- Add a short comment only when it explains a non-obvious choice.
179103

180-
Show the generated file and **STOP for confirmation** before treating it as
181-
final.
104+
Show the generated file and **STOP for confirmation** before treating it as final.
182105

183106
---
184107

@@ -187,9 +110,8 @@ final.
187110
Before finishing:
188111

189112
1. Review the generated YAML for syntax issues and schema validity.
190-
2. Confirm the filters align with the user's intended extraction scope, and
191-
that no type the user wanted is accidentally excluded or fully extracted.
113+
2. Confirm the filters align with the user's intended extraction scope, and that no type the user wanted is accidentally excluded or fully extracted.
192114
3. Remind the user to run the extractor and inspect the artifact output.
193115

194-
If the extractor output is too broad or too narrow, help the user refine the
195-
filter file iteratively.
116+
If the extractor output is too broad or too narrow, help the user refine the filter file iteratively.
117+

0 commit comments

Comments
 (0)