Initial AI validator script (Draft)

ai-guide-validator/prompts/callout-section-validator-prompt.md

@@ -0,0 +1,38 @@
+# Callout Section Validator Prompt
+
+## How to use:
+1. Copy the entire content of this file.
+2. Paste it into your LLM tool (e.g., ChatGPT, Claude, Cursor AI).
+3. Replace <PASTE FILE CONTENT HERE> with the markdown content you want to validate.
+4. Review the output for any violations and suggestions.
+
+---
+
+You are a documentation style guide validator. Validate the following markdown file for callout formatting rules.
+
+**Rules:**
+- All callouts (Note, Caution, Important, Tip, Warning, etc.) must use the following format:
+  ```
+  > [!TYPE]
+  > Your message here.
+  ```
+  Where TYPE is one of: NOTE, CAUTION, IMPORTANT, TIP, WARNING (case-insensitive).
+- Do not use other callout formats (e.g., "**Note:**", ":warning:", etc.).
+- Warn if the callout type is not recognized.
+
+**Output format:**
+```markdown
+<details><summary>Callout Section [PASS/FAIL]</summary>
+
+| Criteria | Pass/Fail/Warning | Location | Suggestions |
+|----------|-------------------|----------|-------------|
+| All callouts use correct format | [✅/❌] | [line/section] | [Fail if not] |
+| Only allowed callout types used | [✅/⚠️] | [line/section] | [Warn if not] |
+
+</details>
+```
+
+**Markdown file to validate:**
+```
+<PASTE FILE CONTENT HERE>
+```

ai-guide-validator/prompts/overview-section-validator-prompt.md

@@ -0,0 +1,38 @@
+# Overview Section Validator Prompt
+
+## How to use:
+1. Copy the entire content of this file.
+2. Paste it into your LLM tool (e.g., ChatGPT, Claude, Cursor AI).
+3. Replace <PASTE FILE CONTENT HERE> with the markdown content you want to validate.
+4. Review the output for any violations and suggestions.
+
+---
+
+You are a documentation style guide validator. Validate the following markdown file for the "Overview Section" rules.
+
+**Rules:**
+- The overview section is defined as all content after the H1 and before the first `## Prerequisites` (or next H2).
+- The overview must start with: `In this guide, you will:`
+- The overview must use second person ("you"/"your").
+- The overview should contain at least one hyperlink to a related topic (raise a warning if missing).
+- The overview can be a list or paragraph.
+
+**Output format:**
+```markdown
+<details><summary>Overview Section [PASS/FAIL]</summary>
+
+| Criteria | Pass/Fail/Warning | Suggestions |
+|----------|-------------------|-------------|
+| Overview content exists between H1 and `## Prerequisites` | [✅/❌] | [Fail if missing] |
+| Overview starts with “In this guide, you will:” | [✅/❌] | [Fail if missing] |
+| Uses second person | [✅/❌] | [Fail if not using “you”/“your”] |
+| Contains at least one hyperlink | [✅/⚠️] | [Warn if missing] |
+| Overview is a list or paragraph | [✅/❌] | [Fail if not] |
+
+</details>
+```
+
+**Markdown file to validate:**
+```
+<PASTE FILE CONTENT HERE>
+```

ai-guide-validator/prompts/prerequisites-section-validator-prompt.md

@@ -0,0 +1,42 @@
+# Prerequisites Section Validator Prompt
+
+## How to use:
+1. Copy the entire content of this file.
+2. Paste it into your LLM tool (e.g., ChatGPT, Claude, Cursor AI).
+3. Replace <PASTE FILE CONTENT HERE> with the markdown content you want to validate.
+4. Review the output for any violations and suggestions.
+
+---
+
+You are a documentation style guide validator. Validate the following markdown file for the "Prerequisites Section" rules.
+
+**Rules:**
+- There must be a `## Prerequisites` section.
+- The `## Prerequisites` section must come after the overview section (i.e., after the content between the H1 and the first H2).
+- The prerequisites must be presented as a bulleted list.
+- Simple items in the list must NOT end with periods.
+- Sentences or fragments in the list must end with periods.
+- The section should only include items relevant to setup or access (not steps or instructions).
+- The section should use second person ("you"/"your") where appropriate (warn if not, but do not fail).
+
+**Output format:**
+```markdown
+<details><summary>Prerequisites Section [PASS/FAIL]</summary>
+
+| Criteria | Pass/Fail/Warning | Suggestions |
+|----------|-------------------|-------------|
+| `## Prerequisites` section present | [✅/❌] | [Fail if missing] |
+| Section comes after overview | [✅/❌] | [Fail if not] |
+| Prerequisites are a bulleted list | [✅/❌] | [Fail if not] |
+| Simple items do not end with periods | [✅/❌] | [Fail if present] |
+| Sentences/fragments end with periods | [✅/❌] | [Fail if missing] |
+| Only setup/access items included | [✅/❌] | [Fail if steps/instructions present] |
+| Uses second person where appropriate | [✅/⚠️] | [Warn if not] |
+
+</details>
+```
+
+**Markdown file to validate:**
+```
+<PASTE FILE CONTENT HERE>
+```

ai-guide-validator/prompts/steps-section-validator-prompt.md

@@ -0,0 +1,57 @@
+# Steps Section Validator Prompt
+
+## How to use:
+1. Copy the entire content of this file.
+2. Paste it into your LLM tool (e.g., ChatGPT, Claude, Cursor AI).
+3. Replace <PASTE FILE CONTENT HERE> with the markdown content you want to validate.
+4. Review the output for any violations and suggestions.
+
+---
+
+You are a documentation style guide validator. Validate the following markdown file for the "Steps Section" rules.
+
+**Rules:**
+- There must be at least one step section, starting with `## Step 1:`.
+- Each step uses an H2 header (`##`).
+- Steps are sequentially numbered, with no missing or duplicate numbers.
+- Each step header is in the format "Step {n}: Verb ..." in Title Case.
+- The first word after the colon is a verb.
+- Each step has at least one line of content.
+- Steps come before Review, Next Steps, and Troubleshooting sections.
+- The review section must be titled exactly `## Review` (not `## Step Review` or any variant).
+- Step instructions use second person ("you"/"your") where appropriate (warn if not, but do not fail).
+- Steps may include images or callouts.
+
+**If the document does not use H2 step headers (## Step n: ...), or otherwise does not follow the standard guide structure, mark the section as FAIL and clearly state this in the summary and suggestions.**
+
+**For each failure or warning, indicate the line number(s) and/or section header(s) where the issue occurs in the "Location" column.**
+
+**Output format:**
+```markdown
+<details><summary>Steps Section [PASS/FAIL]</summary>
+
+[If the overall format does not match the standard (e.g., missing H2 step headers), include a summary statement here, e.g.:]
+**Major Issue:**
+The steps section does not follow the required standard format. Steps are not defined using H2 headers (## Step n: ...). Please rewrite the steps to use H2 headers for each step.
+
+| Criteria | Pass/Fail/Warning | Location | Suggestions |
+|----------|-------------------|----------|-------------|
+| At least one step section present | [✅/❌] | [line/section] | [Fail if missing] |
+| Step headers are H2 | [✅/❌] | [line/section] | [Fail if not] |
+| Step headers are sequential | [✅/❌] | [line/section] | [Fail if missing/duplicate numbers] |
+| Step header format | [✅/❌] | [line/section] | [Fail if not "Step n: Verb ..."] |
+| Step header starts with verb | [✅/❌] | [line/section] | [Fail if not] |
+| Step content is present | [✅/❌] | [line/section] | [Fail if missing] |
+| Steps are before Review/Next Steps/Troubleshooting | [✅/❌] | [line/section] | [Fail if not] |
+| No skipped or duplicate step numbers | [✅/❌] | [line/section] | [Fail if not] |
+| Uses second person where appropriate | [✅/⚠️] | [line/section] | [Warn if not] |
+| Images/callouts are allowed | [✅] | - |  |
+| Review section is titled exactly '## Review' | [✅/❌] | [line/section] | [Fail if not] |
+
+</details>
+```
+
+**Markdown file to validate:**
+```
+
+```

ai-guide-validator/prompts/troubleshooting-section-validator-prompt.md

@@ -0,0 +1,34 @@
+# Troubleshooting Section Validator Prompt
+
+## How to use:
+1. Copy the entire content of this file.
+2. Paste it into your LLM tool (e.g., ChatGPT, Claude, Cursor AI).
+3. Replace <PASTE FILE CONTENT HERE> with the markdown content you want to validate.
+4. Review the output for any violations and suggestions.
+
+---
+
+You are a documentation style guide validator. Validate the following markdown file for the "Troubleshooting" section rules.
+
+**Rules:**
+- There must be a `## Troubleshooting` section. (Fail if missing)
+- The section should be in markdown table format. (Warn if not)
+- The last row of the table must mention "Further Assistance". (Fail if missing or not last)
+
+**Output format:**
+```markdown
+<details><summary>Troubleshooting Section [PASS/FAIL]</summary>
+
+| Criteria | Pass/Fail/Warning | Suggestions |
+|----------|-------------------|-------------|
+| `## Troubleshooting` section present | [✅/❌] | [Fail if missing] |
+| Section is in markdown table format | [✅/⚠️] | [Warn if not] |
+| Last row mentions Further Assistance | [✅/❌] | [Fail if missing or not last] |
+
+</details>
+```
+
+**Markdown file to validate:**
+```
+<PASTE FILE CONTENT HERE>
+```

ai-guide-validator/scripts/validate_guides.py

@@ -0,0 +1,94 @@
+import sys
+import os
+import anthropic
+
+"""
+Instructions:
+- Install the Anthropic SDK: pip install anthropic
+- Set your API key: export ANTHROPIC_API_KEY=sk-ant-...
+
+This script validates a markdown guide file using a series of documentation style guide validators.
+It sends the guide content to Anthropic's Claude LLM for each validator prompt and prints the results.
+"""
+
+# Directory containing the validator prompt files (relative to this script)
+PROMPT_DIR = '../prompts'
+
+# List of validators to run, in order: (Display Name, Prompt Filename)
+VALIDATORS = [
+    ('Overview', 'overview-section-validator-prompt.md'),
+    ('Prerequisites', 'prerequisites-section-validator-prompt.md'),
+    ('Steps', 'steps-section-validator-prompt.md'),
+    ('Troubleshooting', 'troubleshooting-section-validator-prompt.md'),
+    ('Callout', 'callout-section-validator-prompt.md'),
+]
+
+# Initialize the Anthropic client using the API key from the environment
+client = anthropic.Anthropic(
+    api_key=os.environ.get("ANTHROPIC_API_KEY")
+)
+
+def read_file(path):
+    """
+    Read the entire contents of a file and return as a string.
+    """
+    with open(path, 'r', encoding='utf-8') as f:
+        return f.read()
+
+def inject_guide_into_prompt(prompt, guide_content):
+    """
+    Replace the <PASTE FILE CONTENT HERE> placeholder in the validator prompt
+    with the actual guide markdown content.
+    """
+    return prompt.replace('<PASTE FILE CONTENT HERE>', guide_content)
+
+def run_validator_with_claude(prompt):
+    """
+    Send the prompt to Anthropic Claude and return the main text content of the response.
+    """
+    response = client.messages.create(
+        model="claude-3-opus-20240229",
+        max_tokens=2048,
+        temperature=0,
+        messages=[
+            {"role": "user", "content": prompt}
+        ]
+    )
+    # Return the main text content (first content block)
+    return response.content[0].text if response.content else "[No response]"
+
+def main():
+    """
+    Main execution flow:
+    - Parse command line arguments for the guide file path
+    - For each validator:
+        - Read the validator prompt
+        - Inject the guide content
+        - Send to Claude and print the result
+    """
+    if len(sys.argv) < 2:
+        print('Usage: python scripts/validate_guides.py <guide_markdown_file>')
+        sys.exit(1)
+    guide_path = sys.argv[1]
+    if not os.path.isfile(guide_path):
+        print(f'Guide file not found: {guide_path}')
+        sys.exit(1)
+    guide_content = read_file(guide_path)
+
+    print(f'Validating guide: {guide_path}\n')
+    for name, prompt_file in VALIDATORS:
+        prompt_path = os.path.join(PROMPT_DIR, prompt_file)
+        if not os.path.isfile(prompt_path):
+            print(f'Validator prompt not found: {prompt_path}')
+            continue
+        prompt_content = read_file(prompt_path)
+        # Insert the guide markdown into the validator prompt
+        full_prompt = inject_guide_into_prompt(prompt_content, guide_content)
+        print(f'---\n{name} Validator Prompt:\n---')
+        # Send the prompt to Claude and print the result
+        result = run_validator_with_claude(full_prompt)
+        print(result)
+        print('\n')
+
+if __name__ == '__main__':
+    main()

ai-guide-validator/template-reference/guardrails-style-guide-validator.md

@@ -0,0 +1,59 @@
+# Turbot Guardrails Style Guide Validator Prompt
+
+This prompt is designed for use with AI tools (such as Cursor AI, GitHub Copilot Chat, or ChatGPT) to validate Turbot Guardrails documentation guides against the [base-style-guide](../base-format/base-style-guide.md).
+
+## How to Use
+
+1. Open the markdown file you want to validate (e.g., a guide in `docs/guides/`).
+2. Copy the entire content of this prompt.
+3. Paste it into your AI tool (e.g., Cursor AI, Copilot Chat, ChatGPT).
+4. Replace `<PASTE FILE CONTENT HERE>` with the content of your markdown file.
+5. Review the output for any violations and suggested corrections.
+
+---
+
+## Validator Prompt
+
+You are a documentation style guide validator for Turbot Guardrails. Given a markdown file from the `guardrails-docs` repo, check if it adheres to the following style guide rules (summarized from [base-style-guide.md](../base-format/base-style-guide.md)):
+
+### General
+- Uses second person ("you") for instructions.
+- Uses markdown syntax for images and tables where possible.
+- File and directory names are in kebab-case.
+- Images are stored in the same directory as the markdown file that uses them.
+
+### Structure
+- The file starts with an H1 header matching the `title` and `sidebar_label` in the frontmatter.
+- The overview is a paragraph after the title, with no "## Overview" header.
+- Frontmatter includes `title` and `sidebar_label`, usually the same.
+- All headers use Title Case. Steps are H2 headers in the format "Step {n}: Verb ...".
+- Each guide includes: Overview, Prerequisites, Steps, Review, Next Steps, Troubleshooting (with Troubleshooting in a table).
+
+### Formatting
+- Bulleted lists use periods for sentences/fragments, not for simple items.
+- Markdown tables preferred over HTML.
+- Images: PNG, light mode, markdown syntax.
+- Links: no repeated links in the same section.
+- Code blocks specify the language.
+- Blockquotes use GitHub callout types (e.g., > [!NOTE]).
+
+### Steps
+- Each step: H2, "Step {n}: Title", verb at start.
+- "Review" as the final step.
+
+### Troubleshooting
+- Tabular format.
+
+---
+
+**Instructions:**
+1. List any violations of the above rules, referencing the line number and content.
+2. Suggest corrections for each violation.
+3. If the file fully adheres to the style guide, respond: "PASS: No violations found."
+
+---
+
+**Markdown file to validate:**
+```
+<PASTE FILE CONTENT HERE>
+```