feat: configuration hygiene (secrets migration, ignore list, validation config)

README.md

@@ -63,17 +63,40 @@ You can configure code-to-docs behavior with a JSON config file in your reposito
 | Key | Description | Example |
 |-----|-------------|---------|
 | `pr-title-prefix` | Prefix prepended to all PR titles and commit messages created by the tool | `":book:"` |
+| `validation.removal-threshold` | Fraction of original lines that can be removed before flagging (0.0 to 1.0) | `0.30` |
+| `validation.min-lines` | Files shorter than this skip the ratio-based preservation check | `20` |
+| `validation.llm-verification` | Set to `false` to skip the independent LLM verification call | `false` |
 
 **Example `.code-to-docs/config.json`:**
 ```json
 {
-  "pr-title-prefix": ":book:"
+  "pr-title-prefix": ":book:",
+  "validation": {
+    "removal-threshold": 0.30,
+    "min-lines": 20,
+    "llm-verification": true
+  }
 }
 ```
 
-With this config, generated PRs will be titled `:book: docs: update documentation from PR #123` instead of `docs: update documentation from PR #123`.
+This file is optional. Missing or invalid values fall back to defaults with a warning.
 
-This file is optional — if missing, the tool uses default titles with no prefix.
+## Excluding Files
+
+You can exclude files from AI analysis by creating `.code-to-docs/ignore` in your repository root. This file uses gitignore-style glob patterns, one per line:
+
+```
+# Generated API reference (managed by a separate tool)
+generated/*
+
+# Landing page (hand-crafted, should not be auto-updated)
+docs/index.md
+
+# Vendored docs
+vendor/**/*.md
+```
+
+Excluded files are never selected, never read into an LLM prompt, and are omitted from index-based selection results. The file is loaded from the base branch, consistent with other `.code-to-docs/` configuration.
 
 ## How It Works
 
@@ -160,14 +183,14 @@ jobs:
           pr-number: ${{ github.event.issue.number }}
           pr-base: origin/${{ steps.pr_info.outputs.base_ref || 'main' }}
           pr-head-sha: ${{ steps.pr_info.outputs.head_ref }}
-          docs-subfolder: ${{ secrets.DOCS_SUBFOLDER }}
+          docs-subfolder: 'docs'                         # Optional: path to docs within the same repo
           comment-body: ${{ github.event.comment.body }}
-          docs-base-branch: ${{ secrets.DOCS_BASE_BRANCH || 'main' }}
+          docs-base-branch: 'main'                          # Optional: base branch for docs PRs
           jira-url: ${{ secrets.JIRA_URL }}
           jira-username: ${{ secrets.JIRA_USERNAME }}
           jira-api-token: ${{ secrets.JIRA_API_TOKEN }}
           google-sa-key: ${{ secrets.GOOGLE_SA_KEY }}
-          max-context-chars: ${{ secrets.MAX_CONTEXT_CHARS }}
+          max-context-chars: '400000'                       # Optional: decrease for small-context models
           style-config-path: '.code-to-docs/style.md'
 ```
 
@@ -181,22 +204,24 @@ Add these in **Settings → Secrets → Actions**:
 | `MODEL_API_KEY` | API key for the model endpoint (leave empty if not required) |
 | `MODEL_NAME` | Model name to use (e.g., `meta-llama/Llama-3.1-8B-Instruct`, `gemini-2.0-flash`) |
 | `DOCS_REPO_URL` | Docs repository URL (e.g., `https://github.com/org/docs`) |
-| `GH_PAT` | _(Optional)_ GitHub PAT with `repo` scope. Only needed for **separate docs repos** (`docs-repo-url` pointing to a different repo). For same-repo setups, the built-in `GITHUB_TOKEN` works — no PAT required. |
-| `DOCS_SUBFOLDER` | _(Optional)_ Docs subfolder path (e.g., `docs`) |
-| `DOCS_BASE_BRANCH` | _(Optional)_ Base branch for docs PRs (default: `main`) |
+| `GH_PAT` | _(Optional)_ GitHub PAT with `repo` scope. Only needed for **separate docs repos** (`docs-repo-url` pointing to a different repo). For same-repo setups, the built-in `GITHUB_TOKEN` works. |
 | `JIRA_URL` | _(Optional, for `[review-feature]`)_ Jira instance URL (e.g., `https://your-company.atlassian.net`) |
 | `JIRA_USERNAME` | _(Optional, for `[review-feature]`)_ Jira username/email |
 | `JIRA_API_TOKEN` | _(Optional, for `[review-feature]`)_ Jira API token ([create here](https://id.atlassian.com/manage-profile/security/api-tokens)) |
 | `GOOGLE_SA_KEY` | _(Optional, for `[review-feature]`)_ Google service account JSON key for fetching Google Docs. Docs must be shared with the service account email. |
-| `MAX_CONTEXT_CHARS` | _(Optional)_ Maximum characters for LLM prompt content (default: `400000`, ~100K tokens). Decrease for models with smaller context windows (e.g., `32000` for an 8K-token model). |
 
 ### 3. Optional Action Inputs
 
 These are set as `with:` parameters in the workflow step (not as secrets):
 
-| Input | Description |
-|-------|-------------|
-| `style-config-path` | _(Optional)_ Path to a Markdown style configuration file (`.md`) containing documentation style guidelines. If not set, auto-detects `.code-to-docs/style.md`. |
+| Input | Default | Description |
+|-------|---------|-------------|
+| `docs-subfolder` | _(empty)_ | Relative path to docs subfolder within the same repo (e.g., `docs`) |
+| `docs-base-branch` | `main` | Base branch for docs repository PRs |
+| `max-context-chars` | `400000` | Maximum characters for LLM prompt content (~100K tokens). Decrease for models with smaller context windows. |
+| `style-config-path` | _(auto-detect)_ | Path to a Markdown style configuration file. If not set, auto-detects `.code-to-docs/style.md`. |
+
+> **Migration note:** `DOCS_SUBFOLDER`, `DOCS_BASE_BRANCH`, and `MAX_CONTEXT_CHARS` were previously documented as repository secrets. They are not secret values and should be set as action inputs instead. GitHub masks secret values in logs, which obstructs debugging when these are misconfigured. The action still reads from environment variables as a fallback.
 
 ### Supported Model Backends
 

src/config.py

@@ -259,6 +259,100 @@ def get_pr_title_prefix():
     return f"{prefix} " if prefix else ""
 
 
+_VALIDATION_DEFAULTS = {
+    "removal-threshold": 0.20,
+    "min-lines": 30,
+    "llm-verification": True,
+}
+
+
+def get_validation_config(repo_config=None):
+    """Extract and validate the validation settings from repo config.
+
+    Returns a dict with keys: removal_threshold, min_lines, llm_verification.
+    """
+    defaults = _VALIDATION_DEFAULTS
+    if repo_config is None:
+        repo_config = load_repo_config()
+
+    v = repo_config.get("validation", {})
+    if not isinstance(v, dict):
+        print("Warning: validation config is not an object, using defaults")
+        v = {}
+
+    threshold = v.get("removal-threshold", defaults["removal-threshold"])
+    if not isinstance(threshold, (int, float)) or not (0.0 <= threshold <= 1.0):
+        print(
+            f"Warning: invalid removal-threshold {threshold!r}, "
+            f"using default {defaults['removal-threshold']}"
+        )
+        threshold = defaults["removal-threshold"]
+
+    min_lines = v.get("min-lines", defaults["min-lines"])
+    if not isinstance(min_lines, int) or min_lines < 0:
+        print(f"Warning: invalid min-lines {min_lines!r}, using default {defaults['min-lines']}")
+        min_lines = defaults["min-lines"]
+
+    llm_verify = v.get("llm-verification", defaults["llm-verification"])
+    if not isinstance(llm_verify, bool):
+        print(
+            f"Warning: invalid llm-verification {llm_verify!r}, "
+            f"using default {defaults['llm-verification']}"
+        )
+        llm_verify = defaults["llm-verification"]
+
+    return {
+        "removal_threshold": float(threshold),
+        "min_lines": int(min_lines),
+        "llm_verification": llm_verify,
+    }
+
+
+# =============================================================================
+# IGNORE LIST
+# =============================================================================
+
+_IGNORE_FILE = ".code-to-docs/ignore"
+
+
+def load_ignore_patterns():
+    """Load gitignore-style exclusion patterns from the base branch.
+
+    Returns a list of pattern strings. Empty list if the file is absent.
+    """
+    base_branch = os.environ.get("DOCS_BASE_BRANCH") or "main"
+    try:
+        result = run_command_safe(
+            ["git", "show", f"origin/{base_branch}:{_IGNORE_FILE}"],
+            check=False,
+        )
+        if result.returncode != 0 or not result.stdout.strip():
+            return []
+        lines = result.stdout.strip().splitlines()
+        patterns = [ln.strip() for ln in lines if ln.strip() and not ln.strip().startswith("#")]
+        if patterns:
+            print(f"Loaded {len(patterns)} ignore pattern(s) from {base_branch}:{_IGNORE_FILE}")
+        return patterns
+    except Exception as e:
+        print(f"Warning: Could not load ignore patterns: {sanitize_output(str(e))}")
+        return []
+
+
+def is_path_ignored(path, patterns):
+    """Check whether a file path matches any gitignore-style pattern."""
+    if not patterns:
+        return False
+    from fnmatch import fnmatch
+
+    path_str = str(path)
+    for pattern in patterns:
+        if fnmatch(path_str, pattern) or fnmatch(path_str, f"**/{pattern}"):
+            return True
+        if "/" in pattern and fnmatch(path_str, pattern):
+            return True
+    return False
+
+
 def check_context_error(e):
     """
     If e is a context-window error, print actionable guidance.

src/discovery.py

@@ -18,6 +18,8 @@
     get_client,
     get_max_context_chars,
     get_model_name,
+    is_path_ignored,
+    load_ignore_patterns,
     truncate_content,
 )
 
@@ -101,6 +103,15 @@ def get_file_content_or_summaries(line_threshold=300):
     # Filter out internal index files (.doc-index/) - these are for internal use only
     doc_files = [f for f in doc_files if ".doc-index" not in str(f)]
 
+    # Filter out files matching .code-to-docs/ignore patterns
+    ignore_patterns = load_ignore_patterns()
+    if ignore_patterns:
+        before = len(doc_files)
+        doc_files = [f for f in doc_files if not is_path_ignored(f, ignore_patterns)]
+        ignored = before - len(doc_files)
+        if ignored:
+            print(f"Excluded {ignored} file(s) via .code-to-docs/ignore")
+
     # Deduplicate file paths BEFORE processing to avoid duplicate work
     seen_paths = set()
     unique_doc_files = []
@@ -366,4 +377,12 @@ def find_relevant_files_optimized(diff):
         print("Falling back to full scan...")
         return None
 
+    ignore_patterns = load_ignore_patterns()
+    if ignore_patterns and relevant_files:
+        before = len(relevant_files)
+        relevant_files = [f for f in relevant_files if not is_path_ignored(f, ignore_patterns)]
+        ignored = before - len(relevant_files)
+        if ignored:
+            print(f"Excluded {ignored} file(s) via .code-to-docs/ignore")
+
     return relevant_files

src/generation.py

@@ -157,6 +157,7 @@ def generate_updates_parallel(
     file_instructions=None,
     style_guidelines="",
     pr_description="",
+    validation_config=None,
 ):
     """
     Generate documentation updates in parallel.
@@ -238,6 +239,7 @@ def ask_ai_for_updated_content(
     current_content,
     user_instructions="",
     file_instructions=None,
+    validation_config=None,
     style_guidelines="",
     pr_description="",
 ):

src/suggest_docs.py

@@ -33,6 +33,7 @@
     get_max_context_chars,
     get_model_name,
     get_pr_title_prefix,
+    get_validation_config,
     load_style_config_from_branch,
 )
 from discovery import (
@@ -248,6 +249,8 @@ def main():
     # uses the repo's current style config, even if the PR branch predates it.
     style_guidelines = load_style_config_from_branch()
 
+    validation_config = get_validation_config()
+
     # Handle --build-index mode
     if args.build_index:
         print("Building documentation indexes...")
@@ -542,6 +545,7 @@ def main():
             file_instructions=file_instructions,
             style_guidelines=style_guidelines,
             pr_description=pr_description,
+            validation_config=validation_config,
         )
 
         for file_path, _current, updated in files_with_content:
@@ -566,6 +570,7 @@ def main():
                 file_instructions=file_instructions,
                 style_guidelines=style_guidelines,
                 pr_description=pr_description,
+                validation_config=validation_config,
             )
 
             if updated.strip() == "NO_UPDATE_NEEDED":

tests/test_ignore.py

@@ -0,0 +1,55 @@
+"""Tests for .code-to-docs/ignore exclusion list."""
+
+from unittest.mock import MagicMock, patch
+
+from config import is_path_ignored, load_ignore_patterns
+
+
+class TestIsPathIgnored:
+    def test_no_patterns_returns_false(self):
+        assert is_path_ignored("docs/guide.md", []) is False
+
+    def test_exact_match(self):
+        assert is_path_ignored("README.md", ["README.md"]) is True
+
+    def test_glob_match(self):
+        assert is_path_ignored("docs/api-ref.md", ["docs/api-*.md"]) is True
+
+    def test_no_match(self):
+        assert is_path_ignored("docs/guide.md", ["docs/api-*.md"]) is False
+
+    def test_directory_glob(self):
+        assert is_path_ignored("generated/openapi/ref.md", ["generated/*"]) is True
+
+    def test_bare_filename_matches_anywhere(self):
+        assert is_path_ignored("deep/nested/CHANGELOG.md", ["CHANGELOG.md"]) is True
+
+    def test_multiple_patterns(self):
+        patterns = ["CHANGELOG.md", "generated/*", "*.bak"]
+        assert is_path_ignored("docs/old.bak", patterns) is True
+        assert is_path_ignored("docs/guide.md", patterns) is False
+
+
+class TestLoadIgnorePatterns:
+    def test_loads_patterns_from_branch(self):
+        result = MagicMock(returncode=0, stdout="# comment\ngenerated/*\nREADME.md\n\n")
+        with patch("config.run_command_safe", return_value=result):
+            patterns = load_ignore_patterns()
+        assert patterns == ["generated/*", "README.md"]
+
+    def test_returns_empty_when_file_missing(self):
+        result = MagicMock(returncode=1, stdout="")
+        with patch("config.run_command_safe", return_value=result):
+            patterns = load_ignore_patterns()
+        assert patterns == []
+
+    def test_returns_empty_on_error(self):
+        with patch("config.run_command_safe", side_effect=RuntimeError("git failed")):
+            patterns = load_ignore_patterns()
+        assert patterns == []
+
+    def test_skips_comments_and_blanks(self):
+        result = MagicMock(returncode=0, stdout="# skip this\n\n  \nkeep-this.md\n")
+        with patch("config.run_command_safe", return_value=result):
+            patterns = load_ignore_patterns()
+        assert patterns == ["keep-this.md"]

tests/test_repo_config.py

@@ -96,3 +96,35 @@ def test_non_string_prefix_returns_empty(self):
     def test_list_prefix_returns_empty(self):
         config._repo_config_cache = {"pr-title-prefix": [":book:"]}
         assert config.get_pr_title_prefix() == ""
+
+
+class TestGetValidationConfig:
+    def test_defaults_when_no_config(self):
+        vc = config.get_validation_config({})
+        assert vc["removal_threshold"] == 0.20
+        assert vc["min_lines"] == 30
+        assert vc["llm_verification"] is True
+
+    def test_overrides_threshold(self):
+        vc = config.get_validation_config({"validation": {"removal-threshold": 0.50}})
+        assert vc["removal_threshold"] == 0.50
+
+    def test_overrides_min_lines(self):
+        vc = config.get_validation_config({"validation": {"min-lines": 10}})
+        assert vc["min_lines"] == 10
+
+    def test_disables_llm_verification(self):
+        vc = config.get_validation_config({"validation": {"llm-verification": False}})
+        assert vc["llm_verification"] is False
+
+    def test_invalid_threshold_falls_back(self):
+        vc = config.get_validation_config({"validation": {"removal-threshold": "bad"}})
+        assert vc["removal_threshold"] == 0.20
+
+    def test_threshold_out_of_range_falls_back(self):
+        vc = config.get_validation_config({"validation": {"removal-threshold": 1.5}})
+        assert vc["removal_threshold"] == 0.20
+
+    def test_non_dict_validation_falls_back(self):
+        vc = config.get_validation_config({"validation": "not a dict"})
+        assert vc["llm_verification"] is True