Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions apps/penpal/ERD.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ see-also:
- <a id="E-PENPAL-DISCOVERY"></a>**E-PENPAL-DISCOVERY**: `DiscoverWorkspace()` reads all non-hidden immediate subdirectories as projects, calls `DetectSources()` on each, then `DiscoverWorktrees()`. Deduplicates projects that are git worktrees of each other.
← [P-PENPAL-WORKSPACE](PRODUCT.md#P-PENPAL-WORKSPACE), [P-PENPAL-DEDUP](PRODUCT.md#P-PENPAL-DEDUP), [P-PENPAL-WS-ROOT](PRODUCT.md#P-PENPAL-WS-ROOT)

- <a id="E-PENPAL-SOURCE-REGISTRY"></a>**E-PENPAL-SOURCE-REGISTRY**: Source types are registered in a pluggable `SourceType` registry via `init()`. Each `SourceType` entry defines: `AutoDetectDir` or `AutoDetectFile` (trigger), `DetectAtWSRoot` (workspace-level detection), `ScanMode` ("tree" or "files"), `SkipDirs`, `ClassifyFile()` (returns type string or `""` to hide), and optional `GroupFiles()` (returns named sections). `DetectSources()` iterates all registered types and checks for triggers at the project root.
- <a id="E-PENPAL-SOURCE-REGISTRY"></a>**E-PENPAL-SOURCE-REGISTRY**: Source types are registered in a pluggable `SourceType` registry via `init()`. Each `SourceType` entry defines: `AutoDetectDir` or `AutoDetectFile` (trigger), `DetectAtWSRoot` (workspace-level detection), `ScanMode` ("tree" or "files"), `SkipDirs`, `RequireSibling` (only include files from directories containing this file), `ClassifyFile()` (returns type string or `""` to hide), and optional `GroupFiles()` (returns named sections). `DetectSources()` iterates all registered types and checks for triggers at the project root.
← [P-PENPAL-SRC-DETECT](PRODUCT.md#P-PENPAL-SRC-DETECT), [P-PENPAL-SRC-CLASSIFY](PRODUCT.md#P-PENPAL-SRC-CLASSIFY), [P-PENPAL-SRC-GROUP](PRODUCT.md#P-PENPAL-SRC-GROUP), [P-PENPAL-SRC-SKIP](PRODUCT.md#P-PENPAL-SRC-SKIP), [P-PENPAL-AUTO-DETECT](PRODUCT.md#P-PENPAL-AUTO-DETECT), [P-PENPAL-SRC-BADGE](PRODUCT.md#P-PENPAL-SRC-BADGE)

- <a id="E-PENPAL-SRC-THOUGHTS"></a>**E-PENPAL-SRC-THOUGHTS**: The `thoughts` source type auto-detects `thoughts/` directory. `DetectAtWSRoot: true` enables workspace-root detection. Classifies files as `research`, `plan`, or `other`. No custom `GroupFiles` — files appear in a single flat group named after the source. No `SkipDirs`.
Expand All @@ -76,7 +76,7 @@ see-also:
- <a id="E-PENPAL-SRC-RP1"></a>**E-PENPAL-SRC-RP1**: The `rp1` source type auto-detects `.rp1/` directory. `ClassifyFile()` maps path prefixes to types (context/ → knowledge, work/prds/ → prd, work/features/{id}/ → requirement/design/task/etc., work/issues/{id}/ → investigation/analysis/etc.). Returns `""` for `work/archives/`, `work/worktrees/`, `work/notes/` to hide them. `GroupFiles()` organizes files into fixed-order sections (Blueprint, Quick Builds, Research, Reviews, Content, Other) with dynamic Feature: and Issue: groups sorted alphabetically.
← [P-PENPAL-SRC-RP1](PRODUCT.md#P-PENPAL-SRC-RP1), [P-PENPAL-SRC-RP1-CLASSIFY](PRODUCT.md#P-PENPAL-SRC-RP1-CLASSIFY), [P-PENPAL-SRC-RP1-GROUP](PRODUCT.md#P-PENPAL-SRC-RP1-GROUP)

- <a id="E-PENPAL-SRC-ANCHORS"></a>**E-PENPAL-SRC-ANCHORS**: The `anchors` source type auto-detects `ANCHORS.md` file at project root. `ScanMode: "tree"`. `SkipDirs`: `.git`, `node_modules`, `.hg`, `.svn`. `ClassifyFile()` returns a type only for the five recognized filenames (`ANCHORS.md` → anchors, `PRODUCT.md` → product, `ERD.md` → engineering, `TESTING.md` → testing, `DEPENDENCIES.md` → dependencies); all others return `""`. `GroupFiles()` groups by module directory (any directory containing `ANCHORS.md`), root module shown as "(root)", modules sorted alphabetically, files within each module in canonical order (ANCHORS → PRODUCT → ERD → TESTING → DEPENDENCIES). Stray files without a sibling `ANCHORS.md` are dropped.
- <a id="E-PENPAL-SRC-ANCHORS"></a>**E-PENPAL-SRC-ANCHORS**: The `anchors` source type auto-detects `ANCHORS.md` file at project root. `ScanMode: "tree"`. `SkipDirs`: `.git`, `node_modules`, `.hg`, `.svn`. `RequireSibling: "ANCHORS.md"` — the scanner only includes files from directories containing this sibling, which excludes stray files and `ANCHORS.md` itself (since `ClassifyFile()` returns `""` for it). `ClassifyFile()` returns a content type for four document filenames (`PRODUCT.md` → product, `ERD.md` → engineering, `TESTING.md` → testing, `DEPENDENCIES.md` → dependencies); all others (including `ANCHORS.md`) return `""`. `GroupFiles()` groups by directory, root module shown as "(root)", modules sorted alphabetically, files within each module in canonical order (PRODUCT → ERD → TESTING → DEPENDENCIES).
← [P-PENPAL-SRC-ANCHORS](PRODUCT.md#P-PENPAL-SRC-ANCHORS), [P-PENPAL-SRC-ANCHORS-GROUP](PRODUCT.md#P-PENPAL-SRC-ANCHORS-GROUP), [P-PENPAL-SRC-ANCHORS-NESTED](PRODUCT.md#P-PENPAL-SRC-ANCHORS-NESTED)

- <a id="E-PENPAL-SRC-CLAUDE-PLANS"></a>**E-PENPAL-SRC-CLAUDE-PLANS**: The `claude-plans` source type classifies all files as `plan`. No custom grouping. Injected via `DiscoverClaudePlans()` which creates a synthetic standalone project or injects a tree source into an existing manually-added project.
Expand Down
6 changes: 3 additions & 3 deletions apps/penpal/PRODUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,11 +86,11 @@ Source types are the pluggable system that determines how projects discover, cla

### anchors Source Type

- <a id="P-PENPAL-SRC-ANCHORS"></a>**P-PENPAL-SRC-ANCHORS**: Auto-detected by an `ANCHORS.md` file at the project root. Shows a teal "ANCHORS" badge. Scans the full project tree but only shows the five recognized ANCHORS filenames: `ANCHORS.md`, `PRODUCT.md`, `ERD.md`, `TESTING.md`, `DEPENDENCIES.md`. All other files are hidden.
- <a id="P-PENPAL-SRC-ANCHORS"></a>**P-PENPAL-SRC-ANCHORS**: Auto-detected by an `ANCHORS.md` file at the project root. Shows a teal "ANCHORS" badge. Scans the full project tree but only includes files from directories that contain an `ANCHORS.md` sibling. Only four content filenames are shown: `PRODUCT.md`, `ERD.md`, `TESTING.md`, `DEPENDENCIES.md`. `ANCHORS.md` itself and all other files are hidden. A directory with only `ANCHORS.md` and no content files produces no visible output.

- <a id="P-PENPAL-SRC-ANCHORS-GROUP"></a>**P-PENPAL-SRC-ANCHORS-GROUP**: Files are grouped by module directory — a subdirectory that contains its own `ANCHORS.md`. The root module is shown as "(root)". Modules are sorted alphabetically. Within each module, files are sorted in canonical order: ANCHORS → PRODUCT → ERD → TESTING → DEPENDENCIES.
- <a id="P-PENPAL-SRC-ANCHORS-GROUP"></a>**P-PENPAL-SRC-ANCHORS-GROUP**: Files are grouped by module directory. The root module is shown as "(root)". Modules are sorted alphabetically. Within each module, files are sorted in canonical order: PRODUCT → ERD → TESTING → DEPENDENCIES.

- <a id="P-PENPAL-SRC-ANCHORS-NESTED"></a>**P-PENPAL-SRC-ANCHORS-NESTED**: Supports nested modules in monorepos. Stray ANCHORS document files (e.g., a `PRODUCT.md` in a directory without a sibling `ANCHORS.md`) are excluded from the file list.
- <a id="P-PENPAL-SRC-ANCHORS-NESTED"></a>**P-PENPAL-SRC-ANCHORS-NESTED**: Supports nested modules in monorepos. Stray ANCHORS document files (e.g., a `PRODUCT.md` in a directory without a sibling `ANCHORS.md`) are excluded by the sibling requirement at scan time.

### claude-plans Source Type

Expand Down
2 changes: 1 addition & 1 deletion apps/penpal/TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ see-also:
| Source Types — General (P-PENPAL-SRC-DETECT, SRC-CLASSIFY, SRC-GROUP) | discovery_test.go | — | integration_test.go (buildFileGroups) | — |
| Source Types — thoughts (P-PENPAL-SRC-THOUGHTS) | discovery_test.go (classify) | — | grouping_test.go (TestBuildFileGroups_ThoughtsFlat) | — |
| Source Types — rp1 (P-PENPAL-SRC-RP1, SRC-RP1-CLASSIFY, SRC-RP1-GROUP) | discovery_test.go (TestClassifyRP1File, TestGroupRP1Paths) | — | grouping_test.go (TestBuildFileGroups_RP1Grouped) | — |
| Source Types — anchors (P-PENPAL-SRC-ANCHORS, SRC-ANCHORS-GROUP) | discovery_test.go (TestClassifyAnchorsFile, TestGroupAnchorsPaths) | — | — | — |
| Source Types — anchors (P-PENPAL-SRC-ANCHORS, SRC-ANCHORS-GROUP, SRC-ANCHORS-NESTED) | discovery_test.go (TestClassifyAnchorsFile, TestGroupAnchorsPaths, TestGroupAnchorsPaths_MarkerOnlyModule, TestAnchorsFileOrder, TestAnchorsRequireSibling) | — | — | — |
| Source Types — claude-plans (P-PENPAL-SRC-CLAUDE-PLANS) | — | — | — | — |
| Source Types — manual (P-PENPAL-SRC-MANUAL) | — | — | grouping_test.go (TestBuildFileGroups_ManualSourceDirHeadings) | — |
| Cache & File Scanning (E-PENPAL-CACHE, SCAN) | cache_test.go | — | — | — |
Expand Down
7 changes: 7 additions & 0 deletions apps/penpal/internal/cache/cache.go
Original file line number Diff line number Diff line change
Expand Up @@ -575,6 +575,13 @@ func scanProjectSources(project *discovery.Project) []FileInfo {
if !strings.HasSuffix(path, ".md") {
return nil
}
// E-PENPAL-SOURCE-REGISTRY: RequireSibling pre-filter.
if st != nil && st.RequireSibling != "" {
siblingPath := filepath.Join(filepath.Dir(path), st.RequireSibling)
if _, err := os.Stat(siblingPath); err != nil {
return nil
}
}
relToSource, _ := filepath.Rel(rootPath, path)
relToProject, _ := filepath.Rel(project.Path, path)

Expand Down
40 changes: 13 additions & 27 deletions apps/penpal/internal/discovery/discovery.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ type SourceType struct {
GroupFiles func(paths []string) []FileGroup // optional: groups files for display; if nil, single flat list
ShowDirHeadings bool // show directory headings above files in each subdirectory
SkipDirs map[string]bool // directory names to skip during tree scan (e.g., node_modules)
RequireSibling string // if set, only include files from directories containing this file
}

// FileGroup represents a named group of files within a source for display.
Expand Down Expand Up @@ -142,6 +143,7 @@ func init() {
BadgeActiveColor: "#0f766e",
AutoDetectFile: "ANCHORS.md",
ScanMode: "tree",
RequireSibling: "ANCHORS.md",
ClassifyFile: classifyAnchorsFile,
GroupFiles: groupAnchorsPaths,
SkipDirs: map[string]bool{
Expand Down Expand Up @@ -223,13 +225,12 @@ func isRP1TopLevelReport(path string) bool {
}

// classifyAnchorsFile classifies a file by its name within an ANCHORS module.
// Only known ANCHORS document filenames are kept; everything else is skipped.
// E-PENPAL-SRC-ANCHORS: recognizes the five ANCHORS filenames, skips all others.
// Only the four content filenames are kept; everything else (including ANCHORS.md
// itself) is skipped. Module membership is enforced by RequireSibling at scan time.
// E-PENPAL-SRC-ANCHORS: recognizes four content filenames, skips all others.
func classifyAnchorsFile(path string) string {
base := filepath.Base(path)
switch base {
case "ANCHORS.md":
return "anchors"
case "PRODUCT.md":
return "product"
case "ERD.md":
Expand All @@ -239,45 +240,30 @@ func classifyAnchorsFile(path string) string {
case "DEPENDENCIES.md":
return "dependencies"
default:
return "" // skip non-ANCHORS files
return "" // skip ANCHORS.md and non-ANCHORS files

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Prevent manual sources from resurfacing ANCHORS.md

Returning "" for ANCHORS.md means the anchors source no longer claims that path, so scanProjectSources never marks it as seen and a later overlapping manual tree source (for example, path: "." in project sources) will include ANCHORS.md again. In that configuration, the marker file reappears in the sidebar/search despite this change’s goal of hiding it globally, because deduplication in scanProjectSources is first-claim-wins and this source now opts out of claiming the file.

Useful? React with 👍 / 👎.

}
}

// anchorsFileOrder defines the display order for files within a module.
var anchorsFileOrder = map[string]int{
"ANCHORS.md": 0,
"PRODUCT.md": 1,
"ERD.md": 2,
"TESTING.md": 3,
"DEPENDENCIES.md": 4,
"PRODUCT.md": 0,
"ERD.md": 1,
"TESTING.md": 2,
"DEPENDENCIES.md": 3,
}

// groupAnchorsPaths groups ANCHORS source-relative paths by module directory.
// Only files in directories that contain an ANCHORS.md marker are included.
// All incoming files have already passed RequireSibling and ClassifyFile filtering,
// so every file belongs to a valid module directory.
// E-PENPAL-SRC-ANCHORS: groups by module directory with canonical file ordering.
func groupAnchorsPaths(paths []string) []FileGroup {
// First pass: find directories that contain an ANCHORS.md marker
modules := make(map[string]bool)
for _, p := range paths {
if filepath.Base(p) == "ANCHORS.md" {
dir := filepath.Dir(p)
if dir == "." {
dir = ""
}
modules[dir] = true
}
}

// Second pass: group files by module directory
groups := make(map[string][]string)
for _, p := range paths {
dir := filepath.Dir(p)
if dir == "." {
dir = ""
}
if modules[dir] {
groups[dir] = append(groups[dir], p)
}
groups[dir] = append(groups[dir], p)
}

// Sort module names; root ("") sorts first naturally
Expand Down
99 changes: 71 additions & 28 deletions apps/penpal/internal/discovery/discovery_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ func TestDeduplicateWorktreeProjects(t *testing.T) {
}
}

// E-PENPAL-SRC-ANCHORS: verifies ClassifyFile recognizes the five ANCHORS filenames.
// E-PENPAL-SRC-ANCHORS: verifies ClassifyFile recognizes four content filenames and skips all others.
func TestClassifyAnchorsFile(t *testing.T) {
st := GetSourceType("anchors")
if st == nil || st.ClassifyFile == nil {
Expand All @@ -326,13 +326,15 @@ func TestClassifyAnchorsFile(t *testing.T) {
path string
expected string
}{
{"ANCHORS.md", "anchors"},
// ANCHORS.md is skipped (module membership enforced by RequireSibling)
{"ANCHORS.md", ""},
{"auth/ANCHORS.md", ""},
// content files
{"PRODUCT.md", "product"},
{"ERD.md", "engineering"},
{"TESTING.md", "testing"},
{"DEPENDENCIES.md", "dependencies"},
// nested module
{"auth/ANCHORS.md", "anchors"},
// nested module content files
{"auth/PRODUCT.md", "product"},
{"auth/ERD.md", "engineering"},
{"services/payments/TESTING.md", "testing"},
Expand All @@ -354,6 +356,9 @@ func TestClassifyAnchorsFile(t *testing.T) {
}

// E-PENPAL-SRC-ANCHORS: verifies GroupFiles groups by module directory with canonical ordering.
// Note: ANCHORS.md never reaches GroupFiles — it's filtered by ClassifyFile (returns "")
// at scan time. Module membership is enforced by RequireSibling at scan time, so
// GroupFiles receives only pre-validated content files.
func TestGroupAnchorsPaths(t *testing.T) {
st := GetSourceType("anchors")
if st == nil || st.GroupFiles == nil {
Expand All @@ -366,72 +371,55 @@ func TestGroupAnchorsPaths(t *testing.T) {
expected []FileGroup
}{
{
name: "empty input",
name: "empty input produces no groups",
paths: nil,
expected: nil,
},
{
name: "single root module",
paths: []string{
"ANCHORS.md",
"PRODUCT.md",
"ERD.md",
},
expected: []FileGroup{
{Name: "(root)", Paths: []string{"ANCHORS.md", "PRODUCT.md", "ERD.md"}},
{Name: "(root)", Paths: []string{"PRODUCT.md", "ERD.md"}},
},
},
{
name: "root module file ordering",
paths: []string{
"DEPENDENCIES.md",
"PRODUCT.md",
"ANCHORS.md",
"TESTING.md",
"ERD.md",
},
expected: []FileGroup{
{Name: "(root)", Paths: []string{
"ANCHORS.md", "PRODUCT.md", "ERD.md", "TESTING.md", "DEPENDENCIES.md",
"PRODUCT.md", "ERD.md", "TESTING.md", "DEPENDENCIES.md",
}},
},
},
{
name: "nested modules sorted alphabetically",
paths: []string{
"payments/ANCHORS.md",
"payments/PRODUCT.md",
"auth/ANCHORS.md",
"auth/PRODUCT.md",
"auth/ERD.md",
},
expected: []FileGroup{
{Name: "auth", Paths: []string{"auth/ANCHORS.md", "auth/PRODUCT.md", "auth/ERD.md"}},
{Name: "payments", Paths: []string{"payments/ANCHORS.md", "payments/PRODUCT.md"}},
{Name: "auth", Paths: []string{"auth/PRODUCT.md", "auth/ERD.md"}},
{Name: "payments", Paths: []string{"payments/PRODUCT.md"}},
},
},
{
name: "root and nested modules together",
paths: []string{
"ANCHORS.md",
"PRODUCT.md",
"services/auth/ANCHORS.md",
"services/auth/ERD.md",
},
expected: []FileGroup{
{Name: "(root)", Paths: []string{"ANCHORS.md", "PRODUCT.md"}},
{Name: "services/auth", Paths: []string{"services/auth/ANCHORS.md", "services/auth/ERD.md"}},
},
},
{
name: "files without sibling ANCHORS.md are dropped",
paths: []string{
"ANCHORS.md",
"PRODUCT.md",
"stray/PRODUCT.md",
},
expected: []FileGroup{
{Name: "(root)", Paths: []string{"ANCHORS.md", "PRODUCT.md"}},
{Name: "(root)", Paths: []string{"PRODUCT.md"}},
{Name: "services/auth", Paths: []string{"services/auth/ERD.md"}},
},
},
}
Expand Down Expand Up @@ -462,6 +450,61 @@ func TestGroupAnchorsPaths(t *testing.T) {
}
}

// E-PENPAL-SRC-ANCHORS: verifies that a marker-only module (directory with ANCHORS.md
// but no content files) produces no groups. This documents the intended behavior:
// RequireSibling admits the directory, ClassifyFile skips ANCHORS.md, and
// GroupFiles receives an empty slice.
func TestGroupAnchorsPaths_MarkerOnlyModule(t *testing.T) {
st := GetSourceType("anchors")
if st == nil || st.GroupFiles == nil {
t.Fatal("anchors source type not registered or has no GroupFiles")
}

// Simulate what the scanner produces for a directory containing only ANCHORS.md:
// ClassifyFile returns "" for ANCHORS.md, so it never reaches GroupFiles.
got := st.GroupFiles(nil)
if len(got) != 0 {
t.Errorf("expected 0 groups for marker-only module, got %d: %+v", len(got), got)
}
}

// E-PENPAL-SRC-ANCHORS: verifies canonical file ordering is stable for all recognized
// content filenames. An unrecognized filename would get Go's zero-value (0) from the
// map lookup, colliding with PRODUCT.md's position — RequireSibling and ClassifyFile
// prevent this, but this test documents the sort contract.
func TestAnchorsFileOrder(t *testing.T) {
// Every content filename must have a unique position in the order map.
contentFiles := []string{"PRODUCT.md", "ERD.md", "TESTING.md", "DEPENDENCIES.md"}
seen := map[int]string{}
for _, f := range contentFiles {
pos, ok := anchorsFileOrder[f]
if !ok {
t.Errorf("anchorsFileOrder missing entry for %q", f)
continue
}
if prev, dup := seen[pos]; dup {
t.Errorf("anchorsFileOrder position %d shared by %q and %q", pos, prev, f)
}
seen[pos] = f
}

// ANCHORS.md must NOT be in the order map (it never reaches GroupFiles).
if _, ok := anchorsFileOrder["ANCHORS.md"]; ok {
t.Error("anchorsFileOrder should not contain ANCHORS.md")
}
}

// E-PENPAL-SRC-ANCHORS: verifies RequireSibling is set on the anchors source type.
func TestAnchorsRequireSibling(t *testing.T) {
st := GetSourceType("anchors")
if st == nil {
t.Fatal("anchors source type not registered")
}
if st.RequireSibling != "ANCHORS.md" {
t.Errorf("RequireSibling = %q, want %q", st.RequireSibling, "ANCHORS.md")
}
}

// E-PENPAL-SRC-CLAUDE-PLANS: verifies the claude-plans source type is registered with correct properties.
func TestClaudePlansSourceType(t *testing.T) {
st := GetSourceType("claude-plans")
Expand Down
7 changes: 7 additions & 0 deletions apps/penpal/internal/server/search.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,13 @@ func (s *Server) handleAPISearch(w http.ResponseWriter, r *http.Request) {
if !strings.HasSuffix(path, ".md") {
return nil
}
// E-PENPAL-SOURCE-REGISTRY: RequireSibling pre-filter.
if st != nil && st.RequireSibling != "" {
siblingPath := filepath.Join(filepath.Dir(path), st.RequireSibling)
if _, err := os.Stat(siblingPath); err != nil {
return nil
}
}

relToProject, _ := filepath.Rel(project.Path, path)
relToSource, _ := filepath.Rel(source.RootPath, path)
Expand Down