Conversation
Changes required to successfully index large Go source trees that mix
in-tree dependencies (vendored), build-constrained packages, and
generated files outside the module root. The motivating case is
github.com/golang/go itself, but every change is generic.
- Bump golang.org/x/tools v0.43.0 -> v0.44.0. v0.43.0 calls log.Fatalf
inside packages.Load when the loader is asked to import a vendored
stdlib package that has no type information, which made indexing of
GOROOT/src impossible. v0.44.0 contains the upstream fix that turns
this into a recoverable error.
- internal/index/scip.go: skip packages with no syntax files ("unsafe"
pseudo-package, build-constrained packages such as
internal/runtime/wasitest with zero matching .go files for the host
platform). Also tolerate packages whose syntax files were all skipped
(see visitors change below) via a new firstSyntaxWithDocument helper.
- internal/loader/loader.go: promote in-tree dependencies to project
packages. `go list ./...` deliberately excludes vendor/ directories,
so vendored packages would otherwise be loaded only as dependencies
and never indexed even though their source lives in the tree. Detect
in-tree-ness via Syntax (with CompiledGoFiles / GoFiles fallbacks)
because some loaders, including the stdlib's vendored packages
loaded via export data, leave GoFiles empty.
- internal/visitors/visitors.go: drop syntax files whose path is
outside the module root. The Go test driver writes _testmain.go
shims into $GOCACHE for every <pkg>.test package; those paths
resolve to ../../../Library/Caches/... and shouldn't be in the
index.
- internal/visitors/visitor_file.go: when resolving an ImportSpec to
its loaded package, try the source-level import path first and only
fall back to pkgName.Imported().Path(). pkg.Imports is keyed by the
source-level string; a vendor-rewritten resolved path (e.g.
vendor/golang.org/x/crypto/chacha20) misses every entry and produced
"Could not find node" warnings that dropped all import references
to vendored packages.
Adds the Go source tree as a matrix target. Two pieces of plumbing allow it to share the existing job: - A new optional matrix.module_root field threads through setup-go's cache-dependency-path, scip-go's --module-root, and scip stats's --project-root, defaulting to '.' when unset so every other entry behaves exactly as before. - A 3-line GOROOT-export step gated on matrix.name == 'golang'. Without it, scip-go silently indexes the runner's system stdlib while reporting paths under target/src/, producing an index that is missing target's vendor/golang.org/x/... files and includes spurious files from the system Go.
- isInTree: check the first parsed syntax file only; all files in a Go package share a directory, so a single check is sufficient. - visitor_file: look up imports directly by the source-level import path (pkg.Imports is keyed that way for vendored imports).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.