Skip to content
This repository has been archived by the owner on Sep 30, 2024. It is now read-only.

Sketch of SCIP-based GraphQL API #61217

Closed
wants to merge 20 commits into from
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
e50bb3b
WIP: New SCIP-based GraphQL API draft
varungandhi-src Mar 18, 2024
e5b9e38
Make repository and path filters objects instead of primitives
varungandhi-src Apr 22, 2024
df4fb88
Make symbol name and provenance comparison objects instead of primitives
varungandhi-src Apr 22, 2024
203e505
Move usages API to top-level
varungandhi-src Apr 22, 2024
1949594
Address feedback abotu optionality, add comments, introduce doc API, …
varungandhi-src Apr 23, 2024
eebbd29
Add more comments, consistent naming, add spaces
varungandhi-src Apr 25, 2024
072bbc0
Add filter for provenance
varungandhi-src Apr 25, 2024
8e947b7
Rename symbolName -> symbol in Occurrence
varungandhi-src Apr 25, 2024
4032028
Replace DEPRECATED: comments with @deprecated
varungandhi-src Apr 25, 2024
2146fed
Tweak comments/replace symbolName->symbol
varungandhi-src Apr 25, 2024
570798a
Return multiple CodeGraphData due to multiple tool possibility
varungandhi-src May 2, 2024
9269229
Mention toolInfo and build identifiers.
varungandhi-src May 2, 2024
7c42739
Mention file navigation and use common-ish syntax for extension ideas
varungandhi-src May 2, 2024
e90ab20
Fix naming convention + use pagination for symbols & occurrences
varungandhi-src May 2, 2024
5f0c62a
Use Connection for SCIP Document data
varungandhi-src May 2, 2024
dbd4068
Rename Provenance & remove WithProvenance interface
varungandhi-src May 2, 2024
e0af435
Add comment for symbols vs scipDocument.data.symbols
varungandhi-src May 2, 2024
653a197
Add filter equivalence and design comment
varungandhi-src May 2, 2024
e20c0a5
Rename field & make blob: GitBlob optional
varungandhi-src May 2, 2024
38fc951
Move comment to different field
varungandhi-src May 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
319 changes: 317 additions & 2 deletions cmd/frontend/graphqlbackend/codeintel.codenav.graphql
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,103 @@ extend type GitBlob {
Experimental: This API is likely to change in the future.
"""
symbolInfo(line: Int!, character: Int!): SymbolInfo

"""
If filter.provenance? is not provided, then the API will go through
each provenance one by one in the order Precise -> Syntactic -> SearchBased
and stop when some data is available.
varungandhi-src marked this conversation as resolved.
Show resolved Hide resolved
"""
codeGraphData(filter: CodeGraphDataFilter): CodeGraphData
}

input CodeGraphDataFilter {
provenance: ProvenanceComparator
}

"""
Invariants:
1. forall sym in symbols.
(sym.provenance, sym.dataSource) == (scipDocument.provenance, scipDocument.dataSource)
"""
# Implementation notes:
varungandhi-src marked this conversation as resolved.
Show resolved Hide resolved
# 1. For precise code nav, we already have a code for inferring
# 'visible uploads' using the commit graph machinery -> infer best upload
# -> get document for that using the given path.
# 2. Syntactic will be similar to precise
# 3. Search-based -> return empty document/lists.
#
# Ref panel logic:
# 1. Request code graph data, specifying 'filter' if needed.
# Either the scipDocument or occurrences is fine.
# 2. When a hover is triggered, attempt to look up occurrence in document or list
# or some other lookup structure constructed from those.
# 3a. If one matching occurrence is found, use symbolName + range to call
# usagesForSymbol
# 3b. If zero matching occurrences are found, use only range to call usagesForSymbol.
# 3c. If more than one matching occurrence is found, use some more complex logic
# to offer a symbol picker.
# - Problem: For a symbol picker, ideally you'd use SymbolInformation.display_name,
# which is stored on SymbolInformation. However, display_name is not supported
# today by most SCIP indexers. And we don't have any API for getting the detailed
# SCIP SymbolInformation for a bunch of occurrences.
type CodeGraphData {
scipDocument: AssociatedSCIPDocument

"""
List of symbols with definitions in this document.

Invariant:
forall sym in symbols.
exists occ in occurrences such that
sym.name == occ.symbolName && occ.roles.contains(Definition)
"""
# TODO: Should this be paginated to gracefully handle large generated files?
varungandhi-src marked this conversation as resolved.
Show resolved Hide resolved
symbols: [SymbolInformation!]
varungandhi-src marked this conversation as resolved.
Show resolved Hide resolved

"""
Occurrences are guaranteed to be sorted by range.
"""
# TODO: Should this be paginated to gracefully handle large generated files?
occurrences(filter: OccurrencesFilter): [Occurrence!]
}

type Occurrence {
varungandhi-src marked this conversation as resolved.
Show resolved Hide resolved
symbolName: String
varungandhi-src marked this conversation as resolved.
Show resolved Hide resolved
range: Range!
roles: [SymbolRole!]
# We can add diagnostics etc. here in the future if needed.
}

input OccurrencesFilter {
rangeFilter: RangeFilter
rolesFilter: RolesFilter
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Add documentation here describing how this can be used for the symbols side bar so that the symbols sidebar is consistent with the navigation in the file. E.g. you can have a precise symbols sidebar if you have precise code intel instead of having to use type:symbol search in the file.

# We can add a DiagnosticsFilter etc. here in the future if needed.
}

input RangeFilter {
"""
Zero-based line number, inclusive.
"""
startLine: Int
"""
Zero-based line number, inclusive.
"""
endLine: Int
}

input RolesFilter {
or: [RolesFilter!]
equals: SymbolRole
}

enum SymbolRole {
Definition,
Reference,
"""
Applicable for forward declarations in languages with header files (C, C++ etc.)
as well as standalone signatures in languages with separate interface files (OCaml etc.).
"""
ForwardDefinition,
}

"""
Expand Down Expand Up @@ -93,6 +190,8 @@ type GitBlobLSIFData implements TreeEntryLSIFData {

"""
A list of definitions of the symbol under the given document position.

DEPRECATED: Use usagesForSymbol with a Definition filter instead.
"""
definitions(
"""
Expand All @@ -113,17 +212,19 @@ type GitBlobLSIFData implements TreeEntryLSIFData {

"""
A list of references of the symbol under the given document position.

DEPRECATED: Use usagesForSymbol with a Reference filter instead.
"""
references(
"""
The line on which the symbol occurs (zero-based, inclusive).
"""
line: Int!
line: Int

"""
The character (not byte) of the start line on which the symbol occurs (zero-based, inclusive).
"""
character: Int!
character: Int
Comment on lines -121 to +281
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Revert these changes, we don't care about this API anymore


"""
When specified, indicates that this request should be paginated and
Expand All @@ -148,6 +249,8 @@ type GitBlobLSIFData implements TreeEntryLSIFData {

"""
A list of implementations of the symbol under the given document position.

DEPRECATED: Use usagesForSymbol with Implementations filter instead.
"""
implementations(
"""
Expand Down Expand Up @@ -183,6 +286,8 @@ type GitBlobLSIFData implements TreeEntryLSIFData {

"""
A list of prototypes of the symbol under the given document position.

DEPRECATED: Use usagesForSymbol with a Supers filter instead.
"""
prototypes(
"""
Expand Down Expand Up @@ -247,6 +352,216 @@ type GitBlobLSIFData implements TreeEntryLSIFData {
snapshot(indexID: ID!): [SnapshotData!]
}

extend type Query {
"""
Identify usages for either a semantic symbol, or the symbol(s) implied
by a source range.

Ordering and uniqueness guarantees:
1. The usages returned will already be de-duplicated.
2. Results for a single file are contiguous.
3. Results for a single repository are contiguous.

Related: See `codeGraphData` on GitBlob.
"""
usagesForSymbol(
"""
Symbol to perform the lookup for.

If provided, then the start and end position in 'range' are optional.

If not provided, then the start and end position in 'range' must be set.
In that case, the results will be merged across all semantic symbols
in the source range specified by [start, end]
"""
symbol: SymbolComparator,
"""
Information about an existing source range for a usage of the symbol.

TODO: Specify behavior for various cases of overlap between LookupRange
and actual occurrence range.
"""
range: LookupRange!,

filter: UsagesFilter,

"""
When specified, indicates that this request should be paginated and
the first N results (relative to the cursor) should be returned. i.e.
how many results to return per page.
"""
first: Int

"""
When specified, indicates that this request should be paginated and
to fetch results starting at this cursor.
A future request can be made for more results by passing in the
'UsageConnection.pageInfo.endCursor' that is returned.
"""
after: String
): UsageConnection!
}

input SymbolComparator {
name: SymbolNameComparator!
provenance: ProvenanceComparator!
}

# Implementation note: In the future, we may want to extend this
# to allow passing in a suffix or a "symbol pattern".
# See https://github.com/sourcegraph/sourcegraph/issues/59957
input SymbolNameComparator {
equals: String
}

input ProvenanceComparator {
equals: Provenance
varungandhi-src marked this conversation as resolved.
Show resolved Hide resolved
}

type SymbolInformation implements WithProvenance {
"""
Symbol name using syntax specified by the SCIP schema.
https://github.com/sourcegraph/scip/blob/main/scip.proto#L147-L188

This value will generally be used in conjunction with
the `usagesBySymbol` API.
"""
name: String!
"""
Hover documentation for the symbol, in Markdown format.

The caller must take care to escape any particular strings,
such as raw HTML, as necessary.

The value is null when the hover documentation is not found
by 'dataSource'.

The value is empty when `dataSource` is confident that there
is no appropriate hover documentation to display.
"""
documentation: [String!]
"""
Coarse-grained information about the data source.
"""
provenance: Provenance!
"""
Opaque fine-grained information describing the data source.

Provided only for debugging.

This field should be ignored when checking structural equality.
"""
dataSource: String
}

input LookupRange {
"""
Defaults to instance-wide search if the repo is not specified.

This field is necessary if LookupSymbol is a repository-local or
file-local symbol. It is mandatory for cross-repo symbols
due to implementation limitations.
"""
repo: String!
"""
Defaults to HEAD of the default branch if not specified.
"""
revision: String
"""
Defaults to repo-wide search if the path is not specified.

This field is necessary if LookupSymbol is a file-local symbol.
It is mandatory for repository-local and cross-repo symbols
due to implementation limitations.
"""
path: String!
start: LookupPosition
end: LookupPosition
}

input LookupPosition {
"""
Zero-based count of newline (\n or \r\n) characters before this position.
"""
line: Int!
"""
Zero-based UTF-16 code unit offset from preceding newline (\n or \r\n) character.
"""
character: Int!
}

"""
An empty filter allows all kinds of usages for all paths in all repositories.

However, if the symbol used for lookup is a file-local symbol or a
repository-local symbol, then usages will automatically be limited to the
same file or same repository respectively.
"""
input UsagesFilter {
and: [UsagesFilter!]
or: [UsagesFilter!]
not: UsagesFilter
repository: RepositoryFilter
path: PathFilter
kind: SymbolUsageKind
}
varungandhi-src marked this conversation as resolved.
Show resolved Hide resolved

input RepositoryFilter {
name: StringComparator!
}

input PathFilter {
name: StringComparator!
}
Comment on lines +602 to +608
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Should we remove the ! here?


input StringComparator {
equals: String
}

enum SymbolUsageKind {
Definition,
Reference,
Implementation,
Super,
}

type UsageConnection {
nodes: [Usage!]!
pageInfo: PageInfo!
}

type UsageRange {
repo: String!
revision: String!
range: Range!
}

type Usage {
symbol: SymbolInformation!
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Think through the non-null cases more carefully, especially for UsageConnection


range: UsageRange

"""
NOTE: Do not pass
TODO: For this blob, it is generally more useful to get +N/-N lines
rather than having to pass a range internally for the contents since
the offsets from the start will not be known a-priori.
"""
blob: GitBlob!
varungandhi-src marked this conversation as resolved.
Show resolved Hide resolved
"""
Instead of blob { content }, allows accessing a sub-span of the content
using relative coordinates from the range of this usage. If linesBefore
or linesAfter is negative or exceeds the number of available lines,
the value is interpreted as until the start/end of the file.
"""
surroundingBlobContent(surroundingLines: SurroundingLines = {linesBefore: 0, linesAfter: 0}): String!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 meaning "no value given" is a very go-ey thing, I think you don't need a default value here, since surroundingLines: SurroundingLines already indicates nullable and the backend can then decide how to represent that in the given programming language

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, 0 doesn't mean "no value given", it literally means 0 because that makes sense as a default -- only the single line from the blob will be returned in the default case. I wrote the 0 explicitly here as it serves as documentation for an API client for what the behavior will be if they omit this parameter. Another potential default would be INT32_MAX (i.e. get the full blob contents), but we're deliberately picking 0 here, not INT32_MAX.

}

input SurroundingLines {
linesBefore: Int
linesAfter: Int
}

"""
The SCIP snapshot decoration for a single SCIP Occurrence.
"""
Expand Down
Loading
Loading