Scope provider requiring other (resolved) references

[goto40]

I am exploring the langium 0.2.0 project and I try to implement a custom scope provider.

I extracted the problem on a small example (https://github.com/goto40/langium-scoping-example-modeling-method_calls):

The grammar is as follows:

grammar HelloWorld
hidden(WS, SL_COMMENT, ML_COMMENT)

Model: packages+=Pack*;
Pack: 'package' name=ID '{' 
    (class+=Class | instances+=Instance | calls+=Call)* 
    '}';
Class: 'class' name=ID "{" defs+=Def* "}";
Def: 'def' name=ID;
Instance: 'instance' name=ID ':' type=[Class|FQN];
Call: 'call' instance=[Instance|FQN] '->' methodref=[Def];

FQN returns string hidden(): ID ('.' ID)*;

terminal WS: /\s+/;
terminal ID: /[_a-zA-Z][\w_]*/;

terminal ML_COMMENT: /\/\*[\s\S]*?\*\//;
terminal SL_COMMENT: /\/\/[^\n\r]*/;

An example model is used to evaluate the behavior:

package p1 {
    class C1 {
        def a1
    }
    class C2 {
        def b1
    }
    instance i1: C
    instance i2: C2
    call i1->a1
    call i2->b1
}

In the basic configuration I used the hello-world example with the modified grammar.

• Challenge: Then I want to handle the methodref=[Def]; parts with a custom scope provider to resolve the llast two lines of the example, ->a1 and -> b1.
• Note: All other references are correctly resolved (e.g. i1 and i2 in the last two lines). This means I can, e.g., navigate to the definition of ì1 with F12 in vscode.

I tried

export class MyScopeProvider extends DefaultScopeProvider {
    getScope(node: AstNode, referenceId: string): Scope {
        if (referenceId=="Call:methodref" && node.$type=='Call') {
            const scopes: AstNodeDescription[] = [];
            let definitions: Def[]|undefined = [];
            definitions = (node as Call).instance?.ref?.type?.ref?.defs;
            if (definitions) {
                definitions.forEach(element => {
                    scopes.push( { node: element, type: element.$type, name: element.name, documentUri:node.$document!.uri, path: ''} )
                });
                return new SimpleScope(stream(scopes));
            }
            else {
                const result = super.getScope(node, referenceId);
                return result;    
            }
        }
        else {
            const result = super.getScope(node, referenceId);
            return result;
        }
    }
}

It does not work. Obviously it is the wrong solution.

I also observed, that accessing (node as Call).instance?.ref breaks the reference resolution to i1 and i2 (e.g., when accessing the ref field and then calling the super.getScope method)...

[spoenemann]

Your solution approach looks correct. One problem might be node.$document!.uri because $document is defined only on the root node. Use the getDocument utility function:

const descriptions = stream(definitions).map(element =>
        this.descriptionProvider.createDescription(element, element.name, getDocument(element)));
return new SimpleScope(descriptions);

Where this.descriptionProvider = services.index.AstNodeDescriptionProvider is set in the constructor.

[goto40]

Thank you very much for that prompt answer!

I tested you suggestion (https://github.com/goto40/langium-scoping-example-modeling-method_calls/commit/64fdbdf25607b5f39a3ba55af0512377f4972703):

    getScope(node: AstNode, referenceId: string): Scope {
        if (referenceId=="Call:methodref" && node.$type=='Call') {
            let definitions: Def[]|undefined = [];
            // this line breaks the scoping to `i1`/`i2` as illustrated above:
            definitions = (node as Call).instance?.ref?.type?.ref?.defs;
            // ^^^^ Note: commenting the line above makes the references to i1/i2 work again...
            if (definitions!=undefined) {
                // solution suggested here: https://github.com/langium/langium/discussions/401
                const descriptions = stream(definitions).map(element =>
                    this.descriptionProvider.createDescription(element, element.name, getDocument(element)));
                return new SimpleScope(descriptions);
            }
            else {
                const result = super.getScope(node, referenceId);
                return result;    
            }
        }
        else {
            const result = super.getScope(node, referenceId);
            return result;
        }
    }

However, somehow I still have the (same) effect that

• it does not work (same behavior as above)
• accessing ref seems to break the scoping to i1/i2 (which are references stored in the same element as the reference methodref I try to resolve)

Question:
Is it the correct way - in langium 0.2.0 - to access references during reference resolution/scoping - and to wait for references to be resolved...?

• it seems instance?.ref is not resolved or unknown while accessing it. Do I have to wait for it somehow, or to postpone the scope resolution?
• Note: I need access to the instance field, which may not be resolved so far...

[spoenemann]

Conceptually it looks good. You could try whether it works with the intermediate version 0.3.0-next.5e9d27d. If not, please wait for version 0.3.0 which we'd like to publish later this month. If it still does not work, we need to investigate and fix this.

[goto40]

Thank you very much again for the quick reply! Also thank you for the hints - I tried 0.3.0-next.5e9d27d (https://github.com/goto40/langium-scoping-example-modeling-method_calls/commit/25bb06763c4854f68b2b38145f077b95a95a4df9):

• Not sure if I did everything as it should be - but at least I got it compiled (codegen form grammar + build of plugin)
• Unfortunately, I get the same behavior as before...

If you find time you are welcome to have a look (the repo is referenced above). I would be happy to help you if I can do anything to isolate the effect and to find out what went wrong (I am still not sure if I correctly handle/access the (unresolved?) references).

I understand if you focus on the 0.3.0 delivery - in that case I will wait until then to see if this fixes my problems. I am looking forward to see the next release of langium!

[goto40]

I updated to 0.3.0: Now, it seems to work (it resolves the references).
However, I do not get auto-completion for the custom scope provider (I did not investigate).

Thank you.

Notes: grammar and major changes:

grammar HelloWorld

entry Model: packages+=Pack*;
Pack: 'package' name=ID '{' 
    (class+=Class | instances+=Instance | calls+=Call)* 
    '}';
Class: 'class' name=ID "{" defs+=Def* "}";
Def: 'def' name=ID;
Instance: 'instance' name=ID ':' type=[Class:FQN];
Call: 'call' instance=[Instance:FQN] '->' methodref=[Def];

FQN returns string hidden(): ID ('.' ID)*;

hidden terminal WS: /\s+/;
terminal ID: /[_a-zA-Z][\w_]*/;

hidden terminal ML_COMMENT: /\/\*[\s\S]*?\*\//;
hidden terminal SL_COMMENT: /\/\/[^\n\r]*/;

... and in the scope provider return new StreamScope(descriptions);

    getScope(node: AstNode, referenceId: string): Scope {
        console.log(`getScope ${referenceId}: ${node.$type}`);
        if (referenceId=="Call:methodref" && node.$type=='Call') {
            let definitions: Def[]|undefined = [];
            // next line breaks 'i1'/'i2' refs
            definitions = (node as Call).instance?.ref?.type?.ref?.defs;
            console.log(`definitions==${definitions}`);
            if (definitions!==undefined) {
                definitions.forEach( d => { console.log(`name==${d.name}`); });
                // solution suggested here: https://github.com/langium/langium/discussions/401
                const descriptions = stream(definitions).map(element =>
                    this.descriptionProvider.createDescription(element, element.name, getDocument(element)));
                return new StreamScope(descriptions);
            }
            else {
                const result = super.getScope(node, referenceId);
                return result;    
            }
        }
        else {
            const result = super.getScope(node, referenceId);
            return result;
        }
    }

[spoenemann]

Great that it's working now!

@msujew is planning to reimplement the basis of the completion service. But feel free to report issues on that anyway.

[msujew]

@goto40 I just noticed a minor bug in the completion provider which might prevent accurate completions for references. Can you test whether always using a cross reference terminal (i.e. methodref=[Def] -> methodref=[Def:ID]) fixes the completion issue?

[goto40]

Great work, @msujew ! That solved my auto-completion issue... Now I get the correct behavior. Thank you!

Btw.: Do we have examples and/or docu concerning scope providers (except for the three examples in the langium repository), especially for the case where I build and return a StreamScope... Maybe I did not look good enough?