Getting two references out of something like tableName.fieldName

[stutrek]

I'm building a language server for our ERD syntax at Eraser.io. Here's a simple example, with two tables and one connection between them:

users [icon: user, color: blue] {
  id string pk
  teams string
}

teams [icon: users, color: blue] {
  id string pk
  name string
}

users.teams <> teams.id

The problem I'm having right now is linking the table and field names in the connection. Ideally, in a connection, the field name would be linked to the field name in the declaration, and the table name linked to the table declaration.

With my current code, the main connection is not linked at all, and the target is linked only to the field name.

grammar:
The ErdConnectionFieldName entity is its own thing so that I have access to it as a type. If there is a better way, I would be happy to change it.

grammar EraserErd

entry Model:
    (tables+=(TableDeclaration) | connections+=(ErdConnection) | Newline)*;

TableDeclaration:
    name=ID ('[' (settings+=Setting ','?)* ']')? ('{' Newline* (fields+=FieldDefinition*)* '}')? (Newline | EOF);

FieldDefinition:
    name=ID type=ID (keyType=ID)? Newline+;

Setting:
    key=ID ':' value=ID|EscapedText;

ErdConnection:
    table=[TableConnectionItem:ID] field=ErdConnectionFieldName? connectionType=ConnectionToken (to+=ErdConnectionTarget) (Newline | EOF);

ErdConnectionTarget:
    table=[TableConnectionItem:ID] field=ErdConnectionFieldName? ','?;

ErdConnectionFieldName:
    '.' field=[FieldConnectionItem:ID];

type TableConnectionItem = TableDeclaration | ErdConnection | ErdConnectionTarget;
type FieldConnectionItem = FieldDefinition | ErdConnectionFieldName;


hidden terminal WS: /[ \t]+/;

hidden terminal ML_COMMENT: /\/\*[\s\S]*?\*\//;
hidden terminal SL_COMMENT: /(\/\/)[^\n\r]*/;
hidden terminal SL_COMMENT_HASH: /#[^\n\r]*/;

terminal ID: /([^:{}[\]"<>,\n -]+[^:{}[\]"<>,\n ]{0,1})+[^:{}[\]"<>,\n -]|[^:{}[\]"<>,\n -]{1,1}/;
terminal ConnectionToken: /[<>-]+/;
terminal EscapedText: /"[^"]+"/;
terminal Newline: /\n/;

classes:

export class ErdReferences extends DefaultReferences {
  findReferences(
    targetNode: AstNode,
    options: FindReferencesOptions
  ): Stream<ReferenceDescription> {
    // console.log('findReferences', targetNode, options);
    const refs: ReferenceDescription[] = [];
    if (options.includeDeclaration) {
      const ref = this.getReferenceToSelf(targetNode);
      if (ref) {
        refs.push(ref);
      }
    }
    let indexReferences = this.index.findAllReferences(
      targetNode,
      this.nodeLocator.getAstNodePath(targetNode)
    );
    if (options.documentUri) {
      indexReferences = indexReferences.filter((ref) =>
        UriUtils.equals(ref.sourceUri, options.documentUri)
      );
    }
    const filtered = indexReferences.filter((ref) => ref.segment.offset !== refs[0].segment.offset);
    refs.push(...filtered);
    return stream(refs);
  }
}

export class ErdNameProvider extends DefaultNameProvider {
  getName(node: AstNode): string | undefined {
    if (isErdConnectionFieldName(node)) {
      // console.log(
      //   'isErdConnectionFieldName',
      //   node.$container.table.$refText + '.' + node.field.$refText
      // );
      return node.$container.table.$refText + '.' + node.field.$refText;
    } else if (isErdConnection(node) || isErdConnectionTarget(node)) {
      return node.table.$refText; // Use the reference text as the name of the node
    } else if (isFieldDefinition(node)) {
      // console.log('isFieldDefinition', node.$container.name + '.' + node.name);
      return node.$container.name + '.' + node.name;
    } else {
      return super.getName(node);
    }
  }

  getNameNode(node: AstNode): CstNode | undefined {
    if (isErdConnectionFieldName(node)) {
      // console.log('isErdConnectionFieldName', node);
      return node.field.$refNode;
    } else if (isErdConnection(node) || isErdConnectionTarget(node)) {
      return node.table.$refNode;
    } else {
      return super.getNameNode(node);
    }
  }
}

// the goal of this is to flatten the document so connections can access fields
export class ErdScopeProvider extends DefaultScopeProvider {
  getScope(context: ReferenceInfo): Scope {
    const scopes: Array<Stream<AstNodeDescription>> = [];
    const referenceType = this.reflection.getReferenceType(context);

    const precomputed = AstUtils.getDocument(context.container).precomputedScopes;

    if (precomputed) {
      const nodeDescriptions: AstNodeDescription[] = [];
      const found = new Set<string>();
      for (const [node, nodeDescription] of precomputed) {
        const name = isFieldDefinition(node)
          ? node.$container.name + '.' + nodeDescription.name
          : nodeDescription.name;
        if (!found.has(name)) {
          nodeDescriptions.push(nodeDescription);
          found.add(name);
        }
      }
      scopes.push(stream(nodeDescriptions));
    }

    let result: Scope = this.getGlobalScope(referenceType, context);
    for (let i = scopes.length - 1; i >= 0; i--) {
      result = this.createScope(scopes[i], result);
    }
    return result;
  }
}

Thanks for your help

[msujew]

Hey @stutrek,

your NameProvider implementation is doing something very unexpected. It returns ${node.$container.name}.${node.name} for any field definition. As a consequence, only references that parse as teams.id are capable of actually referencing the id field - however, this isn't how Langium is resolving the reference. That's because teams and id are two separate references, none of which have teams.id as their reference text.

It is enough to return node.name (and node.field.$refText for ErdConnectionFieldName).

Additionally, you need to adjust the scope provider for the ErdConnectionFieldName#field property. Something like this:

getScope(context: ReferenceInfo): Scope {
    if (context.property === 'field') {
      const container = context.container.$container as ErdConnectionTarget | ErdConnection;
      const table = container.ref.table;
      if (table) {
        return this.createScopeForNodes(table.fields);
      }
      return EMPTY_SCOPE;
    }
    return super.getScope(context);
  }

[msujew]

The highlight provider highlights whatever you return as the NameProvider#getNameNode. If you return a larger section of text, then it everything inside of it gets highlighted.

As for the issue at hand: It's difficult to tell what exactly is wrong or how to fix it. Can you provide a reproducible example or is the whole code somewhere available to clone?

[stutrek]

Thanks! I appreciate your help!

Let me try tweaking getNameNode, right now it's in our monorepo, but we don't consider our DSL a trade secret, so if I can't get it working I'll try to set up a codesandbox.

[stutrek]

I was not able to figure it out :)

Here's a repo with the code, it's the langium boilerplate with some additions, so yarn langium:watch and yarn dev get it working. You should be able to start a codespace with it. Relevant files are here https://github.com/stutrek/eraser-dsl/tree/main/src/language/erd

[msujew]

See https://github.com/stutrek/eraser-dsl/compare/main...msujew:eraser-dsl:fix?expand=1 for all the necessary changes for this to work :)

[stutrek]

Wow, thanks! You have an even more special place in our hearts now. If you have an Eraser account, send me your email address!

[msujew]

@stutrek I don't, but if you want to support us for our continued effort on Langium and other open source projects, consider sponsoring us :)

[spoenemann]

Thank you so much @stutrek for your sponsorship! ❤️