Skip to content

SwiftSyntax support for module selectors #3091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

beccadax
Copy link
Contributor

@beccadax beccadax commented Jun 2, 2025

This PR adds module selectors to SwiftSyntax and SwiftParser (draft of matching compiler feature in swiftlang/swift#34556). This feature was pitched ages ago; a proper proposal is on my todo list now available.

Note: This PR is still a work in progress—in particular, I need to go back and improve the tests, both by expanding their coverage and by adding assertions about the resulting syntax trees—but I'd appreciate feedback on the design while I'm working on that.

Reviewers: If you review this commit-by-commit, I've separated out the big dumb mechanical changes from the ones that actually change parsing logic.

@nkcsgexi
Copy link
Contributor

nkcsgexi commented Jun 2, 2025

🥳

This capability is currently unused, but it’ll matter soon.
@beccadax beccadax force-pushed the mod-squad branch 3 times, most recently from 1f47aa2 to eaa6956 Compare July 9, 2025 22:04
beccadax added 5 commits July 9, 2025 15:24
And make sure it doesn’t break Objective-C selector lexing.
Treating introducers as keywords is now always conditional on whether `shouldParsePatternBinding(introducer:)` returns `true`. That method has also been modified to correctly handle an edge case with wildcard patterns.
Initializers for nodes with experimental node children need to be marked `@_spi`. This PR:

• Adds that attribute.
• Generates an alternative which *doesn’t* use SPI as part of the compatibility layer.
• As a side effect, adds a `Child.Refactoring.introduced` case that can be used to generate compatibility `unexpected` properties.

No functional change in this commit, but it will affect the code generation in the next one.
@beccadax beccadax force-pushed the mod-squad branch 2 times, most recently from da6eff6 to f94b58a Compare July 10, 2025 03:00
@beccadax
Copy link
Contributor Author

@swift-ci please test

@beccadax
Copy link
Contributor Author

@swift-ci please test

1 similar comment
@beccadax
Copy link
Contributor Author

@swift-ci please test

@beccadax beccadax marked this pull request as ready for review July 11, 2025 20:50
@beccadax
Copy link
Contributor Author

@swift-ci please test

@nkcsgexi
Copy link
Contributor

@swift-ci please test macOS

@nkcsgexi
Copy link
Contributor

@swift-ci please test Linux

@nkcsgexi
Copy link
Contributor

This build failure seems to be real:

12:29:57  /Users/ec2-user/jenkins/workspace/swift-syntax-PR-macOS/branch-main/swift-syntax/Sources/SwiftSyntax/generated/SyntaxTraits.swift:209:7: error: protocol requirement 'moduleSelector' cannot be declared '@_spi' without a default implementation in a protocol extension
12:29:57  207 | 
12:29:57  208 |   @_spi(ExperimentalLanguageFeatures)
12:29:57  209 |   var moduleSelector: ModuleSelectorSyntax? {
12:29:57      |       `- error: protocol requirement 'moduleSelector' cannot be declared '@_spi' without a default implementation in a protocol extension
12:29:57  210 |     get
12:29:57  211 |     set

@beccadax
Copy link
Contributor Author

I can't actually reproduce that failure locally, but I'm pushing a speculative fix.

@beccadax
Copy link
Contributor Author

@swift-ci please test

@beccadax
Copy link
Contributor Author

@swift-ci please test macOS

@beccadax
Copy link
Contributor Author

@swift-ci please test Linux

@beccadax
Copy link
Contributor Author

@swift-ci please test Windows

@nkcsgexi
Copy link
Contributor

We are good now with macOS and Linux. Let's try Windows again. @swift-ci please test Windows.

@beccadax beccadax requested a review from rintaro July 15, 2025 18:21
@nkcsgexi
Copy link
Contributor

hmm, the Windows CI hit a compiler crasher:

[519/528] Compiling SwiftDiagnostics Convenience.swift
error: compile command failed due to exception 3 (use -v to see invocation)
SIL memory lifetime failure in @$s11SwiftParser13TokenConsumerPAAE27consumeModuleSelectorTokens0C0Qz22moduleNameOrUnexpected_AF010colonColonC0SayAFG5extratSgyF: memory is initialized, but shouldn't be

Copy link
Member

@rintaro rintaro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies I didn't review this sooner. I still haven't look into the implementation closely, but here's the first round 🙇

Comment on lines +116 to +108
// Technically the current token *should* be an identifier, but we also want to diagnose other tokens that might be
// used by accident (particularly keywords and `_`). However, we don't want to consume tokens which would make the
// surrounding structure mis-parse.
return self.at(anyIn: StructuralTokens.self) == nil
Copy link
Member

@rintaro rintaro Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's a good idea to (almost) always consider <token> :: as a module qualifier. E..g.

class
::

This looks to me an incomplete class declaration and just an orphan ::. Even if it's on the same line with no-space, I don't think we need to parse it as a module selector.

beccadax added 2 commits July 17, 2025 15:15
Changes the syntax tree to represent module selectors:

• A `ModuleSelectorSyntax` node represents a module selector abstractly.
• The following nodes now have an optional `moduleSelector` child:
    • `DeclReferenceExprSyntax`
    • `IdentifierTypeSyntax`
    • `MacroExpansionExprSyntax`
    • `MemberTypeSyntax`
• BasicFormat knows the preferred format for module selectors.

Other components, particularly the parser, were also updated to continue building, though without any changes in behavior. Parser implementation will come in a future commit.
Changes it to share code with `parseTypeIdentifier()` and clean up the member type parsing a little. Also tweaks call sites of `parseTypeIdentifier()`.
@beccadax
Copy link
Contributor Author

Okay, this is a dramatically simpler implementation that doesn't try to do anything special to handle module selectors at invalid locations. I can't say I'm happy with some of the recovery behavior I'm seeing—in particular, parameter lists and tuple patterns seem to just throw up their hands in a pretty ugly way instead of recovering to a TokenKind.comma—but maybe that's a more general problem.

beccadax added 2 commits July 23, 2025 16:29
This commit ports over tests from the compiler’s (future) `test/NameLookup/module_selector.swift` file and makes sure the correct uses parse as expected. It also tests that ill-formed module selectors (ones with a missing or non-identifier module name) are diagnosed correctly.

This commit doesn’t handle module selectors in scoped `import` statements; the related test has been XFAILed.

extension Parser {
/// Parses one or more module selectors, if present.
mutating func parseModuleSelector() -> RawModuleSelectorSyntax? {
Copy link
Member

@rintaro rintaro Jul 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know there're several parsing functions that doesn't follow this convention, but:

Suggested change
mutating func parseModuleSelector() -> RawModuleSelectorSyntax? {
mutating func parseModuleSelectorIfPresent() -> RawModuleSelectorSyntax? {

or check self.isAtModuleSelector() before calling this and make this non-optional.

@@ -68,6 +213,9 @@ extension Parser {
}

mutating func parseDeclReferenceExpr(_ flags: DeclNameOptions = []) -> RawDeclReferenceExprSyntax {
// Consume a module selector if present.
let moduleSelector = self.parseModuleSelector()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think presence of the module selector should affect flags (DeclNameOptions.keywords).
It would be weird if ModName::default is rejected, but foo.ModName::default is allowed.
related: https://forums.swift.org/t/pitch-module-selectors/80835/82

}

// Technically the current token *should* be an identifier, but we also want to diagnose other tokens that might be
// used by accident (particularly keywords and `_`). However, we don't want to consume tokens which would make the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you show me some examples where consuming keywords as a invalid module identifiers improves the diagnostics?

IMO, keyword as a module name rule should follow existing keyword rules should be like:

  • Bare keywords as module names are allowed after . (following SE-0071). Otherwise it's not a module selector even if there's :: after that
  • Some special keywords such as Self, self and Any should be parsed as reserved names (even without .). And the type checker should reject/diagnose them accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants