SwiftSyntax support for module selectors #3091

beccadax · 2025-06-02T19:44:22Z

This PR adds module selectors to SwiftSyntax and SwiftParser (draft of matching compiler feature in swiftlang/swift#34556). This feature was pitched ages ago; a proper proposal is ~~on my todo list~~ now available.

Note: This PR is still a work in progress—in particular, I need to go back and improve the tests, both by expanding their coverage and by adding assertions about the resulting syntax trees—but I'd appreciate feedback on the design while I'm working on that.

Reviewers: If you review this commit-by-commit, I've separated out the big dumb mechanical changes from the ones that actually change parsing logic.

nkcsgexi · 2025-06-02T19:47:56Z

🥳

Sources/SwiftParser/Names.swift

This capability is currently unused, but it’ll matter soon.

And make sure it doesn’t break Objective-C selector lexing.

Treating introducers as keywords is now always conditional on whether `shouldParsePatternBinding(introducer:)` returns `true`. That method has also been modified to correctly handle an edge case with wildcard patterns.

Initializers for nodes with experimental node children need to be marked `@_spi`. This PR: • Adds that attribute. • Generates an alternative which *doesn’t* use SPI as part of the compatibility layer. • As a side effect, adds a `Child.Refactoring.introduced` case that can be used to generate compatibility `unexpected` properties. No functional change in this commit, but it will affect the code generation in the next one.

beccadax · 2025-07-10T03:10:31Z

@swift-ci please test

beccadax · 2025-07-11T00:11:22Z

@swift-ci please test

beccadax · 2025-07-11T20:50:43Z

@swift-ci please test

beccadax · 2025-07-12T04:10:37Z

@swift-ci please test

nkcsgexi · 2025-07-14T18:41:10Z

@swift-ci please test macOS

nkcsgexi · 2025-07-14T18:41:24Z

@swift-ci please test Linux

nkcsgexi · 2025-07-14T21:06:15Z

This build failure seems to be real:

12:29:57  /Users/ec2-user/jenkins/workspace/swift-syntax-PR-macOS/branch-main/swift-syntax/Sources/SwiftSyntax/generated/SyntaxTraits.swift:209:7: error: protocol requirement 'moduleSelector' cannot be declared '@_spi' without a default implementation in a protocol extension
12:29:57  207 | 
12:29:57  208 |   @_spi(ExperimentalLanguageFeatures)
12:29:57  209 |   var moduleSelector: ModuleSelectorSyntax? {
12:29:57      |       `- error: protocol requirement 'moduleSelector' cannot be declared '@_spi' without a default implementation in a protocol extension
12:29:57  210 |     get
12:29:57  211 |     set

beccadax · 2025-07-15T00:09:10Z

I can't actually reproduce that failure locally, but I'm pushing a speculative fix.

beccadax · 2025-07-15T00:20:09Z

@swift-ci please test

beccadax · 2025-07-15T01:39:59Z

@swift-ci please test macOS

beccadax · 2025-07-15T01:40:08Z

@swift-ci please test Linux

beccadax · 2025-07-15T01:40:19Z

@swift-ci please test Windows

nkcsgexi · 2025-07-15T17:21:22Z

We are good now with macOS and Linux. Let's try Windows again. @swift-ci please test Windows.

nkcsgexi · 2025-07-15T21:54:33Z

hmm, the Windows CI hit a compiler crasher:

[519/528] Compiling SwiftDiagnostics Convenience.swift
error: compile command failed due to exception 3 (use -v to see invocation)
SIL memory lifetime failure in @$s11SwiftParser13TokenConsumerPAAE27consumeModuleSelectorTokens0C0Qz22moduleNameOrUnexpected_AF010colonColonC0SayAFG5extratSgyF: memory is initialized, but shouldn't be

rintaro

Apologies I didn't review this sooner. I still haven't look into the implementation closely, but here's the first round 🙇

CodeGeneration/Sources/SyntaxSupport/CommonNodes.swift

Sources/SwiftParser/Names.swift

rintaro · 2025-07-16T18:56:59Z

Sources/SwiftParser/Names.swift

+    // Technically the current token *should* be an identifier, but we also want to diagnose other tokens that might be
+    // used by accident (particularly keywords and `_`). However, we don't want to consume tokens which would make the
+    // surrounding structure mis-parse.
+    return self.at(anyIn: StructuralTokens.self) == nil


I don't think it's a good idea to (almost) always consider <token> :: as a module qualifier. E..g.

class ::

This looks to me an incomplete class declaration and just an orphan ::. Even if it's on the same line with no-space, I don't think we need to parse it as a module selector.

Changes the syntax tree to represent module selectors: • A `ModuleSelectorSyntax` node represents a module selector abstractly. • The following nodes now have an optional `moduleSelector` child: • `DeclReferenceExprSyntax` • `IdentifierTypeSyntax` • `MacroExpansionExprSyntax` • `MemberTypeSyntax` • BasicFormat knows the preferred format for module selectors. Other components, particularly the parser, were also updated to continue building, though without any changes in behavior. Parser implementation will come in a future commit.

Changes it to share code with `parseTypeIdentifier()` and clean up the member type parsing a little. Also tweaks call sites of `parseTypeIdentifier()`.

beccadax · 2025-07-23T22:08:07Z

Okay, this is a dramatically simpler implementation that doesn't try to do anything special to handle module selectors at invalid locations. I can't say I'm happy with some of the recovery behavior I'm seeing—in particular, parameter lists and tuple patterns seem to just throw up their hands in a pretty ugly way instead of recovering to a TokenKind.comma—but maybe that's a more general problem.

This commit ports over tests from the compiler’s (future) `test/NameLookup/module_selector.swift` file and makes sure the correct uses parse as expected. It also tests that ill-formed module selectors (ones with a missing or non-identifier module name) are diagnosed correctly. This commit doesn’t handle module selectors in scoped `import` statements; the related test has been XFAILed.

rintaro · 2025-07-23T22:59:47Z

Sources/SwiftParser/Names.swift

+
+extension Parser {
+  /// Parses one or more module selectors, if present.
+  mutating func parseModuleSelector() -> RawModuleSelectorSyntax? {


I know there're several parsing functions that doesn't follow this convention, but:

Suggested change

mutating func parseModuleSelector() -> RawModuleSelectorSyntax? {

mutating func parseModuleSelectorIfPresent() -> RawModuleSelectorSyntax? {

or check self.isAtModuleSelector() before calling this and make this non-optional.

rintaro · 2025-07-23T23:09:03Z

Sources/SwiftParser/Names.swift

@@ -68,6 +213,9 @@ extension Parser {
  }

  mutating func parseDeclReferenceExpr(_ flags: DeclNameOptions = []) -> RawDeclReferenceExprSyntax {
+    // Consume a module selector if present.
+    let moduleSelector = self.parseModuleSelector()


I think presence of the module selector should affect flags (DeclNameOptions.keywords).
It would be weird if ModName::default is rejected, but foo.ModName::default is allowed.
related: https://forums.swift.org/t/pitch-module-selectors/80835/82

rintaro · 2025-07-23T23:53:18Z

Sources/SwiftParser/Names.swift

+    }
+
+    // Technically the current token *should* be an identifier, but we also want to diagnose other tokens that might be
+    // used by accident (particularly keywords and `_`). However, we don't want to consume tokens which would make the


Would you show me some examples where consuming keywords as a invalid module identifiers improves the diagnostics?

IMO, keyword as a module name rule should follow existing keyword rules should be like:

Bare keywords as module names are allowed after . (following SE-0071). Otherwise it's not a module selector even if there's :: after that

Some special keywords such as Self, self and Any should be parsed as reserved names (even without .). And the type checker should reject/diagnose them accordingly.

beccadax force-pushed the mod-squad branch from a20cd45 to d7511a3 Compare June 2, 2025 19:48

beccadax commented Jun 2, 2025

View reviewed changes

Sources/SwiftParser/Names.swift Outdated Show resolved Hide resolved

beccadax force-pushed the mod-squad branch from d7511a3 to 4c34e4b Compare June 4, 2025 00:57

[NFC] Thread experimental features through lexer

69d50cd

This capability is currently unused, but it’ll matter soon.

beccadax force-pushed the mod-squad branch 3 times, most recently from 1f47aa2 to eaa6956 Compare July 9, 2025 22:04

beccadax added 5 commits July 9, 2025 15:24

Add experimental colonColon token

1360c37

And make sure it doesn’t break Objective-C selector lexing.

[NFC] Refactor parsing of suppressed types

06f2097

[NFC] Tighten checking for PBD introducers

ee93aa6

Treating introducers as keywords is now always conditional on whether `shouldParsePatternBinding(introducer:)` returns `true`. That method has also been modified to correctly handle an edge case with wildcard patterns.

[NFC] Correct @_implements diagnostic typo

d848067

beccadax force-pushed the mod-squad branch 2 times, most recently from da6eff6 to f94b58a Compare July 10, 2025 03:00

beccadax force-pushed the mod-squad branch from f94b58a to 7652336 Compare July 10, 2025 21:17

beccadax marked this pull request as ready for review July 11, 2025 20:50

beccadax requested review from ahoppen, bnbarham and hamishknight as code owners July 11, 2025 20:50

beccadax force-pushed the mod-squad branch from 7652336 to 8adfdb1 Compare July 11, 2025 23:26

beccadax requested a review from rintaro July 15, 2025 18:21

rintaro reviewed Jul 16, 2025

View reviewed changes

beccadax added 2 commits July 17, 2025 15:15

[NFC] Refactor parseQualifiedTypeIdentifier()

056ec21

Changes it to share code with `parseTypeIdentifier()` and clean up the member type parsing a little. Also tweaks call sites of `parseTypeIdentifier()`.

beccadax force-pushed the mod-squad branch from 46eb9ee to bb512ff Compare July 23, 2025 22:01

beccadax added 2 commits July 23, 2025 16:29

Support module selectors in scoped imports

533ae59

beccadax force-pushed the mod-squad branch from bb512ff to 533ae59 Compare July 23, 2025 23:42

rintaro reviewed Jul 23, 2025

View reviewed changes

	mutating func parseModuleSelector() -> RawModuleSelectorSyntax? {
	mutating func parseModuleSelectorIfPresent() -> RawModuleSelectorSyntax? {

SwiftSyntax support for module selectors #3091

Are you sure you want to change the base?

SwiftSyntax support for module selectors #3091

Conversation

beccadax commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nkcsgexi commented Jun 2, 2025

Uh oh!

Uh oh!

beccadax commented Jul 10, 2025

Uh oh!

beccadax commented Jul 11, 2025

Uh oh!

beccadax commented Jul 11, 2025

Uh oh!

beccadax commented Jul 12, 2025

Uh oh!

nkcsgexi commented Jul 14, 2025

Uh oh!

nkcsgexi commented Jul 14, 2025

Uh oh!

nkcsgexi commented Jul 14, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

beccadax commented Jul 15, 2025

Uh oh!

nkcsgexi commented Jul 15, 2025

Uh oh!

nkcsgexi commented Jul 15, 2025

Uh oh!

rintaro left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rintaro Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

beccadax commented Jul 23, 2025

Uh oh!

rintaro Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rintaro Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

rintaro Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

beccadax commented Jun 2, 2025 •

edited

Loading

rintaro left a comment •

edited

Loading

rintaro Jul 16, 2025 •

edited

Loading

rintaro Jul 23, 2025 •

edited

Loading