benchmarks failing from main

[vanvoorden]

Describe the bug
Benchmarks are failing to build from main eb084da. It looks related to the changes landed in #1198. We might be missing some version gating in the benchmarks themselves.

Configuration (please complete the following information):

• swift-driver version: swift-driver version: 1.127.5.3 Apple Swift version 6.2 (swiftlang-6.2.0.10.950 clang-1700.3.10.950)
• Target: arm64-apple-macosx15.0
• OS Version: macOS 15.5 (24F74)

Actual behavior

swift package --disable-sandbox -c release benchmark run                                                          
warning: 'benchmarks': swift-foundation benchmarks: Using system Foundation.framework
Building for production...
ld: warning: building for macOS-13.0, but linking with dylib '/opt/homebrew/opt/jemalloc/lib/libjemalloc.2.dylib' which was built for newer version 14.0
[7/7] Linking BenchmarkTool-tool
Build of product 'BenchmarkTool' complete! (14.57s)
Build complete!
Building BenchmarkTool in release mode...
Building benchmark targets in release mode for benchmark run...
Building URLBenchmarks
[0/2] Write swift-version-40EA24106824B483.txt
Building for production...
[0/8] Compiling CSystem shims.c
[1/8] Write swift-version-40EA24106824B483.txt
[3/10] Compiling ArgumentParserToolInfo ToolInfo.swift
[4/11] Compiling SystemPackage Errno.swift
[5/11] Compiling ArgumentParser BashCompletionsGenerator.swift
[6/12] Compiling BenchmarkBoilerplateGenerator BenchmarkBoilerplateGenerator.swift
[6/12] Write Objects.LinkFileList
[7/12] Linking BenchmarkBoilerplateGenerator-tool
[8/12] Generating plugin support files
[9/12] Write sources
[11/13] Compiling URLBenchmarks BenchmarkURL.swift
/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:250:25: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
248 |     ]
249 | 
250 |     var variables: [URL.Template.VariableName: URL.Template.Value] = [
    |                         |- error: 'Template' is only available in macOS 26.0 or newer
    |                         `- note: add 'if #available' version check
251 |         .init("count"): ["one", "two", "three"],
252 |         .init("dom"): ["example", "com"],

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:250:52: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
248 |     ]
249 | 
250 |     var variables: [URL.Template.VariableName: URL.Template.Value] = [
    |                                                    |- error: 'Template' is only available in macOS 26.0 or newer
    |                                                    `- note: add 'if #available' version check
251 |         .init("count"): ["one", "two", "three"],
252 |         .init("dom"): ["example", "com"],

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:236:27: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
234 |     Benchmark("URL-Template-parsing") { benchmark in
235 |         for _ in benchmark.scaledIterations {
236 |             blackHole(URL.Template("/api/{version}/accounts/{accountId}/transactions/{transactionId}{?expand*,fields*,embed*,format}")!)
    |                           |- error: 'Template' is only available in macOS 26.0 or newer
    |                           `- note: add 'if #available' version check
237 |             blackHole(URL.Template("/special/{+a}/details")!)
238 |             blackHole(URL.Template("/documents/{documentId}{#section,paragraph}")!)

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:236:27: error: 'init(_:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
234 |     Benchmark("URL-Template-parsing") { benchmark in
235 |         for _ in benchmark.scaledIterations {
236 |             blackHole(URL.Template("/api/{version}/accounts/{accountId}/transactions/{transactionId}{?expand*,fields*,embed*,format}")!)
    |                           |- error: 'init(_:)' is only available in macOS 26.0 or newer
    |                           `- note: add 'if #available' version check
237 |             blackHole(URL.Template("/special/{+a}/details")!)
238 |             blackHole(URL.Template("/documents/{documentId}{#section,paragraph}")!)

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:237:27: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
235 |         for _ in benchmark.scaledIterations {
236 |             blackHole(URL.Template("/api/{version}/accounts/{accountId}/transactions/{transactionId}{?expand*,fields*,embed*,format}")!)
237 |             blackHole(URL.Template("/special/{+a}/details")!)
    |                           |- error: 'Template' is only available in macOS 26.0 or newer
    |                           `- note: add 'if #available' version check
238 |             blackHole(URL.Template("/documents/{documentId}{#section,paragraph}")!)
239 |         }

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:237:27: error: 'init(_:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
235 |         for _ in benchmark.scaledIterations {
236 |             blackHole(URL.Template("/api/{version}/accounts/{accountId}/transactions/{transactionId}{?expand*,fields*,embed*,format}")!)
237 |             blackHole(URL.Template("/special/{+a}/details")!)
    |                           |- error: 'init(_:)' is only available in macOS 26.0 or newer
    |                           `- note: add 'if #available' version check
238 |             blackHole(URL.Template("/documents/{documentId}{#section,paragraph}")!)
239 |         }

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:238:27: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
236 |             blackHole(URL.Template("/api/{version}/accounts/{accountId}/transactions/{transactionId}{?expand*,fields*,embed*,format}")!)
237 |             blackHole(URL.Template("/special/{+a}/details")!)
238 |             blackHole(URL.Template("/documents/{documentId}{#section,paragraph}")!)
    |                           |- error: 'Template' is only available in macOS 26.0 or newer
    |                           `- note: add 'if #available' version check
239 |         }
240 |     }

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:238:27: error: 'init(_:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
236 |             blackHole(URL.Template("/api/{version}/accounts/{accountId}/transactions/{transactionId}{?expand*,fields*,embed*,format}")!)
237 |             blackHole(URL.Template("/special/{+a}/details")!)
238 |             blackHole(URL.Template("/documents/{documentId}{#section,paragraph}")!)
    |                           |- error: 'init(_:)' is only available in macOS 26.0 or newer
    |                           `- note: add 'if #available' version check
239 |         }
240 |     }

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:243:13: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
241 | 
242 |     let templates = [
243 |         URL.Template("/var/{var}/who/{who}/x/{x}{?keys*,count*,list*,y}")!,
    |             |- error: 'Template' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
244 |         URL.Template("/special/{+keys}/details")!,
245 |         URL.Template("x/y/{#path:6}/here")!,

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:243:13: error: 'init(_:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
241 | 
242 |     let templates = [
243 |         URL.Template("/var/{var}/who/{who}/x/{x}{?keys*,count*,list*,y}")!,
    |             |- error: 'init(_:)' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
244 |         URL.Template("/special/{+keys}/details")!,
245 |         URL.Template("x/y/{#path:6}/here")!,

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:244:13: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
242 |     let templates = [
243 |         URL.Template("/var/{var}/who/{who}/x/{x}{?keys*,count*,list*,y}")!,
244 |         URL.Template("/special/{+keys}/details")!,
    |             |- error: 'Template' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
245 |         URL.Template("x/y/{#path:6}/here")!,
246 |         URL.Template("a/b{/var,x}/here")!,

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:244:13: error: 'init(_:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
242 |     let templates = [
243 |         URL.Template("/var/{var}/who/{who}/x/{x}{?keys*,count*,list*,y}")!,
244 |         URL.Template("/special/{+keys}/details")!,
    |             |- error: 'init(_:)' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
245 |         URL.Template("x/y/{#path:6}/here")!,
246 |         URL.Template("a/b{/var,x}/here")!,

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:245:13: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
243 |         URL.Template("/var/{var}/who/{who}/x/{x}{?keys*,count*,list*,y}")!,
244 |         URL.Template("/special/{+keys}/details")!,
245 |         URL.Template("x/y/{#path:6}/here")!,
    |             |- error: 'Template' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
246 |         URL.Template("a/b{/var,x}/here")!,
247 |         URL.Template("a{?var,y}")!,

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:245:13: error: 'init(_:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
243 |         URL.Template("/var/{var}/who/{who}/x/{x}{?keys*,count*,list*,y}")!,
244 |         URL.Template("/special/{+keys}/details")!,
245 |         URL.Template("x/y/{#path:6}/here")!,
    |             |- error: 'init(_:)' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
246 |         URL.Template("a/b{/var,x}/here")!,
247 |         URL.Template("a{?var,y}")!,

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:246:13: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
244 |         URL.Template("/special/{+keys}/details")!,
245 |         URL.Template("x/y/{#path:6}/here")!,
246 |         URL.Template("a/b{/var,x}/here")!,
    |             |- error: 'Template' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
247 |         URL.Template("a{?var,y}")!,
248 |     ]

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:246:13: error: 'init(_:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
244 |         URL.Template("/special/{+keys}/details")!,
245 |         URL.Template("x/y/{#path:6}/here")!,
246 |         URL.Template("a/b{/var,x}/here")!,
    |             |- error: 'init(_:)' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
247 |         URL.Template("a{?var,y}")!,
248 |     ]

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:247:13: error: 'Template' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
245 |         URL.Template("x/y/{#path:6}/here")!,
246 |         URL.Template("a/b{/var,x}/here")!,
247 |         URL.Template("a{?var,y}")!,
    |             |- error: 'Template' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
248 |     ]
249 | 

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:247:13: error: 'init(_:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
245 |         URL.Template("x/y/{#path:6}/here")!,
246 |         URL.Template("a/b{/var,x}/here")!,
247 |         URL.Template("a{?var,y}")!,
    |             |- error: 'init(_:)' is only available in macOS 26.0 or newer
    |             `- note: add 'if #available' version check
248 |     ]
249 | 

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:251:25: error: 'init(arrayLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
249 | 
250 |     var variables: [URL.Template.VariableName: URL.Template.Value] = [
251 |         .init("count"): ["one", "two", "three"],
    |                         |- error: 'init(arrayLiteral:)' is only available in macOS 26.0 or newer
    |                         `- note: add 'if #available' version check
252 |         .init("dom"): ["example", "com"],
253 |         .init("dub"): "me/too",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:252:23: error: 'init(arrayLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
250 |     var variables: [URL.Template.VariableName: URL.Template.Value] = [
251 |         .init("count"): ["one", "two", "three"],
252 |         .init("dom"): ["example", "com"],
    |                       |- error: 'init(arrayLiteral:)' is only available in macOS 26.0 or newer
    |                       `- note: add 'if #available' version check
253 |         .init("dub"): "me/too",
254 |         .init("hello"): "Hello World!",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:253:23: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
251 |         .init("count"): ["one", "two", "three"],
252 |         .init("dom"): ["example", "com"],
253 |         .init("dub"): "me/too",
    |                       |- error: 'Value' is only available in macOS 26.0 or newer
    |                       `- note: add 'if #available' version check
254 |         .init("hello"): "Hello World!",
255 |         .init("half"): "50%",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:253:23: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
251 |         .init("count"): ["one", "two", "three"],
252 |         .init("dom"): ["example", "com"],
253 |         .init("dub"): "me/too",
    |                       |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                       `- note: add 'if #available' version check
254 |         .init("hello"): "Hello World!",
255 |         .init("half"): "50%",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:254:25: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
252 |         .init("dom"): ["example", "com"],
253 |         .init("dub"): "me/too",
254 |         .init("hello"): "Hello World!",
    |                         |- error: 'Value' is only available in macOS 26.0 or newer
    |                         `- note: add 'if #available' version check
255 |         .init("half"): "50%",
256 |         .init("var"): "value",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:254:25: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
252 |         .init("dom"): ["example", "com"],
253 |         .init("dub"): "me/too",
254 |         .init("hello"): "Hello World!",
    |                         |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                         `- note: add 'if #available' version check
255 |         .init("half"): "50%",
256 |         .init("var"): "value",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:255:24: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
253 |         .init("dub"): "me/too",
254 |         .init("hello"): "Hello World!",
255 |         .init("half"): "50%",
    |                        |- error: 'Value' is only available in macOS 26.0 or newer
    |                        `- note: add 'if #available' version check
256 |         .init("var"): "value",
257 |         .init("who"): "fred",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:255:24: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
253 |         .init("dub"): "me/too",
254 |         .init("hello"): "Hello World!",
255 |         .init("half"): "50%",
    |                        |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                        `- note: add 'if #available' version check
256 |         .init("var"): "value",
257 |         .init("who"): "fred",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:256:23: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
254 |         .init("hello"): "Hello World!",
255 |         .init("half"): "50%",
256 |         .init("var"): "value",
    |                       |- error: 'Value' is only available in macOS 26.0 or newer
    |                       `- note: add 'if #available' version check
257 |         .init("who"): "fred",
258 |         .init("base"): "http://example.com/home/",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:256:23: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
254 |         .init("hello"): "Hello World!",
255 |         .init("half"): "50%",
256 |         .init("var"): "value",
    |                       |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                       `- note: add 'if #available' version check
257 |         .init("who"): "fred",
258 |         .init("base"): "http://example.com/home/",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:257:23: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
255 |         .init("half"): "50%",
256 |         .init("var"): "value",
257 |         .init("who"): "fred",
    |                       |- error: 'Value' is only available in macOS 26.0 or newer
    |                       `- note: add 'if #available' version check
258 |         .init("base"): "http://example.com/home/",
259 |         .init("path"): "/foo/bar",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:257:23: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
255 |         .init("half"): "50%",
256 |         .init("var"): "value",
257 |         .init("who"): "fred",
    |                       |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                       `- note: add 'if #available' version check
258 |         .init("base"): "http://example.com/home/",
259 |         .init("path"): "/foo/bar",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:258:24: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
256 |         .init("var"): "value",
257 |         .init("who"): "fred",
258 |         .init("base"): "http://example.com/home/",
    |                        |- error: 'Value' is only available in macOS 26.0 or newer
    |                        `- note: add 'if #available' version check
259 |         .init("path"): "/foo/bar",
260 |         .init("list"): ["red", "green", "blue"],

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:258:24: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
256 |         .init("var"): "value",
257 |         .init("who"): "fred",
258 |         .init("base"): "http://example.com/home/",
    |                        |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                        `- note: add 'if #available' version check
259 |         .init("path"): "/foo/bar",
260 |         .init("list"): ["red", "green", "blue"],

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:259:24: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
257 |         .init("who"): "fred",
258 |         .init("base"): "http://example.com/home/",
259 |         .init("path"): "/foo/bar",
    |                        |- error: 'Value' is only available in macOS 26.0 or newer
    |                        `- note: add 'if #available' version check
260 |         .init("list"): ["red", "green", "blue"],
261 |         .init("keys"): [

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:259:24: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
257 |         .init("who"): "fred",
258 |         .init("base"): "http://example.com/home/",
259 |         .init("path"): "/foo/bar",
    |                        |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                        `- note: add 'if #available' version check
260 |         .init("list"): ["red", "green", "blue"],
261 |         .init("keys"): [

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:260:24: error: 'init(arrayLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
258 |         .init("base"): "http://example.com/home/",
259 |         .init("path"): "/foo/bar",
260 |         .init("list"): ["red", "green", "blue"],
    |                        |- error: 'init(arrayLiteral:)' is only available in macOS 26.0 or newer
    |                        `- note: add 'if #available' version check
261 |         .init("keys"): [
262 |             "semi": ";",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:261:24: error: 'init(dictionaryLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
259 |         .init("path"): "/foo/bar",
260 |         .init("list"): ["red", "green", "blue"],
261 |         .init("keys"): [
    |                        |- error: 'init(dictionaryLiteral:)' is only available in macOS 26.0 or newer
    |                        `- note: add 'if #available' version check
262 |             "semi": ";",
263 |             "dot": ".",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:266:21: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
264 |             "comma": ",",
265 |         ],
266 |         .init("v"): "6",
    |                     |- error: 'Value' is only available in macOS 26.0 or newer
    |                     `- note: add 'if #available' version check
267 |         .init("x"): "1024",
268 |         .init("y"): "768",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:266:21: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
264 |             "comma": ",",
265 |         ],
266 |         .init("v"): "6",
    |                     |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                     `- note: add 'if #available' version check
267 |         .init("x"): "1024",
268 |         .init("y"): "768",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:267:21: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
265 |         ],
266 |         .init("v"): "6",
267 |         .init("x"): "1024",
    |                     |- error: 'Value' is only available in macOS 26.0 or newer
    |                     `- note: add 'if #available' version check
268 |         .init("y"): "768",
269 |         .init("empty"): "",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:267:21: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
265 |         ],
266 |         .init("v"): "6",
267 |         .init("x"): "1024",
    |                     |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                     `- note: add 'if #available' version check
268 |         .init("y"): "768",
269 |         .init("empty"): "",

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:268:21: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
266 |         .init("v"): "6",
267 |         .init("x"): "1024",
268 |         .init("y"): "768",
    |                     |- error: 'Value' is only available in macOS 26.0 or newer
    |                     `- note: add 'if #available' version check
269 |         .init("empty"): "",
270 |         .init("empty_keys"): [:],

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:268:21: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
266 |         .init("v"): "6",
267 |         .init("x"): "1024",
268 |         .init("y"): "768",
    |                     |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                     `- note: add 'if #available' version check
269 |         .init("empty"): "",
270 |         .init("empty_keys"): [:],

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:269:25: error: 'Value' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
267 |         .init("x"): "1024",
268 |         .init("y"): "768",
269 |         .init("empty"): "",
    |                         |- error: 'Value' is only available in macOS 26.0 or newer
    |                         `- note: add 'if #available' version check
270 |         .init("empty_keys"): [:],
271 |     ]

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:269:25: error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
267 |         .init("x"): "1024",
268 |         .init("y"): "768",
269 |         .init("empty"): "",
    |                         |- error: 'init(stringLiteral:)' is only available in macOS 26.0 or newer
    |                         `- note: add 'if #available' version check
270 |         .init("empty_keys"): [:],
271 |     ]

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:270:30: error: 'init(dictionaryLiteral:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
268 |         .init("y"): "768",
269 |         .init("empty"): "",
270 |         .init("empty_keys"): [:],
    |                              |- error: 'init(dictionaryLiteral:)' is only available in macOS 26.0 or newer
    |                              `- note: add 'if #available' version check
271 |     ]
272 | 

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/URL/BenchmarkURL.swift:276:27: error: 'init(template:variables:)' is only available in macOS 26.0 or newer
 20 | #endif
 21 | 
 22 | let benchmarks = {
    |     `- note: add '@available' attribute to enclosing let
 23 | 
 24 |     Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
    :
274 |         for _ in benchmark.scaledIterations {
275 |             for t in templates {
276 |                 blackHole(URL(template: t, variables: variables))
    |                           |- error: 'init(template:variables:)' is only available in macOS 26.0 or newer
    |                           `- note: add 'if #available' version check
277 |             }
278 |         }

Benchmark failed to run due to build error.
error: buildFailed

[parkera]

I think the benchmarks are supposed to be using the Foundation built from the package here, not the one with availability that comes from the system Foundation.

[parkera]

swift package --disable-sandbox -c release benchmark run                                                          
warning: 'benchmarks': swift-foundation benchmarks: Using system Foundation.framework

That seems wrong, and probably is the root of the issue. See this import at the top of the benchmark files:

#if os(macOS) && USE_PACKAGE
import FoundationEssentials
#else
import Foundation
#endif

[vanvoorden]

I think the benchmarks are supposed to be using the Foundation built from the package here, not the one with availability that comes from the system Foundation.

@parkera That's a good point… but if our errors was from benchmarks trying to use the system Foundation then I would also expect a build failure from URL.Template being missing entirely. Not a build failure reporting that URL.Template is present but unavailable. Nevermind… my command line tools were set to Xcode 26.

[vanvoorden]

FWIW I can "hack-fix" fix the build failure from changing this LOC:

swift-foundation/Benchmarks/Package.swift

Line 91 in eb084da

platforms: [.macOS("15"), .iOS("18"), .tvOS("18"), .watchOS("11")], // Should match parent project

to .macOS("26").

And then running:

$ swift build --target URLBenchmarks

We obviously don't want to restrict all the benchmarks to 26. We should prefer some more targeted gating just around the code that benchmarks URL.Template.

[parkera]

cc @jmschonfeld

[vanvoorden]

This is also unbreaking my builds:

diff --git a/Benchmarks/Benchmarks/URL/BenchmarkURL.swift b/Benchmarks/Benchmarks/URL/BenchmarkURL.swift
index ae9912d..23bb847 100644
--- a/Benchmarks/Benchmarks/URL/BenchmarkURL.swift
+++ b/Benchmarks/Benchmarks/URL/BenchmarkURL.swift
@@ -232,6 +232,9 @@ let benchmarks = {
     }
 
     Benchmark("URL-Template-parsing") { benchmark in
+        guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, watchOS 26.0, *) else {
+            return
+        }
         for _ in benchmark.scaledIterations {
             blackHole(URL.Template("/api/{version}/accounts/{accountId}/transactions/{transactionId}{?expand*,fields*,embed*,format}")!)
             blackHole(URL.Template("/special/{+a}/details")!)
@@ -239,38 +242,40 @@ let benchmarks = {
         }
     }
 
-    let templates = [
-        URL.Template("/var/{var}/who/{who}/x/{x}{?keys*,count*,list*,y}")!,
-        URL.Template("/special/{+keys}/details")!,
-        URL.Template("x/y/{#path:6}/here")!,
-        URL.Template("a/b{/var,x}/here")!,
-        URL.Template("a{?var,y}")!,
-    ]
-
-    var variables: [URL.Template.VariableName: URL.Template.Value] = [
-        .init("count"): ["one", "two", "three"],
-        .init("dom"): ["example", "com"],
-        .init("dub"): "me/too",
-        .init("hello"): "Hello World!",
-        .init("half"): "50%",
-        .init("var"): "value",
-        .init("who"): "fred",
-        .init("base"): "http://example.com/home/",
-        .init("path"): "/foo/bar",
-        .init("list"): ["red", "green", "blue"],
-        .init("keys"): [
-            "semi": ";",
-            "dot": ".",
-            "comma": ",",
-        ],
-        .init("v"): "6",
-        .init("x"): "1024",
-        .init("y"): "768",
-        .init("empty"): "",
-        .init("empty_keys"): [:],
-    ]
-
     Benchmark("URL-Template-expansion") { benchmark in
+        guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, watchOS 26.0, *) else {
+            return
+        }
+        let templates = [
+            URL.Template("/var/{var}/who/{who}/x/{x}{?keys*,count*,list*,y}")!,
+            URL.Template("/special/{+keys}/details")!,
+            URL.Template("x/y/{#path:6}/here")!,
+            URL.Template("a/b{/var,x}/here")!,
+            URL.Template("a{?var,y}")!,
+        ]
+
+        let variables: [URL.Template.VariableName: URL.Template.Value] = [
+            .init("count"): ["one", "two", "three"],
+            .init("dom"): ["example", "com"],
+            .init("dub"): "me/too",
+            .init("hello"): "Hello World!",
+            .init("half"): "50%",
+            .init("var"): "value",
+            .init("who"): "fred",
+            .init("base"): "http://example.com/home/",
+            .init("path"): "/foo/bar",
+            .init("list"): ["red", "green", "blue"],
+            .init("keys"): [
+                "semi": ";",
+                "dot": ".",
+                "comma": ",",
+            ],
+            .init("v"): "6",
+            .init("x"): "1024",
+            .init("y"): "768",
+            .init("empty"): "",
+            .init("empty_keys"): [:],
+        ]
         for _ in benchmark.scaledIterations {
             for t in templates {
                 blackHole(URL(template: t, variables: variables))

[parkera]

Perhaps, but that shouldn't be required because URL.Template is marked as available for FoundationPreview 6.2. That means it can be used as part of the package on previous builds (of course, not interoperating with the framework version on 26 or later).

[vanvoorden]

That seems wrong, and probably is the root of the issue. See this import at the top of the benchmark files:

#if os(macOS) && USE_PACKAGE
import FoundationEssentials
#else
import Foundation
#endif

Ahh… ok… I think I see what's happening now. The benchmarks package should be running with the SWIFTCI_USE_LOCAL_DEPS flag from env:

https://github.com/swiftlang/swift-foundation/blob/swift-6.1.2-RELEASE/Benchmarks/Package.swift#L62

If I try to build benchmarks without SWIFTCI_USE_LOCAL_DEPS then it's not setting the USE_PACKAGE flag:

https://github.com/swiftlang/swift-foundation/blob/swift-6.1.2-RELEASE/Benchmarks/Package.swift#L70

Setting SWIFTCI_USE_LOCAL_DEPS unblocks me on building benchmarks locally. When I try to then run benchmarks locally I see what looks like a different error:

swift package --disable-sandbox -c release benchmark run --filter "URL-Template-expansion"
Building for production...
ld: warning: building for macOS-13.0, but linking with dylib '/opt/homebrew/opt/jemalloc/lib/libjemalloc.2.dylib' which was built for newer version 14.0
[7/7] Linking BenchmarkTool-tool
Build of product 'BenchmarkTool' complete! (16.49s)
Build complete!
Building BenchmarkTool in release mode...
Building benchmark targets in release mode for benchmark run...
Building URLBenchmarks
Building StringBenchmarks
[0/4] Compiling CSystem shims.c
[1/4] Write swift-version-40EA24106824B483.txt
[3/5] Compiling SystemPackage Errno.swift
[3/5] Write Objects.LinkFileList
[4/5] Linking BenchmarkBoilerplateGenerator-tool
Building for production...
[0/9] Compiling CSystem shims.c
[1/9] Write swift-version-40EA24106824B483.txt
[3/9] Compiling SystemPackage Errno.swift
[3/9] Write Objects.LinkFileList
[4/9] Linking BenchmarkBoilerplateGenerator-tool
[5/9] Generating plugin support files
[6/9] Write sources
[8/10] Compiling StringBenchmarks BenchmarkString.swift
/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/String/BenchmarkString.swift:70:13: error: cannot find 'autoreleasepool' in scope
 68 |     Benchmark("utf16-encode") { benchmark in
 69 |         for _ in benchmark.scaledIterations {
 70 |             autoreleasepool {
    |             `- error: cannot find 'autoreleasepool' in scope
 71 |                 blackHole(asciiSmallStr.data(using: .utf16BigEndian))
 72 |                 blackHole(nonAsciiSmallStr.data(using: .utf16BigEndian))

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/String/BenchmarkString.swift:91:13: error: cannot find 'autoreleasepool' in scope
 89 |     Benchmark("utf16-decode") { benchmark in
 90 |         for _ in benchmark.scaledIterations {
 91 |             autoreleasepool {
    |             `- error: cannot find 'autoreleasepool' in scope
 92 |                 blackHole(String(bytes: asciiSmallStrDataUTF16BE, encoding: .utf16BigEndian))
 93 |                 blackHole(String(bytes: nonAsciiSmallStrDataUTF16BE, encoding: .utf16BigEndian))

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/String/BenchmarkString.swift:115:13: error: cannot find 'autoreleasepool' in scope
113 |     Benchmark("utf32-encode") { benchmark in
114 |         for _ in benchmark.scaledIterations {
115 |             autoreleasepool {
    |             `- error: cannot find 'autoreleasepool' in scope
116 |                 blackHole(asciiSmallStr.data(using: .utf32BigEndian))
117 |                 blackHole(nonAsciiSmallStr.data(using: .utf32BigEndian))

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/String/BenchmarkString.swift:136:13: error: cannot find 'autoreleasepool' in scope
134 |     Benchmark("utf32-decode") { benchmark in
135 |         for _ in benchmark.scaledIterations {
136 |             autoreleasepool {
    |             `- error: cannot find 'autoreleasepool' in scope
137 |                 blackHole(String(bytes: asciiSmallStrDataUTF32BE, encoding: .utf32BigEndian))
138 |                 blackHole(String(bytes: nonAsciiSmallStrDataUTF32BE, encoding: .utf32BigEndian))

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/String/BenchmarkString.swift:160:44: error: cannot find 'NSTemporaryDirectory' in scope
158 | 
159 |     Benchmark("read-utf8") { benchmark in
160 |         let rootURL = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true).appendingPathComponent(UUID().uuidString, isDirectory: true)
    |                                            `- error: cannot find 'NSTemporaryDirectory' in scope
161 |         #if compiler(>=6)
162 |         let fileURL = rootURL.appending(path: "benchmark.txt", directoryHint: .notDirectory)

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/String/BenchmarkString.swift:174:13: error: cannot find 'autoreleasepool' in scope
172 | 
173 |         for _ in benchmark.scaledIterations {
174 |             autoreleasepool {
    |             `- error: cannot find 'autoreleasepool' in scope
175 |                 blackHole(try! String(contentsOf: fileURL, encoding: .utf8))
176 |             }

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/String/BenchmarkString.swift:181:44: error: cannot find 'NSTemporaryDirectory' in scope
179 | 
180 |     Benchmark("read-utf16") { benchmark in
181 |         let rootURL = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true).appendingPathComponent(UUID().uuidString, isDirectory: true)
    |                                            `- error: cannot find 'NSTemporaryDirectory' in scope
182 |         #if compiler(>=6)
183 |         let fileURL = rootURL.appending(path: "benchmark.txt", directoryHint: .notDirectory)

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/String/BenchmarkString.swift:195:13: error: cannot find 'autoreleasepool' in scope
193 | 
194 |         for _ in benchmark.scaledIterations {
195 |             autoreleasepool {
    |             `- error: cannot find 'autoreleasepool' in scope
196 |                 blackHole(try! String(contentsOf: fileURL, encoding: .utf16LittleEndian))
197 |             }

/Users/rick/Developer/swift-foundation-fork/Benchmarks/Benchmarks/String/BenchmarkString.swift:46:63: error: ambiguous use of 'utf16BigEndian'
 44 |     }
 45 |     
 46 |     let asciiSmallStrDataUTF16BE = asciiSmallStr.data(using: .utf16BigEndian)!
    |                                                               `- error: ambiguous use of 'utf16BigEndian'
 47 |     let asciiSmallStrDataUTF16LE = asciiSmallStr.data(using: .utf16LittleEndian)!
 48 |     let nonAsciiSmallStrDataUTF16BE = nonAsciiSmallStr.data(using: .utf16BigEndian)!

Foundation.String.Encoding.utf16BigEndian:3:19: note: found this candidate in module 'Foundation'
1 | struct String {
2 |   struct Encoding {
3 | public static let utf16BigEndian: String.Encoding  }
  |                   `- note: found this candidate in module 'Foundation'
4 | }
5 | 

/Users/rick/Developer/swift-foundation-fork/Sources/FoundationEssentials/String/String+Encoding.swift:46:27: note: found this candidate in module 'FoundationEssentials'
 44 |         public static let macOSRoman = Encoding(rawValue: 30)
 45 |         public static let utf16 = Encoding.unicode
 46 |         public static let utf16BigEndian = Encoding(rawValue: 0x90000100)
    |                           `- note: found this candidate in module 'FoundationEssentials'
 47 |         public static let utf16LittleEndian = Encoding(rawValue: 0x94000100)
 48 |         public static let utf32 = Encoding(rawValue: 0x8c000100)

Benchmark failed to run due to build error.
error: buildFailed

[itingliu]

Ahh… ok… I think I see what's happening now. The benchmarks package should be running with the SWIFTCI_USE_LOCAL_DEPS flag from env:

https://github.com/swiftlang/swift-foundation/blob/swift-6.1.2-RELEASE/Benchmarks/Package.swift#L62

If I try to build benchmarks without SWIFTCI_USE_LOCAL_DEPS then it's not setting the USE_PACKAGE flag:

https://github.com/swiftlang/swift-foundation/blob/swift-6.1.2-RELEASE/Benchmarks/Package.swift#L70

Setting SWIFTCI_USE_LOCAL_DEPS unblocks me on building benchmarks locally.

This is what I figured too. I couldn't figure out where this env was exported since it's not set for me either.

[itingliu]

159 |     Benchmark("read-utf8") { benchmark in
160 |         let rootURL = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true).appendingPathComponent(UUID().uuidString, isDirectory: true)
    |                                            `- error: cannot find 'NSTemporaryDirectory' in scope

We can fix this by replacing NSTemporaryDirectory with URL.temporaryDirectory

But this below means we're still definitely using the toolchain Foundation somewhere:

Foundation.String.Encoding.utf16BigEndian:3:19: note: found this candidate in module 'Foundation'
1 | struct String {
2 |   struct Encoding {
3 | public static let utf16BigEndian: String.Encoding  }
  |                   `- note: found this candidate in module 'Foundation'
4 | }
5 | 

[vanvoorden]

But this below means we're still definitely using the toolchain Foundation somewhere:

Foundation.String.Encoding.utf16BigEndian:3:19: note: found this candidate in module 'Foundation'
1 | struct String {
2 |   struct Encoding {
3 | public static let utf16BigEndian: String.Encoding  }
  |                   `- note: found this candidate in module 'Foundation'
4 | }
5 | 

What about this:

https://github.com/swiftlang/swift-foundation/blob/swift-6.1.2-RELEASE/Benchmarks/Benchmarks/JSON/JSONBenchmark.swift#L46-L47

And ordo one itself also uses Foundation in Benchmark if it's possible that is having weird side effects.

[itingliu]

But this below means we're still definitely using the toolchain Foundation somewhere:

Foundation.String.Encoding.utf16BigEndian:3:19: note: found this candidate in module 'Foundation'
1 | struct String {
2 |   struct Encoding {
3 | public static let utf16BigEndian: String.Encoding  }
  |                   `- note: found this candidate in module 'Foundation'
4 | }
5 | 

What about this:

https://github.com/swiftlang/swift-foundation/blob/swift-6.1.2-RELEASE/Benchmarks/Benchmarks/JSON/JSONBenchmark.swift#L46-L47

Good find! I don't think we can remove that from JSON benchmark without disabling that benchmark test, so not sure what we can do there besides working around it

But even if I remove that import and all places that use Bundle(and resolve all other minor errors such as missing autoreleasepool)

I'm still seeing this thanks to .unsafeFlags(["-Rmodule-loading"]) (thanks @jmschonfeld )

/swift-foundation/Benchmarks/Benchmarks/String/BenchmarkString.swift:13:8: remark: 'Benchmark' has a required transitive dependency on 'OSLog'
 11 | //===----------------------------------------------------------------------===//
 12 | 
 13 | import Benchmark
    |        `- remark: 'Benchmark' has a required transitive dependency on 'OSLog'
 14 | import func Benchmark.blackHole
 15 | 

<unknown>:0: remark: 'OSLog' has a required transitive dependency on 'Foundation'

I wonder if they switch to internal import would help.

And this

<unknown>:0: remark: 'ArgumentParser' has a required transitive dependency on 'Foundation'

Not sure about this one though.

[jmschonfeld]

I suspect enabling the MemberImportVisibility flag in the benchmark targets would help with that (the String.Encoding extension is an example of where members previously leaked through private imports but with the flag enabled they should not) - I suspect we likely forgot to add them to the benchmark targets when we added them to the main project.

For some added context here: SWIFTCI_USE_LOCAL_DEPS is set by the swift repo CI when performing a full toolchain test. In that scenario, it needs to use the locally checked out copies rather than fetching from git, and the package manifest assumes that the repos are all in the same parent directory. All of our dependencies (swift-foundation, swift-corelibs-foundation, swift-foundation-icu, etc.) are all set up to respect this flag. When we added the benchmark target, we also expanded it to allow specifying the directory that contains all of these repos, so SWIFTCI_USE_LOCAL_DEPS=1 indicates all checkouts are in the .. directory and SWIFTCI_USE_LOCAL_DEPS=/foo/bar indicates all checkouts are in the /foo/bar directory. That's opposed to the USE_PACKAGE=1 configuration where it uses the package but checks out main from GitHub to compare to a known ToT commit. I think we could probably use some documentation to help explain this (and maybe some tweaks to the configuration. I wonder if we could switch to using package traits if there's better support for listing available configurations with descriptions with those)

[vanvoorden]

For some added context here: SWIFTCI_USE_LOCAL_DEPS is set by the swift repo CI when performing a full toolchain test. In that scenario, it needs to use the locally checked out copies rather than fetching from git, and the package manifest assumes that the repos are all in the same parent directory. All of our dependencies (swift-foundation, swift-corelibs-foundation, swift-foundation-icu, etc.) are all set up to respect this flag.

Hmm… could you show me the job that runs Benchmarks on Foundation? Do we know how could CI not be running into either of the issues here that blocked me from running locally?