Skip to content

[android][test] Fix or disable the remaining failing tests on the Android CI #81398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 17, 2025

Conversation

finagolfin
Copy link
Member

@finagolfin finagolfin commented May 9, 2025

Also, fix and enable IRGen/lto_autolink for all non-Wasm targets and IRGen/static_initializer for aarch64.

This should get the community Android CI green again, @marcprux and @weliveindetail, try it out locally if you can.

@finagolfin
Copy link
Member Author

@swift-ci smoke test

// UNSUPPORTED: OS=ios && CPU=arm64e
// UNSUPPORTED: OS=watchos && (CPU=arm64_32 || CPU=armv7k)
// UNSUPPORTED: OS=linux-gnu && CPU=aarch64
// XFAIL: OS=linux-android && CPU=aarch64
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mac and linux arm64 exclusions were added before @tshortli updated this test last year to build against %target-cpu, not just x86_64, f6d419c, so probably unneeded now?

The Android AArch64 failure is only because the community Android CI also builds the stdlib for linux x86_64, which confuses the compiler:

+ /usr/bin/python3.12 /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/swift/utils/PathSanitizingFileCheck --allow-unused-prefixes --sanitize BUILD_DIR=/home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64 --sanitize SOURCE_DIR=/home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/swift --ignore-runtime-warnings --use-filecheck /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/llvm-linux-x86_64/bin/FileCheck /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/swift/test/IRGen/lto_autolink.swift --check-prefix CHECK-ELF-MERGE
<unknown>:0: error: could not find module 'Swift' for target 'aarch64-unknown-linux-gnu'; found: x86_64-unknown-linux-gnu, at: /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/lib/swift/linux/Swift.swiftmodule
FileCheck error: '<stdin>' is empty.

If I disable building the host linux stdlib locally, the test passes for aarch64, as it does on the Android x86_64 CI, because then the host arch happens to match the target arch. The compiler shouldn't be erroring only because an unrelated stdlib is also installed, so disabling this for Android AArch64 alone until that bug is fixed.

@xymus, is the pasted error above the same reason you had to disable iOS and watchOS also?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not just add a requires for the stdlib on the architecture?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean: the problem is that this test creates Swift modules for linux/mac triples with the %target-cpu, for which no linux/mac stdlib is required. When running the tests for Android AArch64 on linux x86_64 with building that host linux x86_64 stdlib explicitly disabled, this test passed locally when I tried it.

The problem is the preset that the Android AArch64 CI uses also builds the linux x86_64 stdlib, which then causes the error pasted above, even though no linux stdlib seems to be required! This appears to be a clear bug in the compiler, so I'm disabling this for Android AArch64 until we get that compiler bug tracked down.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that this is a bug in the compiler, this sounds like a bug in the test then. If no stdlib is required, then you should do -no-stdimport when invoking the compiler.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I was unaware of that flag and so apparently are the rest of the Swift devs, as it is entirely unused in the compiler validation suite! 😃 It does fix the test in the scenario I mentioned, stopping the autolinking from looking in the stdlib resource directory.

I will experiment a bit and see if it fixes the problem on Apple platforms too. 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried enabling this test on Darwin arm64 platforms again and ran the Apple Silicon tester on it, but that CI appears to be broken when building some embedded Swift files right now, so I couldn't check if this test needs -nostdimport there too.

@tshortli, since you disabled this test for iOS arm64, can you check if it works now on whatever internal CI it broke on before, particularly if -nostdimport is added for mac as I just did for linux?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Evan tells me he's not seeing any issue with this test on Darwin, so let's go ahead and get it in.

@compnerd, I rebased and kicked off the tester one last time: if you'd approve, I'll go ahead and merge.

@@ -1,4 +1,4 @@
// RUN: %target-swift-frontend(mock-sdk: %clang-importer-sdk) %s -emit-sil -swift-version 5 -use-clang-function-types -experimental-print-full-convention -o - | %FileCheck %s --check-prefix=CHECK-%target-ptrsize
// RUN: %target-swift-frontend(mock-sdk: %clang-importer-sdk) %s -Xllvm -sil-print-types -emit-sil -swift-version 5 -use-clang-function-types -experimental-print-full-convention -o - | %FileCheck %s --check-prefix=CHECK-%target-ptrsize
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simply bringing over Erik's recent change

if os.path.exists(path):
# We should then have the architecture in the name.
for lib in os.listdir(path):
match = re.match(r'(?:lib)?clang_rt\.(\w+)-' + run_cpu, lib)
match = re.match(r'(?:lib)?clang_rt\.(\w+)-' + run_cpu + env, lib)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If running the Android x86_64 tests on linux x86_64, this was detecting the linux clang_rt libraries and wrongly enabling all these sanitizer tests for Android too, which then fail because the compiler correctly checks for this environment suffix. Checking for the suffix here too makes sure the Android-specific libraries are available, and disables the sanitizer tests on the Android CI when they aren't.

@finagolfin
Copy link
Member Author

@swift-ci smoke test macos

@finagolfin
Copy link
Member Author

@swift-ci smoke test macos

@finagolfin
Copy link
Member Author

@swift-ci smoke test

// UNSUPPORTED: OS=ios && CPU=arm64e
// UNSUPPORTED: OS=watchos && (CPU=arm64_32 || CPU=armv7k)
// UNSUPPORTED: OS=linux-gnu && CPU=aarch64
// XFAIL: OS=linux-android && CPU=aarch64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not just add a requires for the stdlib on the architecture?

@finagolfin
Copy link
Member Author

@swift-ci smoke test

@finagolfin
Copy link
Member Author

@swift-ci test macos

@finagolfin
Copy link
Member Author

@swift-ci please test Apple Silicon

@finagolfin
Copy link
Member Author

@swift-ci test apple silicon

@finagolfin
Copy link
Member Author

@swift-ci smoke test

…roid CI

Also, fix and enable IRGen/lto_autolink for all non-Wasm targets and
IRGen/static_initializer for aarch64.
@finagolfin
Copy link
Member Author

@swift-ci smoke test

@@ -12,7 +12,7 @@
// APPLE-NEXT: (End of search path lists.)

// Non-Apple platforms don't have any implicit framework search paths.
// RUN: %target-swift-frontend(mock-sdk: %clang-importer-sdk-nosource) -target x86_64-unknown-linux-android -parse %s -Rmodule-loading 2>&1 | %FileCheck -check-prefix=ANDROID %s
// RUN: %target-swift-frontend(mock-sdk: %clang-importer-sdk-nosource) -target x86_64-unknown-linux-android -parse -parse-stdlib %s -Rmodule-loading 2>&1 | %FileCheck -check-prefix=ANDROID %s
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed this test failing on the Android AArch64 CI alone in the last couple days with no output since it was added in #81269, so I ran the same command locally and got this error:

> ~/swift-DEVELOPMENT-SNAPSHOT-2025-04-12-a-fedora39/usr/bin/swift-frontend -target aarch64-unknown-linux-android -sdk /home/finagolfin/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot -resource-dir /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift -module-cache-path /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/swift-test-results/aarch64-unknown-linux-android/clang-module-cache -swift-version 4 -typo-correction-limit 10 -sdk /home/finagolfin/swift/test/Inputs/clang-importer-sdk -target x86_64-unknown-linux-android -parse foo.swift -Rmodule-loading
<unknown>:0: warning: libc not found for 'x86_64-unknown-linux-android'; C stdlib may be unavailable
Module import search paths:
Framework search paths:
Runtime library import search paths:
[0] /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift/android
[1] /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift/android/x86_64
[2] /home/finagolfin/swift/test/Inputs/clang-importer-sdk/usr/lib/swift/android
[3] /home/finagolfin/swift/test/Inputs/clang-importer-sdk/usr/lib/swift/android/x86_64
(End of search path lists.)
<unknown>:0: error: could not find module '_Concurrency' for target 'x86_64-unknown-linux-android'; found: aarch64-unknown-linux-android, at: /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift/android/_Concurrency.swiftmodule

Since this passes on the Android x86_64 CI, this is another variant of the stdlib search path issue I raised for lto_autolink last week, which I'm able to avoid locally by passing in this extra -parse-stdlib flag also, as done in that test already.

@ian-twilightcoder, let me know what you think of this minor tweak to your new test for the Android CI.

@finagolfin
Copy link
Member Author

Asked @compnerd to review but he's probably busy: since these are all minor tweaks that passed CI and I addressed all his review comments, going ahead and merging to get Android CI green. If any test breaks any internal or external CI later, we can always fix that test then.

@finagolfin finagolfin merged commit 35ee368 into swiftlang:main May 17, 2025
3 checks passed
@finagolfin finagolfin deleted the droid-test branch May 17, 2025 08:06
@aschwaighofer
Copy link
Contributor

Seems like this CI bot is failing due to this change:

https://ci.swift.org/job/oss-swift_tools-R_stdlib-RD_test-simulator/7781/consoleText

[2025-05-18T14:47:37.647Z] RUN: at line 1: /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/build/Ninja-Release/swift-macosx-x86_64/bin/swift-frontend -target x86_64-apple-macosx13.0  -module-cache-path /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/build/Ninja-Release/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx13.0/clang-module-cache -sdk '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk'  -swift-version 4  -define-availability 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -define-availability 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -define-availability 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -define-availability 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -define-availability 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -define-availability 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -define-availability 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -define-availability 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -define-availability 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -define-availability 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -define-availability 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -define-availability 'SwiftStdlib 5.10:macOS 14.4, iOS 17.4, watchOS 10.4, tvOS 17.4, visionOS 1.1' -define-availability 'SwiftStdlib 6.0:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -define-availability 'SwiftStdlib 6.1:macOS 15.4, iOS 18.4, watchOS 11.4, tvOS 18.4, visionOS 2.4' -define-availability 'SwiftStdlib 6.2:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -define-availability 'SwiftCompatibilitySpan 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2, visionOS 1.1' -define-availability 'SwiftCompatibilitySpan 6.2:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0'  -typo-correction-limit 10  -primary-file /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/swift/test/IRGen/framepointer.sil -emit-ir | /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/venv/bin/python3.9 /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/swift/utils/PathSanitizingFileCheck --allow-unused-prefixes --sanitize BUILD_DIR=/Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/build/Ninja-Release/swift-macosx-x86_64 --sanitize SOURCE_DIR=/Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/swift --ignore-runtime-warnings --use-filecheck /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/build/Ninja-Release/llvm-macosx-x86_64/bin/FileCheck   /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/swift/test/IRGen/framepointer.sil -check-prefix CHECK --check-prefix=CHECK-SYSV --check-prefix=CHECK-SYSV-macosx
[2025-05-18T14:47:37.647Z] + /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/build/Ninja-Release/swift-macosx-x86_64/bin/swift-frontend -target x86_64-apple-macosx13.0 -module-cache-path /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/build/Ninja-Release/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx13.0/clang-module-cache -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk -swift-version 4 -define-availability 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -define-availability 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -define-availability 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -define-availability 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -define-availability 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -define-availability 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -define-availability 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -define-availability 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -define-availability 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -define-availability 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -define-availability 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -define-availability 'SwiftStdlib 5.10:macOS 14.4, iOS 17.4, watchOS 10.4, tvOS 17.4, visionOS 1.1' -define-availability 'SwiftStdlib 6.0:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -define-availability 'SwiftStdlib 6.1:macOS 15.4, iOS 18.4, watchOS 11.4, tvOS 18.4, visionOS 2.4' -define-availability 'SwiftStdlib 6.2:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -define-availability 'SwiftCompatibilitySpan 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2, visionOS 1.1' -define-availability 'SwiftCompatibilitySpan 6.2:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -typo-correction-limit 10 -primary-file /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/swift/test/IRGen/framepointer.sil -emit-ir
[2025-05-18T14:47:37.647Z] + /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/venv/bin/python3.9 /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/swift/utils/PathSanitizingFileCheck --allow-unused-prefixes --sanitize BUILD_DIR=/Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/build/Ninja-Release/swift-macosx-x86_64 --sanitize SOURCE_DIR=/Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/swift --ignore-runtime-warnings --use-filecheck /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/build/Ninja-Release/llvm-macosx-x86_64/bin/FileCheck /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/swift/test/IRGen/framepointer.sil -check-prefix CHECK --check-prefix=CHECK-SYSV --check-prefix=CHECK-SYSV-macosx
[2025-05-18T14:47:37.647Z] swift runtime: unknown backtracing setting 'warnings'
[2025-05-18T14:47:37.647Z] swift runtime: unknown backtracing setting 'warnings'
[2025-05-18T14:47:37.647Z] swift runtime: unknown backtracing setting 'warnings'
[2025-05-18T14:47:37.647Z] /Users/ec2-user/jenkins/workspace/oss-swift_tools-R_stdlib-RD_test-simulator/swift/test/IRGen/framepointer.sil:36:16: error: CHECK-SYSV: expected string not found in input
[2025-05-18T14:47:37.647Z] // CHECK-SYSV: Function Attrs:

[2025-05-18T15:28:24.407Z] Failed Tests (1):
[2025-05-18T15:28:24.407Z]   Swift(macosx-x86_64) :: IRGen/framepointer.sil

@aschwaighofer
Copy link
Contributor

@finagolfin Can you take a look?

@finagolfin
Copy link
Member Author

@aschwaighofer, I only added that line so that it would match for all non-mac/linux/android SYSV OS's that might be added later, while the lines below match those three major OSs. It passed the CI for all those OSs, so is that test-simulator CI you linked doing something different that I need to account for?

@finagolfin
Copy link
Member Author

Here's the same test passing on a recent normal macOS x86_64 CI run after this pull:

[2025-05-20T12:16:02.823Z] PASS: Swift(macosx-x86_64) :: IRGen/framepointer.sil (3190 of 19062)

I'm not familiar with this other CI you're linking.

Ultimately, the line is not important: it was only chosen as something that should always be true, and is on all the other CI. The question is why it isn't on this one test-simulator CI, and if we can find some text that always matches there too.

@finagolfin
Copy link
Member Author

@aschwaighofer, I'm examining the IR and will submit a fix...

@finagolfin
Copy link
Member Author

Submitted in #81643

@finagolfin
Copy link
Member Author

@aschwaighofer, this test now passes on the test-simulator CI because of #81643, let me know if anything else crops up.

Comment on lines -46 to -48
// UNSUPPORTED: OS=macosx && CPU=arm64
// UNSUPPORTED: OS=ios && CPU=arm64e
// UNSUPPORTED: OS=watchos && (CPU=arm64_32 || CPU=armv7k)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing these seems to have caused failures in iOS and macOS bots, e.g., https://ci.swift.org/job/oss-swift_tools-RA_stdlib-DA_test-device-non_executable/8901. Is there a specific reason to believe that these are no longer necessary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed it on linux and Android, so I asked about fixing it on Darwin too. Allan, who originally disabled it, didn't respond, but Evan ran some other CI and said it seemed to work fine.

Looking at the error, it's exactly the same as the linux issue I fixed, so I'll submit the same fix for Darwin and if you can run this same failing CI on the Darwin fix, we can finally get it running on Darwin too.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Submitted the fix in #81695

hamishknight added a commit to hamishknight/swift that referenced this pull request May 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants