Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GR-63337] [Native Image] GraalVM 24 deadlocks building with two jar files in the class path when only one CPU is available #10891

Open
1 task done
puni073 opened this issue Mar 20, 2025 · 4 comments
Open
1 task done

[GR-63337] [Native Image] GraalVM 24 deadlocks building with two jar files in the class path when only one CPU is available #10891

puni073 opened this issue Mar 20, 2025 · 4 comments
Assignees
@wirthi @olpaw
Labels
bug native-image

Comments

@puni073
Copy link

puni073 commented Mar 20, 2025

Describe the Issue

GraalVM 24 taskset -c 0 native-image -cp Main.jar:HelloWorld.jar Main -o MainApp deadlocks.
GraalVM 23.0.2 works.
taskset -c 0,1 or -cp Main.jar works.

Using the latest version of GraalVM can resolve many issues.

GraalVM Version

openjdk version "24" 2025-03-18
OpenJDK Runtime Environment GraalVM CE 24+36.1 (build 24+36-jvmci-b01)
OpenJDK 64-Bit Server VM GraalVM CE 24+36.1 (build 24+36-jvmci-b01, mixed mode, sharing)

Operating System and Version

DISTRIB_ID=Ubuntu DISTRIB_RELEASE=24.04 DISTRIB_CODENAME=noble DISTRIB_DESCRIPTION="Ubuntu 24.04.2 LTS"

Build Command

taskset -c 0 native-image -cp Main.jar:HelloWorld.jar Main -o MainApp

Expected Behavior

Building starts.

Actual Behavior

Building never started and was aborted by the watchdog.

Steps to Reproduce

cat Main.java

public class Main {
    public static void main(String[] args) {
        HelloWorld.sayHello();
    }
}

cat HelloWorld.java

public class HelloWorld {
    public static void sayHello() {
        System.out.println("Hello, World!");
    }
}

javac Main.java
javac HelloWorld.java
jar cf Main.jar Main.class
jar cf HelloWorld.jar HelloWorld.class
taskset -c 0 native-image -cp Main.jar:HelloWorld.jar Main -o MainApp

Additional Context

No response

Build Log Output and Error Messages

Loading classes is taking a long time. This can be caused by class- or module-path entries that point to large directory structures.
Total processed entries: 12324, current entry: jar:file:///judge/hello/HelloWorld.jar!/META-INF
Total processed entries: 12324, current entry: jar:file:///judge/hello/HelloWorld.jar!/META-INF
Total processed entries: 12324, current entry: jar:file:///judge/hello/HelloWorld.jar!/META-INF
Total processed entries: 12324, current entry: jar:file:///judge/hello/HelloWorld.jar!/META-INF
Total processed entries: 12324, current entry: jar:file:///judge/hello/HelloWorld.jar!/META-INF

=== Image generator watchdog detected no activity. This can be a sign of a deadlock during image building. Dumping all stack traces. Current time: Thu Mar 20 01:03:20 UTC 2025
"main" Id=3 in WAITING on lock=java.util.stream.ForEachOps$ForEachTask@6b4ad5f5
    at java.base@24/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@24/java.util.concurrent.locks.LockSupport.park(LockSupport.java:369)
    at java.base@24/java.util.concurrent.ForkJoinTask.awaitDone(ForkJoinTask.java:441)
    at java.base@24/java.util.concurrent.ForkJoinTask.awaitDone(ForkJoinTask.java:496)
    at java.base@24/java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:662)
    at java.base@24/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:677)
    at java.base@24/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:162)
    at java.base@24/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:176)
    at java.base@24/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:264)
    at java.base@24/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
    at java.base@24/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:810)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageClassLoaderSupport$LoadClassHandler.run(NativeImageClassLoaderSupport.java:744)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageClassLoaderSupport.loadAllClasses(NativeImageClassLoaderSupport.java:268)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ImageClassLoader.loadAllClasses(ImageClassLoader.java:88)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:400)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:732)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:151)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:99)

"Reference Handler" Id=7 in RUNNABLE
    at java.base@24/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
    at java.base@24/java.lang.ref.Reference.processPendingReferences(Reference.java:246)
    at java.base@24/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:208)

"Finalizer" Id=8 in WAITING on lock=java.lang.ref.ReferenceQueue$Lock@682e40b3
    at java.base@24/java.lang.Object.wait0(Native Method)
    at java.base@24/java.lang.Object.wait(Object.java:389)
    at java.base@24/java.lang.Object.wait(Object.java:351)
    at java.base@24/java.lang.ref.ReferenceQueue.remove0(ReferenceQueue.java:138)
    at java.base@24/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:229)
    at java.base@24/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:165)

"Signal Dispatcher" Id=9 in RUNNABLE

"Common-Cleaner" Id=14 in TIMED_WAITING on lock=java.lang.ref.ReferenceQueue$Lock@40ab9b6b
    at java.base@24/java.lang.Object.wait0(Native Method)
    at java.base@24/java.lang.Object.wait(Object.java:389)
    at java.base@24/java.lang.ref.ReferenceQueue.remove0(ReferenceQueue.java:124)
    at java.base@24/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:215)
    at java.base@24/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:140)
    at java.base@24/java.lang.Thread.runWith(Thread.java:1460)
    at java.base@24/java.lang.Thread.run(Thread.java:1447)
    at java.base@24/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:148)

"Notification Thread" Id=15 in RUNNABLE

"native-image pid watcher" Id=16 in TIMED_WAITING on lock=java.util.TaskQueue@188c1536
    at java.base@24/java.lang.Object.wait0(Native Method)
    at java.base@24/java.lang.Object.wait(Object.java:389)
    at java.base@24/java.util.TimerThread.mainLoop(Timer.java:569)
    at java.base@24/java.util.TimerThread.run(Timer.java:522)

"Thread-1" Id=17 in RUNNABLE
    at java.management@24/sun.management.ThreadImpl.dumpThreads0(Native Method)
    at java.management@24/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
    at java.management@24/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.DeadlockWatchdog.threadDump(DeadlockWatchdog.java:135)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.DeadlockWatchdog.reportFailureState(DeadlockWatchdog.java:103)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.DeadlockWatchdog.watchdogThread(DeadlockWatchdog.java:86)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.DeadlockWatchdog$$Lambda/0x000000003813cc40.run(Unknown Source)
    at java.base@24/java.lang.Thread.runWith(Thread.java:1460)
    at java.base@24/java.lang.Thread.run(Thread.java:1447)

"pool-2-thread-1" Id=18 in TIMED_WAITING on lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@25548eaf
    at java.base@24/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@24/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:271)
    at java.base@24/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1802)
    at java.base@24/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1166)
    at java.base@24/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:883)
    at java.base@24/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1021)
    at java.base@24/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1081)
    at java.base@24/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619)
    at java.base@24/java.lang.Thread.runWith(Thread.java:1460)
    at java.base@24/java.lang.Thread.run(Thread.java:1447)


=== Memory statistics (in MB):
=== Used heap size: 981
=== Free heap size: 908
=== Maximum heap size: 27127
=== Image generator watchdog is aborting image generation. To configure the watchdog, use the options -H:DeadlockWatchdogInterval=10 and -H:+DeadlockWatchdogExitOnTimeout
Error: Image build request for 'MainApp' (pid: 1234, path: /judge/hello) failed with exit status 30
@selhagani selhagani assigned selhagani and wirthi and unassigned wirthi and selhagani Mar 20, 2025
@olpaw
Copy link
Member

olpaw commented Mar 20, 2025 •
edited
Loading

@puni073 thanks you for your reproducer.
I can confirm that this is reproducible on master as well as on the GraalVM 24 release.

The moment there is more than one classpath-entry on -cp and only a single core is available we get a deadlock.

@olpaw
Copy link
Member

olpaw commented Mar 20, 2025

As expected, using

diff --git a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/NativeImageClassLoaderSupport.java b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/NativeImageClassLoaderSupport.java
--- a/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/NativeImageClassLoaderSupport.java	(revision Staged)
+++ b/substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/NativeImageClassLoaderSupport.java	(date 1742469424089)
@@ -817,7 +817,7 @@
                     initModule(moduleReference, true);
                 }
 
-                classpath().parallelStream().forEach(this::loadClassesFromPath);
+                classpath().stream().forEach(this::loadClassesFromPath);
             } finally {
                 scheduledExecutor.shutdown();
             }

makes the issue go away. But this is not a proper fix. I will look into the root cause and let you know.

@selhagani
Copy link
Member

Hi @puni073,

Thank you for reaching out to us and for sharing the reproducer. I tested things on my end and as you said on the new release we do notice the error you mentioned unlike testing on an older release (23.0.2)

@selhagani selhagani changed the title [Native Image] GraalVM 24 deadlocks building with two jar files in the class path when only one CPU is available [GR-63337] [Native Image] GraalVM 24 deadlocks building with two jar files in the class path when only one CPU is available Mar 20, 2025
@olpaw
Copy link
Member

olpaw commented Mar 20, 2025 •
edited
Loading

If I replace
-cp Main.jar:HelloWorld.jar
with
-cp Main_dir_with_jar_contents:HelloWorld_dir_with_jar_contents
the issue goes away.

I assume FileSystems.newFileSystem(...) has a problem if called from more than one thread with taskset -c 0.

@olpaw olpaw self-assigned this Mar 24, 2025
@olpaw olpaw moved this from To do to In progress in Native Image Mar 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wirthi wirthi

@olpaw olpaw

Labels
bug native-image
Projects
Native Image
Status: In progress
Milestone
No milestone
Development

No branches or pull requests
4 participants
@wirthi @olpaw @selhagani @puni073