Test sometimes fails on GH-Action Java 21

[OyvindLGjesdal]

Version

main

What happened?

When I manually start github actions on linux it sometimes gives the following error:

[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.070 s -- in org.apache.jena.fuseki.mod.TS_FusekiMods
[INFO]
[INFO] Results:
[INFO]
Error: Failures:
Error: TestFusekiStdReadOnlySetup.stdSetup_endpoint_bad_2:158 POST http://localhost:45875/ds/query :: ==> expected: <415> but was: <-1>
[INFO]
Error: Tests run: 580, Failures: 1, Errors: 0, Skipped: 0

Other times it passes.

Haven't seen it apply to the java 17 job.

To me it looks unrelated to the commits where it happens, e.g OyvindLGjesdal@8cf9c8b

Relevant output and stacktrace

Error:  Tests run: 19, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.088 s <<< FAILURE! -- in org.apache.jena.fuseki.main.TestFusekiStdReadOnlySetup
Error:  org.apache.jena.fuseki.main.TestFusekiStdReadOnlySetup.stdSetup_endpoint_bad_2 -- Time elapsed: 0.009 s <<< FAILURE!
org.opentest4j.AssertionFailedError: POST http://localhost:45875/ds/query :: ==> expected: <415> but was: <-1>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
	at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:563)
	at org.apache.jena.fuseki.test.HttpTest.execWithHttpException(HttpTest.java:78)
	at org.apache.jena.fuseki.test.HttpTest.expect415(HttpTest.java:66)
	at org.apache.jena.fuseki.main.TestFusekiStdReadOnlySetup.stdSetup_endpoint_bad_2(TestFusekiStdReadOnlySetup.java:158)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

[INFO] Running org.apache.jena.fuseki.main.TestConfigFile
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.593 s -- in org.apache.jena.fuseki.main.TestConfigFile
[INFO] Running org.apache.jena.fuseki.main.TestCrossOriginFilter
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.030 s -- in org.apache.jena.fuseki.main.TestCrossOriginFilter
[INFO] Running org.apache.jena.fuseki.main.TestFusekiServerBuild
[INFO] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.354 s -- in org.apache.jena.fuseki.main.TestFusekiServerBuild
[INFO] Running org.apache.jena.fuseki.main.TestFusekiDatasetSharing
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.080 s -- in org.apache.jena.fuseki.main.TestFusekiDatasetSharing
[INFO] Running org.apache.jena.fuseki.main.TestFileUpload
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 s -- in org.apache.jena.fuseki.main.TestFileUpload
[INFO] Running org.apache.jena.fuseki.main.TestAuthQuery_JDK
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 s -- in org.apache.jena.fuseki.main.TestAuthQuery_JDK
[INFO] Running org.apache.jena.fuseki.main.TestAuthUpdate_JDK
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 s -- in org.apache.jena.fuseki.main.TestAuthUpdate_JDK
[INFO] Running org.apache.jena.fuseki.main.TestHttpOperations
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.150 s -- in org.apache.jena.fuseki.main.TestHttpOperations
[INFO] Running org.apache.jena.fuseki.main.TestHttpOptions
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.042 s -- in org.apache.jena.fuseki.main.TestHttpOptions
[INFO] Running org.apache.jena.fuseki.main.TestQuery
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.315 s -- in org.apache.jena.fuseki.main.TestQuery
[INFO] Running org.apache.jena.fuseki.main.TestSPARQLProtocol
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 s -- in org.apache.jena.fuseki.main.TestSPARQLProtocol
[INFO] Running org.apache.jena.fuseki.main.TestPatchFuseki
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.052 s -- in org.apache.jena.fuseki.main.TestPatchFuseki
[INFO] Running org.apache.jena.fuseki.main.TestFusekiCustomScriptFunc
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.865 s -- in org.apache.jena.fuseki.main.TestFusekiCustomScriptFunc
[INFO] Running org.apache.jena.fuseki.main.TestMetrics
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.013 s -- in org.apache.jena.fuseki.main.TestMetrics
[INFO] Running org.apache.jena.fuseki.main.TestFusekiShaclValidation
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.119 s -- in org.apache.jena.fuseki.main.TestFusekiShaclValidation
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.667 s -- in org.apache.jena.fuseki.main.TS_FusekiMain
[INFO] Running org.apache.jena.fuseki.mod.TS_FusekiMods
[INFO] Running org.apache.jena.fuseki.mod.admin.TestAdmin
[INFO] Tests run: 40, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.653 s -- in org.apache.jena.fuseki.mod.admin.TestAdmin
[INFO] Running org.apache.jena.fuseki.mod.admin.TestFusekiReload
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.050 s -- in org.apache.jena.fuseki.mod.admin.TestFusekiReload
[INFO] Running org.apache.jena.fuseki.mod.admin.TestTemplateAddDataset
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.030 s -- in org.apache.jena.fuseki.mod.admin.TestTemplateAddDataset
[INFO] Running org.apache.jena.fuseki.mod.metrics.TestModPrometheus
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 s -- in org.apache.jena.fuseki.mod.metrics.TestModPrometheus
[INFO] Running org.apache.jena.fuseki.mod.shiro.TestModShiro
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.185 s -- in org.apache.jena.fuseki.mod.shiro.TestModShiro
[INFO] Running org.apache.jena.fuseki.mod.TestFusekiServer
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.126 s -- in org.apache.jena.fuseki.mod.TestFusekiServer
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.070 s -- in org.apache.jena.fuseki.mod.TS_FusekiMods
[INFO] 
[INFO] Results:
[INFO] 
Error:  Failures: 
[INFO] Apache Jena - Extras ............................... SKIPPED
[INFO] Apache Jena - Extras - Query Builder ............... SKIPPED
[INFO] Apache Jena - CommonsRDF for Jena .................. SKIPPED
[INFO] Apache Jena - Extras - Service Enhancer ............ SKIPPED
[INFO] Apache Jena - Code Examples ........................ SKIPPED
[INFO] Apache Jena - BOM .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  06:21 min
[INFO] Finished at: 2025-02-12T18:32:16Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.2:test (default-test) on project jena-fuseki-main: There are test failures.
Error:  
Error:  See /home/runner/work/jena/jena/jena-fuseki2/jena-fuseki-main/target/surefire-reports for the individual test results.
Error:  See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
Error:  -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Error:  
Error:  After correcting the problems, you can resume the build with the command
Error:    mvn <args> -rf :jena-fuseki-main
Error: Process completed with exit code 1.

Are you interested in making a pull request?

None

[afs]

expected: <415> but was: <-1>

-1 means the server has gone away - no HTTP status code.

#3006 may be related. i.e., it passed both build actions which currently seems to be only 50%. Rerunning a workflow step is usually successful - which does make much sense to me.

I don't know whats happening - my (very wild) guess is that the server isn't GC-reachable and gets killed by a being GC-collected.
The action isn't at some natural timeout point like "5 minutes".
The guess is as much because I can't see anything else.

TestFusekiStdReadOnlySetup.stdSetup_endpoint_bad_2 is one place it occurs - with a different job, it is TestFusekiShaclValidation.shacl_no_data_graph (in the past, it has moved around).

[afs]

The same thing happen with java17 - less frequent.
https://github.com/afs/jena/actions/runs/13312699736 is for the fixes applied to all test suites I could find with the bad pattern.

It is only two classes where it happens.

I would be nervous of making it go away by creating a server per test because there is a root issue somewhere.

[afs]

Experiment:

I added a static lock in TestFusekiStdReadOnlySetup and TestFusekiShaclValidation so tests and AfterClass are synchronized, then ran the Linux GH action 15 times.

There were 3 Java21 and 1 Java17 failures.

21: TestFusekiShaclValidation.shacl_no_data_graph
17: TestFusekiStdReadOnlySetup.stdSetup_endpoint_7
21: TestFusekiStdReadOnlySetup.stdSetup_endpoint_bad_2
21: TestRDFConnectionFuseki.named_graph_load_remote_4