compiler.batch.FileSystem.getClasspath() may return ClasspathJar for non existing classpath entry

[iloveeclipse]

See eclipse-pde/eclipse.pde#236 (comment)

java.nio.file.NoSuchFileException: eclipse.pde/ui/org.eclipse.pde.core/target/ant_tasks/pde-ant.jar-classes
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
	at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
	at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
	at java.base/java.nio.file.Files.readAttributes(Files.java:1851)
	at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1264)
	at java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:709)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:243)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:172)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:186)
	at org.eclipse.jdt.internal.compiler.batch.ClasspathJar.initialize(ClasspathJar.java:201)
	at org.eclipse.jdt.internal.compiler.batch.ClasspathMultiReleaseJar.initialize(ClasspathMultiReleaseJar.java:38)
	at org.eclipse.jdt.internal.compiler.batch.FileSystem.<init>(FileSystem.java:235)
	at org.eclipse.jdt.internal.compiler.batch.Main.getLibraryAccess(Main.java:3495)
	at org.eclipse.jdt.internal.compiler.batch.Main.performCompilation(Main.java:4741)
	at org.eclipse.jdt.internal.compiler.batch.Main.compile(Main.java:1802)

The code in FileSystem.getClasspath() may initialize a ClasspathJar for a classpath entry that is not existing, because after "false" returned by File.isDirectory() it doesn't check if the file exists at all - which is the valid case for File.isDirectory() API.

I have a patch, I fear however, it will break build because we most likely have the problem for a long time and most likely classpaths constructed for ECJ are not "clean", because that was silently ignored till #239.

[laeubi]

The code in FileSystem.getClasspath() may initialize a ClasspathJar for a classpath entry that is not existing,

Just wanted to note, the folder does exits and it contains the classes, just if this makes a difference at all...

[iloveeclipse]

The code in FileSystem.getClasspath() may initialize a ClasspathJar for a classpath entry that is not existing,

Just wanted to note, the folder does exits and it contains the classes, just if this makes a difference at all...

No, it doesn't at the moment ECJ is called.
Is this supposed to be created afterwards and therefore added to the classpath?
Where is this folder passed as classpath to ECJ?

[laeubi]

No, it doesn't at the moment ECJ is called.

That's possible I think ...

Is this supposed to be created afterwards and therefore added to the classpath?

The process in Tycho is that each of these "subjars" are compiled to a seperate folder (pde-ant.jar-classes -> will be pde-ant.jar after the package phase), but it is possible that Tycho just passes everything to EJC and even the folder might not be created at that point in time.

Where is this folder passed as classpath to ECJ?

If you like you can run PDE maven build with -X flag, then it shows all arguments passed to ECJ in details, the erro occurs at the very beginning of the build.

[iloveeclipse]

If you like you can run PDE maven build with -X flag

Is there a way to run this build with patched ecj? if yes, could you provide full maven command line / pom / whatever needed? I have zero maven knowledge.

[laeubi]

But I think the most interesting question would be:

Is it valid to pass a non existing file/folder as the classpath? If not, I think just enable this and let the build fail so we are forced to fix that...

Is there a way to run this build with patched ecj? if yes, could you provide full maven command line / pom / whatever needed? I have zero maven knowledge.

I have tried to write down a small walk-trough: https://github.com/eclipse/tycho/blob/master/CONTRIBUTING.md#running-with-a-locally-build-version-of-jdt-compiler let me know if anything is unclear, it is a bit cumbersome because the Maven-GAV of JDT do not match the Maven-GAV that is deployed to central and the eclipse-batch-compiler is currently not deployed at automatically.

[iloveeclipse]

Is it valid to pass a non existing file/folder as the classpath

I see the code in Main that seem to produce an error later if constructed classpath is empty, but I honestly don't know what is the rule here. Typically we check what javac does, I haven't tried yet.

I don't plan to merge this soon anyway.

[laeubi]

Maybe it could be turned in a warning if any thing point to an non existing file? This could be annoying for a while but otherwise its really hard to diagnose such things, the classpath is very very huge, enabling -X produces a bloat of other things, and I assume that's the reason why no one has ever noticed that, but swallowing exception is also not nice...

[laeubi]

Or maybe one could even add a compiler flag --strict-classpath or something?

[laeubi]

Another one for a --strict-classpath option: Tycho currently passes the same path multiple times... I should have not taken a deeper look here :-\

[iloveeclipse]

For the record: here is the bug that added "manifest based" classpath entries support to ecj: https://bugs.eclipse.org/bugs/show_bug.cgi?id=97332
Looking on related tests / comments, it seems that non-existing jar files on classpath are silently tolerated by both ecj/javac.
However, ecj complains about non-existing directories while javac doesn't.

ecj from 4.21

rm Main.class; /usr/lib/jvm/java-17/bin/java -jar /opt/hp93000rt/el7/x86_64/ecj/3.27.0/ecj.jar -cp /missing/dir:/hello/test.jar Main.java; ll Main.class
incorrect classpath: /missing/dir
-rw-rw-r-- 1 aloskuto users 472 Aug  1 15:34 Main.class

Same with javac:

rm Main.class; /usr/lib/jvm/java-17/bin/javac -cp /missing/dir:/hello/test.jar Main.java; ll Main.class
-rw-rw-r-- 1 aloskuto users 472 Aug  1 15:35 Main.class

So ecj is inconsistent with javac and also inconsistent to itself.

rm Main.class; /usr/lib/jvm/java-17/bin/java -jar /opt/hp93000rt/el7/x86_64/ecj/3.27.0/ecj.jar -cp /missing/dir:/missing/dir2:/hello/test.jar Main.java; ll Main.class
incorrect classpath: /missing/dir
incorrect classpath: /missing/dir2
-rw-rw-r-- 1 aloskuto users 472 Aug  1 15:54 Main.class

With that above I don't see why don't we add incorrect classpath: /hello/test.jar message too.

I will work towards this direction.

[laeubi]

Just in case I created a PR for the "silently ignore this" case here: #278

[iloveeclipse]

Just in case I created a PR for the "silently ignore this" case here: #278

That has same issues with classpath tests like this PR. I see also there two (not necessarily connected) changes in one commit, I'm thinking now on splitting it and fix one by one. Give me some time to find some sensible solution.