Description
reproduction steps
using Scala 2.12.12, and the extended version of DeterminismTest
I'm working on in retronym/scala#96, the pickle for scala.Symbol.class
differs depending on the order/subset of source files being compiled:
sbt:root> junit/test:runMain scala.tools.nsc.DeterminismTester /Users/jz/code/scala/src/library
--- /var/folders/22/g1sv634d11j1d_lqlnhz9p2r0000gn/T/reference1538611564431125389/scala/Symbol.class
+++ /var/folders/22/g1sv634d11j1d_lqlnhz9p2r0000gn/T/recompileOutput2404148374994820649/scala/Symbol.class
@@ -1,17 +1,17 @@
// class version 52.0 (52)
// access flags 0x31
public final class scala/Symbol implements scala/Serializable {
// compiled from: Symbol.scala
- @Lscala/reflect/ScalaSignature;(bytes="\u0006\u0001!4Aa\u0004\u0009\u0003'!A1\u0004\u0001BC\u0002\u0013\u0005A\u0004\u0003\u0005)\u0001\u0009\u0005\u0009\u0015!\u0003\u001e\u0011\u0015I\u0003\u0001\"\u0003+\u0011\u0015i\u0003\u0001\"\u0011/\u0011\u0015y\u0003\u0001\"\u00031\u0011\u0015\u0009\u0005\u0001\"\u0011C\u0011\u00151\u0005\u0001\"\u0011H\u000f\u0015i\u0005\u0003#\u0001O\r\u0015y\u0001\u0003#\u0001P\u0011\u0015I\u0013\u0002\"\u0001T\u0011\u0015!\u0016\u0002\"\u0011V\u0011\u00159\u0016\u0002\"\u0005Y\u0011\u0015Q\u0016\u0002\"\u0005\\\u0011\u001dy\u0013\"!A\u0005\n\u0005\u0014aaU=nE>d'\"A\u0009\u0002\u000bM\u001c\u0017\r\\1\u0004\u0001M\u0019\u0001\u0001\u0006\r\u0011\u0005U1R\"\u0001\u0009\n\u0005]\u0001\"AB!osJ+g\r\u0005\u0002\u00163%\u0011!\u0004\u0005\u0002\r'\u0016\u0014\u0018.\u00197ju\u0006\u0014G.Z\u0001\u0005]\u0006lW-F\u0001\u001e!\u0009qRE\u0004\u0002 GA\u0011\u0001\u0005E\u0007\u0002C)\u0011!EE\u0001\u0007yI|w\u000e\u001e \n\u0005\u0011\u0002\u0012A\u0002)sK\u0012,g-\u0003\u0002'O\u000911\u000b\u001e:j]\u001eT!\u0001\n\u0009\u0002\u000b9\u000cW.\u001a\u0011\u0002\rqJg.\u001b;?)\u0009YC\u0006\u0005\u0002\u0016\u0001!)1d\u0001a\u0001;\u0005AAo\\*ue&tw\rF\u0001\u001e\u0003-\u0011X-\u00193SKN|GN^3\u0015\u0003E\u0002\"!\u0006\u001a\n\u0005M\u0002\"aA!os\"\u001aQ!\u000e!\u0011\u0007U1\u0004(\u0003\u00028!\u00091A\u000f\u001b:poN\u0004\"!\u000f \u000e\u0003iR!a\u000f\u001f\u0002\u0005%|'\"A\u001f\u0002\u0009)\u000cg/Y\u0005\u0003�i\u0012Qc\u00142kK\u000e$8\u000b\u001e:fC6,\u0005pY3qi&|gnI\u00019\u0003!A\u0017m\u001d5D_\u0012,G#A\"\u0011\u0005U!\u0015BA#\u0011\u0005\rIe\u000e^\u0001\u0007KF,\u0018\r\\:\u0015\u0005![\u0005CA\u000bJ\u0013\u0009Q\u0005CA\u0004C_>dW-\u00198\u0009\u000b1;\u0001\u0019A\u0019\u0002\u000b=$\u0008.\u001a:\u0002\rMKXNY8m!\u0009)\u0012bE\u0002\n!b\u0001B!F)\u001eW%\u0011!\u000b\u0005\u0002\u0010+:L\u0017/^3oKN\u001c8)Y2iKR\u0009a*A\u0003baBd\u0017\u0010\u0006\u0002,-\")1d\u0003a\u0001;\u0005aa/\u00197vK\u001a\u0013x.\\&fsR\u00111&\u0017\u0005\u000671\u0001\r!H\u0001\rW\u0016LhI]8n-\u0006dW/\u001a\u000b\u00039~\u00032!F/\u001e\u0013\u0009q\u0006C\u0001\u0004PaRLwN\u001c\u0005\u0006A6\u0001\raK\u0001\u0004gflG#\u00012\u0011\u0005\r4W\"\u00013\u000b\u0005\u0015d\u0014\u0001\u00027b]\u001eL!a\u001a3\u0003\r=\u0013'.Z2u\u0001")
Diffing extra trace output added to the pickler:
analysis
We end up with two symbols representing type alias Predef.String = java.lang.String
. One of these symbols is created when the Predef.class
is loaded from the classpath when loading the scala.package
package object
java.lang.Throwable
at scala.reflect.internal.Types$AliasTypeRef.$init$(Types.scala:1957)
at scala.reflect.internal.Types$AliasNoArgsTypeRef.<init>(Types.scala:2411)
at scala.reflect.internal.Types$TypeRef$.apply(Types.scala:2423)
at scala.reflect.internal.pickling.UnPickler$Scan.readType(UnPickler.scala:415)
at scala.reflect.internal.pickling.UnPickler$Scan.$anonfun$readTypeRef$1(UnPickler.scala:656)
at scala.reflect.internal.pickling.UnPickler$Scan.at(UnPickler.scala:190)
at scala.reflect.internal.pickling.UnPickler$Scan.readTypeRef(UnPickler.scala:656)
at scala.reflect.internal.pickling.UnPickler$Scan.readNonEmptyTree(UnPickler.scala:605)
at scala.reflect.internal.pickling.UnPickler$Scan.readTree(UnPickler.scala:619)
at scala.reflect.internal.pickling.UnPickler$Scan.$anonfun$readAnnotArg$1(UnPickler.scala:467)
at scala.reflect.internal.pickling.UnPickler$Scan.at(UnPickler.scala:190)
at scala.reflect.internal.pickling.UnPickler$Scan.readAnnotArg(UnPickler.scala:467)
at scala.reflect.internal.pickling.UnPickler$Scan.readAnnotationInfo(UnPickler.scala:501)
at scala.reflect.internal.pickling.UnPickler$Scan.readSymbolAnnotation(UnPickler.scala:513)
at scala.reflect.internal.pickling.UnPickler$Scan.$anonfun$run$2(UnPickler.scala:108)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.reflect.internal.pickling.UnPickler$Scan.runAtIndex(UnPickler.scala:90)
at scala.reflect.internal.pickling.UnPickler$Scan.run(UnPickler.scala:108)
at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:47)
at scala.tools.nsc.symtab.classfile.ClassfileParser.unpickleOrParseInnerClasses(ClassfileParser.scala:1182)
at scala.tools.nsc.symtab.classfile.ClassfileParser.parseClass(ClassfileParser.scala:466)
at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parse$2(ClassfileParser.scala:160)
at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parse$2$adapted(ClassfileParser.scala:146)
at scala.reflect.internal.util.ReusableInstance.using(ReusableInstance.scala:30)
at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parse$1(ClassfileParser.scala:146)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.tools.nsc.symtab.classfile.ClassfileParser.pushBusy(ClassfileParser.scala:129)
at scala.tools.nsc.symtab.classfile.ClassfileParser.parse(ClassfileParser.scala:145)
at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader.doComplete(SymbolLoaders.scala:333)
at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:240)
at scala.reflect.internal.Symbols$Symbol.completeInfo(Symbols.scala:1542)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:355)
at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:410)
at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader.doComplete(SymbolLoaders.scala:303)
at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:240)
at scala.reflect.internal.Symbols$Symbol.completeInfo(Symbols.scala:1542)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
at scala.reflect.internal.Types$TypeRef.decls(Types.scala:2285)
at scala.tools.nsc.typechecker.Namers$Namer.enterPackage(Namers.scala:765)
at scala.tools.nsc.typechecker.Namers$Namer.dispatch$1(Namers.scala:288)
at scala.tools.nsc.typechecker.Namers$Namer.standardEnterSym(Namers.scala:301)
at scala.tools.nsc.typechecker.AnalyzerPlugins.pluginsEnterSym(AnalyzerPlugins.scala:479)
at scala.tools.nsc.typechecker.AnalyzerPlugins.pluginsEnterSym$(AnalyzerPlugins.scala:478)
at scala.tools.nsc.Global$$anon$5.pluginsEnterSym(Global.scala:483)
at scala.tools.nsc.typechecker.Namers$Namer.enterSym(Namers.scala:279)
at scala.tools.nsc.typechecker.Analyzer$namerFactory$$anon$1.apply(Analyzer.scala:50)
at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:454)
at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:402)
This is referred to when computing the root imports:
java.lang.Throwable
at scala.reflect.internal.Symbols$AliasTypeSymbol.<init>(Symbols.scala:3059)
at scala.reflect.internal.Symbols$Symbol.createAliasTypeSymbol(Symbols.scala:1331)
at scala.reflect.internal.Symbols$Symbol.newNonClassSymbol(Symbols.scala:1397)
at scala.reflect.internal.pickling.UnPickler$Scan.readSymbol(UnPickler.scala:346)
at scala.reflect.internal.pickling.UnPickler$Scan.$anonfun$run$1(UnPickler.scala:98)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.reflect.internal.pickling.UnPickler$Scan.runAtIndex(UnPickler.scala:90)
at scala.reflect.internal.pickling.UnPickler$Scan.run(UnPickler.scala:98)
at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:47)
at scala.tools.nsc.symtab.classfile.ClassfileParser.unpickleOrParseInnerClasses(ClassfileParser.scala:1182)
at scala.tools.nsc.symtab.classfile.ClassfileParser.parseClass(ClassfileParser.scala:466)
at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parse$2(ClassfileParser.scala:160)
at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parse$2$adapted(ClassfileParser.scala:146)
at scala.reflect.internal.util.ReusableInstance.using(ReusableInstance.scala:30)
at scala.tools.nsc.symtab.classfile.ClassfileParser.$anonfun$parse$1(ClassfileParser.scala:146)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.tools.nsc.symtab.classfile.ClassfileParser.pushBusy(ClassfileParser.scala:129)
at scala.tools.nsc.symtab.classfile.ClassfileParser.parse(ClassfileParser.scala:145)
at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader.doComplete(SymbolLoaders.scala:333)
at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:240)
at scala.reflect.internal.Symbols$Symbol.completeInfo(Symbols.scala:1542)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
at scala.reflect.internal.Symbols$Symbol.tpeHK(Symbols.scala:1468)
at scala.reflect.internal.Types$Type.computeMemberType(Types.scala:735)
at scala.reflect.internal.Types$Type.memberType(Types.scala:732)
at scala.reflect.internal.TreeGen.mkAttributedSelect(TreeGen.scala:248)
at scala.reflect.internal.TreeGen.mkAttributedRef(TreeGen.scala:174)
at scala.reflect.internal.TreeGen.mkAttributedStableRef(TreeGen.scala:209)
at scala.tools.nsc.ast.TreeGen.mkImportFromSelector(TreeGen.scala:43)
at scala.tools.nsc.ast.TreeGen.mkWildcardImport(TreeGen.scala:33)
at scala.tools.nsc.typechecker.Contexts.$anonfun$rootContext$1(Contexts.scala:111)
at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
at scala.collection.immutable.List.foldLeft(List.scala:91)
at scala.collection.TraversableOnce.$div$colon(TraversableOnce.scala:179)
at scala.collection.TraversableOnce.$div$colon$(TraversableOnce.scala:179)
at scala.collection.AbstractTraversable.$div$colon(Traversable.scala:108)
at scala.tools.nsc.typechecker.Contexts.rootContext(Contexts.scala:111)
at scala.tools.nsc.typechecker.Contexts.rootContext$(Contexts.scala:110)
at scala.tools.nsc.Global$$anon$5.rootContext(Global.scala:483)
at scala.tools.nsc.typechecker.Analyzer$namerFactory$$anon$1.apply(Analyzer.scala:50)
at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:454)
at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:402)
at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1511)
at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1495)
at scala.tools.nsc.Global$Run.compileSources(Global.scala:1488)
at scala.tools.nsc.DeterminismTester.compile$1(DeterminismTester.scala:56)
at scala.tools.nsc.DeterminismTester.$anonfun$test$7(DeterminismTester.scala:97)
at scala.tools.nsc.DeterminismTester.$anonfun$test$7$adapted(DeterminismTester.scala:94)
at scala.collection.immutable.List.foreach(List.scala:431)
at scala.tools.nsc.DeterminismTester.test(DeterminismTester.scala:94)
at scala.tools.nsc.DeterminismTester$.main(DeterminismTester.scala:34)
at scala.tools.nsc.DeterminismTester.main(DeterminismTester.scala)
Later, the namer enters a source file loader for scala.Predef
, running through this code:
def enterClassSymbol(tree: ClassDef): Symbol = {
val existing = context.scope.lookup(tree.name)
val isRedefinition = (
existing.isType
&& existing.isTopLevel
&& context.scope == existing.owner.info.decls
&& currentRun.canRedefine(existing)
)
val clazz: Symbol = {
if (isRedefinition) {
updatePosFlags(existing, tree.pos, tree.mods.flags)
setPrivateWithin(tree, existing)
clearRenamedCaseAccessors(existing)
existing
}
else enterInScope(assignMemberSymbol(tree)) setFlag inConstructorFlag
}
clazz match {
case csym: ClassSymbol if csym.isTopLevel => enterClassSymbol(tree, csym)
case _ => clazz
}
}
updatePosFlags
resets the existing symbolOf[Predef.type]
to represent the source file. But it seems we've already performed some typechecking sees (potentially stale) information from Predef.class
. The error mode here isn't related to stateness, but rather to having two distinct identical symbols for the same entity, a detail which shows up in the number of entries of in the pickle.
We should see if this problem generalizes to user code (that isn't redefining Predef
from source). Even if the problem is ours alone, a fix is desirable as it would let us use src/library as a regression test for determism without needed to doctor to the test to pass Predef.scala
first on the source file list or somesuch.
Activity
som-snytt commentedon Jul 17, 2020
I wonder if 😕 plus ❤️ adequately conveys the feeling, in the absence of 😿 .
retronym commentedon Jul 18, 2020
retronym commentedon Jul 20, 2020
Here's a variant in userland, where the package object does not participate in root imports.
When
th.scala
appears beforepackage.scala
, naming thePackageDef
inth.scala
with:... forces the info of the the package class symbol, including entering the members of the
package.class
. Again, we end up with two symbols representing the same entity which results in a different (albeit semantically equivalent) pickle. I'm pretty sure there is also a correctness bug here in that references to members ofpackage.class
that no longer inpackage.scala
might not result in "no such symbol" errors.dwijnand commentedon Oct 20, 2020
Pushed as I assumed you wouldn't work on this in the next 10 days...
retronym commentedon Jun 10, 2021
Possibly helpful is this idea for lazier package object loading: scala/scala#9664