Skip to content

Conversation

vladak
Copy link
Member

@vladak vladak commented Nov 12, 2024

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Nov 12, 2024
@vladak
Copy link
Member Author

vladak commented Nov 12, 2024

Looks like this needs some work in the suggester:

[INFO] Running org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest
Error:  Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 2.381 s <<< FAILURE! - in org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest
Error:  org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest.testResize  Time elapsed: 0.254 s  <<< ERROR!
com.thoughtworks.xstream.converters.ConversionException: 
unable to convert node named=org.apache.lucene.util.BytesRef
---- Debugging information ----
message             : unable to convert node named=org.apache.lucene.util.BytesRef
class               : net.openhft.chronicle.map.VanillaChronicleMap
required-type       : net.openhft.chronicle.map.VanillaChronicleMap
converter-type      : net.openhft.xstream.converters.VanillaChronicleMapConverter
line number         : -1
version             : 1.4.21
-------------------------------
	at net.openhft.xstream.converters.AbstractChronicleMapConverter.unmarshal(AbstractChronicleMapConverter.java:133)
	at net.openhft.xstream.converters.VanillaChronicleMapConverter.unmarshal(VanillaChronicleMapConverter.java:28)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1468)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1445)
	at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1334)
	at net.openhft.chronicle.map.JsonSerializer.putAll(JsonSerializer.java:70)
	at net.openhft.chronicle.map.AbstractChronicleMap.putAll(AbstractChronicleMap.java:64)
	at org.opengrok.suggest.popular.impl.chronicle.ChronicleMapAdapter.resize(ChronicleMapAdapter.java:139)
	at org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest.testResize(ChronicleMapAdapterTest.java:88)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Error:  org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest.dataNotLostAfterResizeTest  Time elapsed: 0.028 s  <<< ERROR!
com.thoughtworks.xstream.converters.ConversionException: 
unable to convert node named=org.apache.lucene.util.BytesRef
---- Debugging information ----
message             : unable to convert node named=org.apache.lucene.util.BytesRef
class               : net.openhft.chronicle.map.VanillaChronicleMap
required-type       : net.openhft.chronicle.map.VanillaChronicleMap
converter-type      : net.openhft.xstream.converters.VanillaChronicleMapConverter
line number         : -1
version             : 1.4.21
-------------------------------
	at net.openhft.xstream.converters.AbstractChronicleMapConverter.unmarshal(AbstractChronicleMapConverter.java:133)
	at net.openhft.xstream.converters.VanillaChronicleMapConverter.unmarshal(VanillaChronicleMapConverter.java:28)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1468)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1445)
	at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1334)
	at net.openhft.chronicle.map.JsonSerializer.putAll(JsonSerializer.java:70)
	at net.openhft.chronicle.map.AbstractChronicleMap.putAll(AbstractChronicleMap.java:64)
	at org.opengrok.suggest.popular.impl.chronicle.ChronicleMapAdapter.resize(ChronicleMapAdapter.java:139)
	at org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest.dataNotLostAfterResizeTest(ChronicleMapAdapterTest.java:67)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

@vladak
Copy link
Member Author

vladak commented Nov 13, 2024

It seems that the recent version of xtream got more strict in terms of (de)serialization and now demands explicit converter, in our case for the Lucene's BytesRef. The converter needs to be implemented and plugged into xstream using the registerConverter() API as described on http://x-stream.github.io/javadoc/index.html

Looking at com.thoughtworks.xstream.converters.extended no such thing is there.

Some inspiration can be found e.g. on x-stream/xstream#293

@vladak
Copy link
Member Author

vladak commented Nov 13, 2024

The XStream object is created inside ChronicleMap code and there does not seem to be a way how to access it or modify its converters. This would be possible if the putAll() method allowed to pass the converters to the JsonSerializer in AbstractChronicleMap/VanillaChronicleMap however this is not the case as it uses just the empty list (decompiled code) :

default void putAll(File fromFile) throws IOException {
        synchronized(this) {
            JsonSerializer.putAll(fromFile, this, Collections.emptyList());
        }
    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement. suggester
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants