[FLINK-38054] Fix OperationSerializer issue for es8 sink - #159
Conversation
Co-authored-by: Andrey Starostin <andrey.starostin@choco.com>
| DefaultInstantiatorStrategy instantiatorStrategy = new DefaultInstantiatorStrategy(); | ||
| instantiatorStrategy.setFallbackInstantiatorStrategy(new StdInstantiatorStrategy()); | ||
| kryo.setInstantiatorStrategy(instantiatorStrategy); |
There was a problem hiding this comment.
DefaultInstantiatorStrategy is the recommended way of creating objects with Kryo. It runs constructors just like would be done with Java code. StdInstantiatorStrategy uses JVM specific APIs to create an instance of a class without calling any constructor at all. Using this is dangerous because most classes expect their constructors to be called. Creating the object by bypassing its constructors may leave the object in an uninitialized or invalid state. (https://github.com/EsotericSoftware/kryo?tab=readme-ov-file#instantiatorstrategy)
This address #136 (comment)
|
@mayorandrew, Thanks for reporting and fixing this 👍 , it's a critical bug. Since you haven't updated the original PR for a long time, I took over it to unblock the plan of release next version. Your copyright has already been retained through co-authorship. |
|
Thanks for the review. |
2 participants
By default,
OperationSerializerusesAppClassLoaderas the class loader forKryo. When the user ship the es connector uber jar into the job's jar (Loading viaUserCodeClassLoader), they will encounter aClassNotFoundissue.BTW, I created FLINK-40085 to add an end-to-end test for the es8 sink also.
This coauthored by @mayorandrew, the original pr is #136. I think using a custom serializer for JsonNode is a bit of a hack and it cannot cover all situations. This pr manage to address this issue in another way.