[SPARK-53958][BUILD] Simplify Jackson deps management by using BOM #52668
Conversation
| <scalafmt.validateOnly>true</scalafmt.validateOnly> | ||
| <scalafmt.changedOnly>true</scalafmt.changedOnly> | ||
| <fasterxml.jackson.version>2.19.2</fasterxml.jackson.version> | ||
| <fasterxml.jackson.databind.version>2.19.2</fasterxml.jackson.databind.version> |
There was a problem hiding this comment.
in case that jackson-databind has a standalone patched version, the jackson-bom also has a new release, for example
https://mvnrepository.com/artifact/com.fasterxml.jackson/jackson-bom/2.13.4.20221013
|
I hit some Jackson deps issues (#52664 (comment)) in upgrading Avro to 1.12.1, and haven't figured out the root cause, so I am looking into this new simple approach to address the issues. cc @LuciferYang @sarutak @dongjoon-hyun WDYT? |
There was a problem hiding this comment.
Could you revert all elimination of <exclusion> tags from this PR?
I believe this PR should work without touching the existing exclusion rules in order to verify that BOM is actually working correctly, @pan3793 .
|
@dongjoon-hyun, either w/ or w/o the elimination of |
There was a problem hiding this comment.
+1, LGTM. Thank you for revising this PR, @pan3793 .
Merged to master for Apache Spark 4.1.0-preview3.
3 participants
What changes were proposed in this pull request?
Currently, it's relatively complicated with dependency version management due to the different resolution mechanisms between Maven and SBT.
Take Jackson deps as an example, to ensure all Jackson deps have the same version on the classpath, for both Maven and SBT, we must exclude all transitive Jackson deps and re-declare them manually, otherwise SBT may pick the higher version from transitive deps then breaks the building.
This PR proposes to use BOM to simplify the above process, see technical details at https://github.com/FasterXML/jackson-bom
Maven has built-in support for BOM, SBT also has a plugin https://github.com/heremaps/here-sbt-bom
Why are the changes needed?
Simplify dependency management.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass GHA.
Manually checked the output of
build/sbt dependencyTree, all Jackson deps are correctly resolved to the expected version - 2.19.2.Was this patch authored or co-authored using generative AI tooling?
No.