HDDS-15808. Replace json-simple with Jackson in httpfsgateway#10710
Conversation
The httpfsgateway module was the only remaining consumer of the unmaintained com.googlecode.json-simple:json-simple library. Jackson (jackson-databind) is already a compile dependency, so this migrates all JSON serialization to Jackson and drops json-simple from the build. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR migrates the hadoop-ozone/httpfsgateway module off the unmaintained json-simple library by switching JSON serialization to Jackson (already present as a dependency), and then removing json-simple from the build and distribution inventories.
Changes:
- Replaced
json-simple-based JAX-RS JSON serialization with Jackson in providers and response-building code paths. - Updated httpfsgateway JSON-producing server operations to build
Map/Liststructures instead ofJSONObject/JSONArray. - Added focused unit tests for the new Jackson serialization behavior and removed
json-simplefrom module and distro license inventories.
Reviewed changes
Copilot reviewed 8 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| hadoop-ozone/httpfsgateway/src/test/java/org/apache/ozone/lib/wsrs/TestJSONMapProvider.java | Adds tests validating JSON framing (newline) and stream-close behavior for JSONMapProvider. |
| hadoop-ozone/httpfsgateway/src/test/java/org/apache/ozone/lib/service/instrumentation/TestInstrumentationSerialization.java | Adds tests validating @JsonValue-driven Jackson serialization shape for instrumentation snapshots. |
| hadoop-ozone/httpfsgateway/src/main/java/org/apache/ozone/lib/wsrs/JSONProvider.java | Removes the json-simple JSONStreamAware writer provider. |
| hadoop-ozone/httpfsgateway/src/main/java/org/apache/ozone/lib/wsrs/JSONMapProvider.java | Switches map serialization from json-simple to Jackson with stream-close behavior preserved. |
| hadoop-ozone/httpfsgateway/src/main/java/org/apache/ozone/lib/service/instrumentation/InstrumentationService.java | Removes json-simple interfaces and uses @JsonValue to serialize snapshot types as maps. |
| hadoop-ozone/httpfsgateway/src/main/java/org/apache/ozone/fs/http/server/HttpFSServer.java | Updates endpoints to return Map-backed JSON payloads instead of JSONObject. |
| hadoop-ozone/httpfsgateway/src/main/java/org/apache/ozone/fs/http/server/FSOperations.java | Replaces JSONObject/JSONArray construction with Map/List and uses JsonUtil for serialization. |
| hadoop-ozone/httpfsgateway/pom.xml | Removes the json-simple dependency from the httpfsgateway module. |
| hadoop-ozone/dist/src/main/license/jar-report.txt | Removes json-simple.jar from the distribution jar inventory. |
| hadoop-ozone/dist/src/main/license/bin/LICENSE.txt | Removes com.googlecode.json-simple:json-simple from the distribution LICENSE inventory. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
PMD's UnusedPrivateMethod flagged the three @JsonValue-annotated getJSON() methods (Timer, VariableHolder, Sampler) as unused because it does not see Jackson's reflective invocation. Reducing them to package-private takes them out of the rule's scope while keeping them effectively internal; Jackson serializes @jsonvalue methods at any visibility. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks @smengcl for the patch. There are some PMD failures: Oh, I see already fixed. |
…ale comment FSCreateSnapshot no longer strips all backslashes from the serialized JSON. That strip existed only to undo json-simple's forward-slash escaping (\/); Jackson does not escape forward slashes, so the strip was a no-op for normal paths and would corrupt any value Jackson legitimately escapes (\", \\, \n). Return the Jackson output as-is. Also reword the misspelled, stale VisibilityModifier comment on VariableHolder.var to describe the actual reason for the suppression. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…onAware The toJSON Javadoc still described the value as a json-simple JsonAware instance. Update it to reflect Jackson serialization. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 10 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
hadoop-ozone/httpfsgateway/pom.xml:52
- The PR description says the json-simple version property and dependencyManagement entry were removed from the root
pom.xml, but they are still present (egpom.xml:121andpom.xml:459-463). If json-simple is truly no longer used anywhere, please remove those root-pom entries as well (or update the PR description/dist license inventory accordingly) so the dependency is fully eliminated.
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
</dependency>
…ypes Tighten the generic types of the JSON array locals introduced by the migration (previously json-simple JSONArray) to their actual element types: List<Map<String, Object>> for status/policy lists and List<String> for ACL entries and storage type names. No change to serialized JSON. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 10 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
hadoop-ozone/httpfsgateway/pom.xml:52
- The module dependency on json-simple is removed here, but the repository root
pom.xmlstill defines<json-simple.version>and adependencyManagemententry forcom.googlecode.json-simple:json-simple(egpom.xml:121andpom.xml:459-463). This contradicts the PR description’s claim that the root pom cleanup is included, and it leaves stale dependency metadata behind. Please remove those root pom entries (and any other remaining references) to complete the migration/removal.
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>jakarta.ws.rs</groupId>
<artifactId>jakarta.ws.rs-api</artifactId>
</dependency>
The root pom still declared the json-simple version property and a dependencyManagement entry after the module dependency was dropped. No module or source references json-simple anymore, so remove both to complete the removal. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Narrow the JSON array introduced by the migration to List<Map<String, Object>> and parameterize the raw Map locals and return type in xAttrsToJSON. The unchecked/rawtypes suppression is no longer needed. No change to serialized JSON. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…eway The migration references jackson-core (JsonGenerator.Feature in JSONMapProvider) and jackson-annotations (@jsonvalue in InstrumentationService) directly, but the module only pulled them in transitively via jackson-databind. maven-dependency-plugin analyze-only flagged them as used-undeclared. Declare both explicitly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks @smengcl for the patch. |
|
Thanks @adoroszlai for the review and merging this. |
Generated-by: Claude Code (Opus 4.8)
What changes were proposed in this pull request?
The
hadoop-ozone/httpfsgatewaymodule was the only remaining consumer ofcom.googlecode.json-simple:json-simple(1.1.1), a library that has been unmaintained since 2012. Jackson (jackson-databind) is already a compile dependency of the module, so json-simple was redundant and carried needless maintenance and supply chain risk.This PR migrates all JSON serialization in httpfsgateway from json-simple to Jackson, then removes the dependency:
JSONMapProvidernow serializes the responseMapwith a sharedObjectMapper.AUTO_CLOSE_TARGETis disabled so the underlying response stream stays open for the trailing newline and container managed close, matching the previous behavior.JSONProvider, whose only purpose was serializing json-simpleJSONStreamAwareentities, is removed. JAX-RS providers are registered by Jersey package scanning viajersey.config.server.provider.packages, so deleting the class is sufficient to unregister it.InstrumentationService, theTimer,VariableHolder, andSamplersnapshot types no longer implementJSONAware/JSONStreamAware. TheirgetJSON()methods return aLinkedHashMapannotated with@JsonValue, so Jackson serializes each object as that map and produces the same JSON.FSOperationsandHttpFSServeruseMap/List(LinkedHashMap/ArrayList) in place ofJSONObject/JSONArray, and reuse the module localJsonUtil.toJsonStringhelper.dependencyManagemententry are removed from the root pom. Thecom.googlecode.json-simple:json-simpleentries in the distributionbin/LICENSE.txtand the generatedjar-report.txtinventory are removed accordingly. The module also declaresjackson-coreandjackson-annotationsexplicitly, since the code now referencesJsonGeneratorand@JsonValuedirectly rather than only transitively throughjackson-databind.The wire format is preserved. Both libraries emit standard JSON,
AtomicLongserializes as a bare number in both, and switching fromHashMaptoLinkedHashMaponly makes key order deterministic, which WebHDFS clients (which parse by key) are unaffected by. ThereplaceAll("\\\\", "")call inFSCreateSnapshot, originally there to undo json-simple's forward slash escaping (\/), is removed. Jackson does not escape slashes, and stripping all backslashes could corrupt legitimately escaped values.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15808
How was this patch tested?
Two focused unit tests were added to the module (which previously had almost no coverage for these paths):
TestInstrumentationSerializationverifies the@JsonValueserialization ofTimer,VariableHolder, andSamplerproduces the expected JSON keys and values.TestJSONMapProviderverifies the provider emits compact JSON followed by a trailing newline and does not close the underlying stream.Verified locally: the module compiles, all module unit tests pass (2 existing + 5 new),
checkstyle.shis clean, andrat.shreports 0 unknown licenses.