KMS-522: Functionality to handle syncing with gcmd.earthdata.nasa.gov/kms

[cgokey]

Overview

What is the feature?

We need to be able turn on sync so RDF database stays up to date over the next couple months while we build
additional features.

What is the Solution?

gcmd.earthdata.nasa.gov/kms/concepts endpoint will provide both xml dump and json dump of its concept data.
This concept data is used to build the RDF.

Schedules a nightly cron that will download xml and json data for both draft and published and recreate these versions nightly.

I support of this feature, I added a couple npm run scripts:
npm run download-files will download the json/xml files for processing.
npm run create-rdf-files will create rdf xml data files from these json/xml files.

What areas of the application does this impact?

Keeping production kms data in sync with new system.

Testing

To test locally, once kms sit endpoints are ready, you should be able to start offline mode with:
SHOULD_SYNC=true SYNC_API_ENDPOINT=https://gcmd.sit.earthdata.nasa.gov RDF4J_SERVICE_URL=http://localhost:8080 npm run offline
curl -X POST http://localhost:4001/dev/sync-concept-data -H "Content-Type: application/json" -d '{"version": "published"}'
Watch the logs in the console of the terminal running offline mode and verify you don't see any errors.

The publish endpoint has also changed to be asynchronous, as this endpoint was dangerously close to taking more than 30 seconds and API gateway has a max timeout of 30 seconds, to test this new endpoint, do the following:
curl -X POST http://localhost:4001/dev/publish -H "Content-Type: application/json" -d '{"name": "9.1.6"}'
and will kick off a publish. Check the logs in terminal to verify no errors occurred.
Will create a 9.6.1 published version.

I've also added some business logic to create/update dcterms:created and dcterms:modified whenever a concept is created or updated.

Checklist

• I have added automated tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.74%. Comparing base (873ae24) to head (b70a8d5).
Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #35      +/-   ##
==========================================
- Coverage   99.75%   99.74%   -0.01%     
==========================================
  Files         103      120      +17     
  Lines        1646     1977     +331     
  Branches      385      473      +88     
==========================================
+ Hits         1642     1972     +330     
- Misses          4        5       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.