Refactor Job Management to Centralize Database Access via FIA-API

[keiranjprice101]

Description of the issue

Description

Currently, the job-creator and job-watcher services interact directly with the database to create and update job records. This creates an issue where the fia-api is unable to return the job ID for jobs it initiates via RabbitMQ since it does not have visibility into job creation.

To resolve this, we will refactor job management so that all job-related database interactions go through the fia-api. This will ensure consistency and allow the fia-api to return job IDs when initiating jobs.

Proposed Work Plan:

Modify Job Creator Behavior

Instead of writing directly to the database, the job-creator will call a new FIA-API endpoint to register new jobs.
The job creator will always need to recieve the id of the job that it is using for a simple job and a rerun.
For the rerun it will need to rename it's job_id parameter (in the rabbit mq message) to original_id. We should also probably rename job_id to id

Modify Job Watcher Behavior

Instead of updating the database directly, the job-watcher will call a FIA-API endpoint to update job statuses when jobs complete or fail.

Modify FIA-API

Implement an API endpoint for job creation that inserts job records into the database.
Implement an API endpoint for updating job statuses.

Possible Approach: As there is already a jobs endpoint, it might make sense to simply define a POST and PATCH methods on it rather than creating a separate endpoint. This should be considered further during implementation.

Decide on the Future of the db Package

The db package is currently imported by all three services. Since database interactions will now be restricted to the FIA-API, the package is no longer explicitly required by the job-creator and job-watcher.
Possible approaches:
Migrate the db package into the FIA-API, removing it from the other services.
Keep it as a type library, allowing job-creator and job-watcher to reference job types while removing its database interaction logic.

Decisions to be Finalized

Should the db package be fully removed from job-creator and job-watcher, or retained as a type library?
Should the POST/PATCH /jobs methods be implemented on the existing jobs endpoint, or should a separate endpoint be created?