Background

Currently, the CU stores its json results in sqlite or postgres. This is not optimal for large results especially when getting a list. Also when booting up a new CU and using it for the same processes, unless it is using postgres it will not have access to the previously cached results and will have to evaluate them.

Solution

1. Create 2 environment variables EVALUATION_RESULTS_DIR and EVALUATION_RESULTS_BUCKET they will default to not being set and if they are set the feature is enabled.
2. Create a hydrator worker modelled off of the evaluation worker. and compose it into ao-evaluation.js.
3. ao-evaluation.js will ask hydrator for eval results, hydrator will then use EVALUATION_RESULTS_DIR and EVALUATION_RESULTS_BUCKET to try and retrieve cached results. ao-evaluation will use these in output.
4. Current persistance in ao-evaluation will remain the same the hydrator will just hydrate and flush from the persistance in ao-evaluation.js
5. EVALUATION_RESULTS_BUCKET will be a remote storage system like s3.
6. hydrator will flush a single process or all processes (compose the single process function into the many one) upon receiving a signal.
7. The CU will signal the hydrator upon receiving its own signal to flush a single or many processes.