Database support across runs

[adayton1]

In one of the applications I work on, we try to monitor performance over time. We do a simulation run nightly, and use spot to track the data. However, generating a new file for every run is turning out to not be very scalable for visualization. What I'd like to see is the ability to collect all simulation runs into a single database, not a different database for every run.

[daboehme]

Hi @adayton1 , are you looking for a convenient way to work with a set of runs or are you running into scalability issues with too many files? For the former, check out Thicket https://github.com/LLNL/thicket, which is a Python framework designed for ensemble analysis based on the spot .cali files. We also have a way to merge .cali files into a SQL database that works with the Spot web framework. Might be a little more involved to get working, so I'd like to understand your use case a bit better first.