User stories as gherkin feature files

[dwhswenson]

This will document our user stories as Gherkin-formatted tests. Gherkin is easily human-readable and focuses on functionality, not implementation. Therefore, it makes a good format for specifying requirements. The tests can also be translated to code using tools like pytest-bdd, making our requirements document directly testable.

Another advantage of Gherkin tests is that they are also relatively straightforward to perform manually. I suspect that we will start by doing the manual tests, and get to automating them much later.

[ethanholz]

This is kind of inherent to the architecture we seek to design. Not sure if we need to add this.

[dwhswenson]

The two "And" statements here are the test that we ran all jobs in parallel. If we want running the matrix in parallel to be a requirement, then we should probably test the requirement. The initial "Then" statement would also be satisfied if the matrix was run serially.

It might be better gherkin to combine the two "And" statements into a single "And all matrix jobs should run in parallel"? There's a trade-off between making the statement represent less code (better for the developer) and making the statement's purpose clearer to readers (better for the client).

[ethanholz]

I have added three new scenarios around ML/AI workloads and ROCm. Please take a look whenever you get the chance!

[dwhswenson]

I commented out the ROCM scenario, since it isn't in the MVP. The main goal here is to capture the MVP as features/scenarios. I'm happy to put future plans in comments, but this makes it more clear that it will not be tested now.

Others looked good, though! Thanks!

[ethanholz]

I think we are good to start moving forward on getting this merged. A couple things that could be nice to see:

1. Rebase/merge this with the current main and determine where the tests should live so they work best with pytest-bdd
2. Add pytest-bdd to our pyproject.toml file to ensure that we include this in the testing toolchain