🧹 Introduce Python/HTML tests

[rix0rrr]

Motivation

Because it is hard to write Cypress tests, people often don't do it, which increases the risk of regressions in the project. In this PR, I'm making some changes to make it easier to write tests on the Flask code in Python in the future, which should hopefully make it easier to write tests overall. I'm thinking that Cypress tests have the following downsides which lead us to not writing very many of them:

• Setup is awkward: to test a UI feature, you need the database in a situation where the tested UI element would be relevant. This means a program in state X, or a user in state Y. It is awkward to use the UI in Cypress to set up a database this way.
• Wait problems: complex interactions often have multiple steps, and it is easy to have race conditions in Cypress tests, where tests work on your machine but fail on the build server because of timing issues. Instead, in Python tests we'll make assertions on the HTML sent to the client, which is either good or bad in one shot.

Testing in Python is not a panacea: we will obviously not be able to test anything that involves JavaScript, and we'll be asserting on the HTML only which of course tests nothing about presentation or element visibility.

I'm not proposing to replace all our tests with Python/HTML tests, just to get more coverage more easily this way.

If nothing else, the changes in this PR will help clean up the code base a little 😉.

Motivation

Previous, we used to create a global

app = Flask(...)

object, and initialize it over the course of the app.py file.

This works well to get a server up and running, but it makes it harder to test the code: for tests, you want fresh objects for every run.

In this PR, move the initialization code into a create_app() function, so that we can create fresh instances of a Flask object on demand. Also write some functions to clear the in-memory database state, and write some Python/HTML tests to show what it looks like.

This suite can also replace our current out-of-process "e2e" tests (tests_e2e.py) because it is ~the same, but hopefully easier to write.

[boryanagoncharenko]

Awesome refactoring.

Just want to point out that the biggest issue with adding cypress tests is that the cypress suite takes 20-25 minutes to run. Several months ago this has been brought up and it was clear that the execution time was considered to be very long. I am convinced that the python tests will help in this regard.

A small comment on the implementation, when I run the validate-tests script, I get the following 3 messages which I did not get before:

[2024-12-19 12:10:10,810] log_fetcher.py from_env 28 <WARNING>: Unable to initialize Athena client (missing AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY)
[2024-12-19 12:10:10,811] aws_helpers.py s3_parselog_transmitter_from_env 31 <WARNING>: Unable to initialize S3 parse logger (missing AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY)
[2024-12-19 12:10:10,811] aws_helpers.py s3_querylog_transmitter_from_env 20 <WARNING>: Unable to initialize S3 querylogger (missing AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY)

I noticed that the first message is produced by dead code. The athena_client and the log_fetcher are only used for the enpoints /logs/results and /logs/query which have no incoming traffic.

Let me know if you want to address the warnings.

[rix0rrr]

I'll remove those lines. I don't think they add a lot of value anymore.

[mergify]

Thank you for contributing! Your pull request is now going on the merge train (choo choo! Do not click update from main anymore, and be sure to allow changes to be pushed to your fork).