Improve our e2e setup#190
Conversation
|
manuelnaranjo
force-pushed
the
mnaranjo/revamp-e2e
branch
15 times, most recently
from
1099183 to
408f40d
Compare
|
Blocked by #191, we need 191 merged first as this PR builds on top |
Make sure some golang dependencies are up to date
Start using rules_bazel_integration_test to drive unit tests, this initial step allows us to simplify our e2e testing, next commit will let us speed up the build by avoiding rebuilding the toolchain over and over again
Make sure that all our e2e tests use a prebuilt golang binary for the host platform and that we don't build it once for each test
Make sure all of our e2e use a shared prebuilt bazeldnf binary instead of each one building their own, this simplifies our MODULE.bazel for all of our e2e tests considerably as we no longer to keep prebuilt protoc or golang dependencies
Make sure changes to MODULE.bazel.lock and lock files don't invalidate the cache
manuelnaranjo
force-pushed
the
mnaranjo/revamp-e2e
branch
from
38751f6 to
83bff54
Compare
manuelnaranjo
force-pushed
the
mnaranjo/revamp-e2e
branch
2 times, most recently
from
f5ae488 to
2dfc1db
Compare
|
As a result it looks like we're running way less checks on the PR. Any particular reason why we want to do this? |
That's a good observation but it's misleading. Before we had n GitHub CI actions 1 for each e2e job + build and unit tests, because we needed on each one of those to build the bazeldnf binary and that is expensive, now we can build once and reuse on a single action. The GitHub UI reports 1 check per ci action which makes it look like less amount of checks. But if you expand on the build and test toi will find sections for unit tests, e2e that has to pass and e2e allowed to fail. Not sure if GitHub supports integrating the Bazel report of checks from a report like junit which Bazel can generate. I can investigate that and do as a follow up improvement |
|
This is how it looks like |
https://g.co/gemini/share/9a29c1317f54 alright Gemini gives something that looks doable, not sure if it's hallucinating |
manuelnaranjo
force-pushed
the
mnaranjo/revamp-e2e
branch
from
2dfc1db to
624c6e7
Compare
|
@kellyma2 once we merge the test reporter should start commenting the PRs with the test report, due to github we need a workflow to run from our protected branch, and then it needs to be merged first |
Drive tests entirely through bazel even on CI. Given we're using setup-bazel in our CI we need a little of a workaround for a bug
give each job in the build and test workflow a meaningful name
With the help of Claude I was able to come with a decent contributing doc, of course it needed some cosmetics and modifications, but the result is decent
Make sure we use prebuilt bazeldnf when testing bzlmod releases
manuelnaranjo
force-pushed
the
mnaranjo/revamp-e2e
branch
2 times, most recently
from
deeff50 to
5e2ca56
Compare
Actually it doesn't work as I expected, it creates a test report https://github.com/bookingcom/bazeldnf/actions/runs/23236541743 but it doesn't show up in the Pull Request bookingcom#4, so I reverted that change as it's useless |
2 participants
Our e2e setup was a bit messy and not easy or fast enough to run locally, making us rely heavily on CI.
With
rules_bazel_integration_testwe can have Bazel drive the e2e tests and this means we can build the tool once,and reuse on every single one of our e2e test, this also means not only we can save on CPU by not rebuilding but also
we can paralelize the executions speeding up our CI and our local testing.
With this setup it's now possible to do
bazel test e2eto get everything tested. Given these tests are very expensive they are marked as manual to avoid running them withbazel test //...