How are multiple dependencies on one itest_service handled?

[peakschris]

I can't find this in the documentation:

Assume that I have multiple service_tests that all depend on a single itest_service with fixed port.

Questions

• does rules_itest handle this gracefully when running on a local machine (no remote execution)?
• It appears that rules_itest does not serialize the tests, so presumably allows both tests to concurrently access the service, but I'm not sure if the startup/shutdown of the service is coordinated between the multiple test cases?
• If the service does not support concurrency, can this be controlled?

If ports are auto-assigned, do two tests depending on one itest_service cause two services to be started on different ports, or do they still share one?

Thanks for the clarifications!

Chris

[dzbarsky]

There is no sharing of services between tests, this is mentioned on the front page. (Let me know if this needs to be reworded to be clearer, or if you missed it by only looking at the autogenerated docs page, in which case we can add some prose there as well)

This means that every test will try to start up and healthcheck the services according to the provided specs. How well this works depends on your sandboxing strategy; for example docker-sandbox or linux-sandbox with block-network implemented network namespacing, so you can have the same service bind the same port across different tests in parallel, and they won't interfere. Without network namespacing, a service that tries to bind an already-bound port will likely crash, which would fail the test, but it depends on how it is implemented. There's a chance that it stays alive and the healthcheck picks up the other service, which would be an example of non-hermetic behavior and likely lead to test flakes.

One of the goals of this ruleset is to support local execution on darwin, which doesn't have network namespaces; that's why we support the port autoassignment. Under this model, we bind port 0 to let the OS assign one, so two tests will get different ports.

There is a small race in port autoassignment because the port is bound by the service manager, then released, then bound by the service, so it may be reassigned by the kernel during that short window. I have observed that in practice this doesn't happen on darwin - ports keep incrementing and wrap around after 65K. It does happen on linux, but the network namespace prevents it there. I'm not sure what happens on Windows; if you try it, let me know!

There's also an option to use SO_REUSEPORT and avoid releasing the port, thus preventing the race completely, but I believe this capability doesn't exist on windows

[peakschris]

Ah, that's good to know, thanks. I did miss that sentence on the front page (looking at autogen docs), but also I think it could be reworded, because I don't know precisely what 'fresh' means ;-)... something along these lines? ... it is guaranteed to be provisioned with its own set of newly started services .... Actually, maybe a whole separate section covering test concurrency and including the stuff you wrote above would be helpful.

So I'm trying to run two tests on windows without sandboxing that depend on a single service with fixed port, and they both run concurrently and pass. It appears that our service doesn't return an error when it fails to bind; presumably both test cases then end up hitting the first service. I think this is our issue to fix.

I've switched to using autoassigned IP addresses and this is working - thanks!