Do not rely on Redis compiled from source code #3734

gerzse · 2024-02-20T21:17:45Z

Instead of building Redis server from source code, use Docker images. Prepare docker-compose stacks that accept the version of the Docker image as parameter, so we can run against multiple versions in CI.

The purpose of this change is to strictly move everything to Docker, without trying to change anything else. Some Java tests had to be adapted, still, because now the Redis instances are accessed via localhost, while internally in the docker-compose network they see each other with other IPs and ports.

Polish a bit the Makefile, for example to run exactly the same thing that runs in CI, so when we run the tests locally we get exactly the same coverage.

.github/workflows/integration.yml

.gitignore

chayim

Top-line comment:

Is there some way to remove all the hardcoded ports (or another PR?). It seems to be a constant theme throughout, and worth the debt removal.
That raises the question, is there a (painless) way to inject into the various configs, like stunnel and the dockers.

src/test/java/redis/clients/jedis/ClusterPipeliningTest.java

chayim · 2024-02-21T07:17:39Z

src/test/java/redis/clients/jedis/JedisClusterTest.java

@@ -216,14 +216,14 @@ public void testMigrate() {
        node2.set("e", "e");
      } catch (JedisMovedDataException jme) {
        assertEquals(15363, jme.getSlot());
-        assertEquals(new HostAndPort(LOCAL_IP, nodeInfo3.getPort()), jme.getTargetNode());
+        assertEquals(new HostAndPort(JedisClusterTestUtil.getClusterIp(3), 6379), jme.getTargetNode());


Same general comment re ports - throughout.

Yes, we have to do something about this. It's so spaghetti right now, implied relations between ports and stuff. My intention was to leave that for another PR, go step by step. Here I did a minimal mapping to the new Docker IPs and ports.

chayim · 2024-02-21T07:23:26Z

test-infra/no-cluster/stunnel/pki/private.pem

@@ -0,0 +1,48 @@
+-----BEGIN RSA PRIVATE KEY-----


So we made this exact same mistake in redis-py, which IMHO is tech debt. When I originally committed it, I thought it smarter to commit junk certs (and a README), but it led to fallout. Since they're generated via this script from redis, maybe we should just fetch it, and run it first? We'd need a modification given the last line, but the only requirement is openssl.

I agree, let's integrate that. In another PR?

Issue redis#3710 Instead of building Redis server from source code, use Docker images. Prepare docker-compose stacks that accept the version of the Docker image as parameter, so we can run against multiple versions in CI. The purpose of this change is to strictly move everything to Docker, without trying to change anything else. Some Java tests had to be adapted, still, because now the Redis instances are accessed via localhost, while internally in the docker-compose network they see each other with other IPs and ports. Polish a bit the Makefile, for example to run exactly the same thing that runs in CI, so when we run the tests locally we get exactly the same coverage.

Don't change some imports so much. Fix a typo, that was me trying to open pavucontrol.

Fix some failing tests, the mapping of hosts/ports was broken by the earlier changes. Move some core around, as suggested during code review.