[SPEC]: Test failures on the Linux/ARM64 platform

[odidev]

Improvement/Addition type

What changes to the current SPEC does this issue focuses upon?

• [] Add a benchmark variant
• [] Improve a benchmark variant
• [] Raise awareness for a new Redis benchmark tool
• [] Suggest a different Redis Build Variant
• [ * ] Other (fill details in section bellow)

Detail the requested use-case

I am working with redis-benchmark-specification on the Linux/ARM64 Ubuntu AWS instance.
I am following this documentation < https://github.com/redis/redis-benchmarks-specification#development > to execute tests with tox.
Out of 26 tests, 2 tests are failing, as can be seen from the logs below:

utils/tests/test_self_contained_coordinator.py ....F                                                                                                                                     [ 80%] 
utils/tests/test_self_contained_coordinator_memtier.py F..                                                                                                                               [ 92%] 

These 2 test cases are failing for the same assertion issue (while comparing the value of “result”) after calling the function “test_self_contained_coordinator_blocking_read()” in files test_self_contained_coordinator_memtier.py and test_self_contained_coordinator.py.

=================================================================================== short test summary info ==================================================================================== 
FAILED utils/tests/test_self_contained_coordinator.py::test_self_contained_coordinator_blocking_read - assert False == True 
FAILED utils/tests/test_self_contained_coordinator_memtier.py::test_self_contained_coordinator_blocking_read - assert False == True 
==================================================================== 2 failed, 24 passed, 249 warnings in 567.41s (0:09:27) ==================================================================== 
ERROR: InvocationError for command /home/ubuntu/redis-benchmarks-specification/.tox/integration-tests/bin/coverage run '--include=redis_benchmarks_specification/*' -m pytest -ra (exited with code 1) 

On further debugging, it looks like the tests are failing to run docker container using image redislabs/memtier_benchmark:edge here < https://github.com/redis/redis-benchmarks-specification/blob/main/redis_benchmarks_specification/__self_contained_coordinator__/self_contained_coordinator.py#L1025 >, as this docker image is not available for the Linux/ARM64 platform at DockerHub.

I have added a comment in this already raised ticker < RedisLabs/memtier_benchmark#213 > in the memtier-benchmark GitHub repo, requesting the same.

Can you please provide some pointers whether my analysis seems correct?

Also, may I know, are you interested in adding Linux/ARM64 platform support to redis-benchmark-specification? Kindly provide your suggestions on the same.

Screenshots/CLI snippets of what's expected to happen
If applicable, add screenshots to help explain your problem. For example a chart.

Detail the used benchmark tool

What benchamrk tool to use:

• [] memtier_benchmark
• [] redis-benchmark
• [] redis-benchmark-go
• [] Other (name it):

Sample benchmark tool command to implement the requested use-case:

$ git clone https://github.com/redis/redis-benchmarks-specification.git && cd redis-benchmarks-specification
$ pip install poetry
$ poetry install
$ poetry run black .
$ poetry run flake8
$ pip3 install -r ./dev_requirements.txt
$ tox

Additional context
Add any other context about the problem here.

[odidev]

I built multi-arch memtier-benchmark docker image using Dockerfile here < https://github.com/RedisLabs/memtier_benchmark/blob/master/Dockerfile > and replaced “redislabs/memtier_benchmark:edge” with my image in the source code. Also, installed and started redis-server on port 6379. Now, all 26 tests are passing via tox on the Linux/ARM64 platform.

========================================================================= 26 passed, 255 warnings in 437.64s (0:07:17) =========================================================================
integration-tests run-test: commands[4] | coverage report -m
Name                                                                                          Stmts   Miss  Cover   Missing
---------------------------------------------------------------------------------------------------------------------------
redis_benchmarks_specification/__api__/__init__.py                                                0      0   100%
redis_benchmarks_specification/__api__/app.py                                                   118     17    86%   46-49, 54, 59-67, 76, 78, 115
redis_benchmarks_specification/__builder__/__init__.py                                            0      0   100%
redis_benchmarks_specification/__builder__/builder.py                                           191     58    70%   54-144, 167-168, 195, 222, 231-236, 241-258, 270, 296, 342, 373, 381-387, 424-430, 439
redis_benchmarks_specification/__builder__/schema.py                                             13      0   100%
redis_benchmarks_specification/__cli__/__init__.py                                                0      0   100%
redis_benchmarks_specification/__cli__/args.py                                                   38      0   100%
redis_benchmarks_specification/__cli__/cli.py                                                   122     33    73%   44-55, 90, 108, 129-135, 158, 188, 194, 207, 215-230, 243-277
redis_benchmarks_specification/__cli__/stats.py                                                 317    308     3%   20-502
redis_benchmarks_specification/__common__/__init__.py                                             0      0   100%
redis_benchmarks_specification/__common__/builder_schema.py                                      86     11    87%   53, 81-86, 103-106, 114, 151
redis_benchmarks_specification/__common__/env.py                                                 45      1    98%   79
redis_benchmarks_specification/__common__/package.py                                             19     11    42%   10-21
redis_benchmarks_specification/__common__/runner.py                                              84     13    85%   22, 27, 29-40, 45, 50-51, 119-120
redis_benchmarks_specification/__common__/spec.py                                                22      0   100%
redis_benchmarks_specification/__init__.py                                                        6      2    67%   14-15
redis_benchmarks_specification/__runner__/__init__.py                                             0      0   100%
redis_benchmarks_specification/__runner__/args.py                                                47      0   100%
redis_benchmarks_specification/__runner__/runner.py                                             430    156    64%   92-100, 105-106, 142-150, 166-174, 177, 235, 251-254, 266-314, 380, 389-390, 420, 436-460, 473-480, 500, 505-510, 522-532, 537-554, 564-570, 573-574, 577-601, 620-621, 627, 651, 692-695, 719-736, 795-797, 864-873, 877-886, 889-899, 903, 908-909, 929-938, 941, 953, 982-987, 991-997, 1066-1172
redis_benchmarks_specification/__self_contained_coordinator__/__init__.py                         0      0   100%
redis_benchmarks_specification/__self_contained_coordinator__/args.py                            34     31     9%   25-153
redis_benchmarks_specification/__self_contained_coordinator__/artifacts.py                       12      0   100%
redis_benchmarks_specification/__self_contained_coordinator__/build_info.py                      41      3    93%   21, 37, 42
redis_benchmarks_specification/__self_contained_coordinator__/clients.py                          8      0   100%
redis_benchmarks_specification/__self_contained_coordinator__/cpuset.py                          11      0   100%
redis_benchmarks_specification/__self_contained_coordinator__/docker.py                          27     14    48%   37-46, 59-90
redis_benchmarks_specification/__self_contained_coordinator__/prepopulation.py                   22     14    36%   28-86
redis_benchmarks_specification/__self_contained_coordinator__/runners.py                        195    158    19%   88-89, 120-621
redis_benchmarks_specification/__self_contained_coordinator__/self_contained_coordinator.py     333    137    59%   98-252, 315, 351, 359, 385, 440-448, 452, 463-468, 529-538, 620, 751-831, 838, 902-920, 935-945, 957-973, 1055-1060
redis_benchmarks_specification/__setups__/__init__.py                                             0      0   100%
redis_benchmarks_specification/__setups__/topologies.py                                          13      0   100%
---------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                          2234    967    57%
integration-tests docker: remove 'd6f48015f9' (from 'datasink')
integration-tests docker: remove '4ec0386d4c' (from 'db_server')
___________________________________________________________________________________________ summary ____________________________________________________________________________________________
  integration-tests: commands succeeded
  congratulations :)

[odidev]

Closing this ticket since the redislabs/memtier-benchmark:edge docker image is now available for multi-arch platform at docker hub, and the above issue got resolved.