[Bug] Remote socket error-path UT races connect against peer RST

[doraemonmj]

Platform

All / Unknown

Runtime Variant

All / Unknown

Description

RemoteSocketTransport.ProgressErrorClearsActiveCommand intermittently fails before reaching the behavior it intends to test. Its helper server accepts the transport connection and immediately closes it with SO_LINGER={1,0}, so the peer RST races the client's nonblocking connect completion.

When connect wins, transport construction succeeds and the test observes the socket error through poll_progress_reply(). When the RST wins, RemoteL3SocketTransport construction itself throws Connection reset by peer, so the active-command cleanup assertion is never exercised.

This is a test synchronization defect rather than a product transport failure: a real peer reset during connect should continue to propagate as a connection error.

Steps to Reproduce

1. Configure and build the no-hardware C++ unit tests:

   cmake -B tests/ut/cpp/build -S tests/ut/cpp
   cmake --build tests/ut/cpp/build -j4
   

2. Repeatedly run the target test, preferably while the host is under concurrent CI load:

   for i in $(seq 1 1000); do
     tests/ut/cpp/build/test_remote_endpoint \
       --gtest_filter=RemoteSocketTransport.ProgressErrorClearsActiveCommand || break
   done
   

3. Alternatively, inspect PR Feature: enable SDMA and URMA in the default A5 runtime #2026 run 33046351050, Ubuntu UT job 98431725257.

Expected Behavior

The helper should deterministically allow RemoteL3SocketTransport construction to complete, then inject the RST. The test should observe an error from poll_progress_reply(), verify that the active command was cleared, and verify that another progress frame can be submitted.

Actual Behavior

The RST can arrive before nonblocking connect completion is confirmed, causing the constructor to throw before the intended assertions:

C++ exception with description
"RemoteL3SocketTransport(command): connect failed to 127.0.0.1:43777: Connection reset by peer"

In the observed CI run, all Python UTs passed and 121 of 122 C++ tests passed. The same test and product sources passed in the preceding CI run with an identical functional tree.

Git Commit ID

ae940ac

CANN Version

N/A — pure no-hardware C++ socket unit test.

Driver Version

N/A — pure no-hardware C++ socket unit test.

Host Platform

Linux (x86_64)

Additional Context

Failing job: https://github.com/hw-native-sys/simpler/actions/runs/33046351050/job/98431725257

The race is in tests/ut/cpp/hierarchical/test_remote_endpoint.cpp: start_closing_server() accepts and resets immediately, while ProgressErrorClearsActiveCommand assumes transport construction wins that race. The fix should use an explicit accepted/close-now handshake so the test triggers RST only after construction, instead of adding a product connect retry or relying on a fixed sleep.