testcases: Reorder remoteproc tests to fix AudioRecord dependency

[qcom-anilyada]

Move cdsp_remoteproc and adsp_remoteproc tests to run after AudioRecord test to prevent service dependency conflicts. The remoteproc tests hold services that AudioRecord waits for, causing AudioRecord to hang on iq-9075-evk when remoteproc runs first.

Remove AudioRecord from iq-9075-evk exclusion list as this reordering resolves the test failure, allowing AudioRecord to pass successfully.

Signed-off-by: Anil Yadav anilyada@qti.qualcomm.com

[qcom-anilyada]

Failure before the shuffle on two different targets:

• https://lava.infra.foundries.io/scheduler/job/143672 [hyd-worker-11]
• https://lava.infra.foundries.io/scheduler/job/143656 [hyd-worker-17]

PASSED job after changing the order: https://lava.infra.foundries.io/scheduler/job/144019 [hyd-worker-11]

if this gets merged, we can close the PR#1566

[lumag]

You are basically stating that things freez, let's stop testing it. Waht is the reason for the freeze?

[mattface]

I also tried moving the audiorecord test into a different test action so I could give it a sensible timeout, and then was surprised that all the tests passed LAVA Job 143047
I agree with @lumag that altering the tests to make them pass means the original failure will not get correctly triaged.

How about keeping the tests in the original order, but putting each test suite into a separate test action so that each test suite can have a sensible timeout. I believe AudioRecord only takes a few seconds to successfully run.
This will stop our LAVA jobs getting stuck for a long period of time, whilst also showing the failing tests as a failure to be investigated.

[qcom-anilyada]

We raised this to unblock PR testing, which was getting stalled due to a single testcase running for 4 hours. Our analysis showed the issue was not build-related but caused by a testkit script problem in the remoteproc testcase, so the tests were temporarily shuffled until it is fixed.

So, we can either wait for the below PR to get merged and getting timed out, or merge this. Anyway, now we know what was causing this and provided fix for that.

The below issue and PR have RCA and fix for this issue:

• qcom-linux-testkit/pull/295
• qcom-linux-testkit/issues/291

[lumag]

We raised this to unblock PR testing, which was getting stalled due to a single testcase running for 4 hours. Our analysis showed the issue was not build-related but caused by a testkit script problem in the remoteproc testcase, so the tests were temporarily shuffled until it is fixed.

So, we can either wait for the below PR to get merged and getting timed out, or merge this. Anyway, now we know what was causing this and provided fix for that.

The below issue and PR have RCA and fix for this issue:

• qcom-linux-testkit/pull/295
• qcom-linux-testkit/issues/291

If the issue is on the AudioReach side, my preference would be to drop AudioReach from qcom-distro rather than disabling the tests. Please send corresponding change.

[mattface]

My suggestion would have unblocked the tests. We can make each test case have a sane timeout, meaning the hanging test would have timed out in a reasonable time and the job queue would have continued.

Or, lowering the massive 4 hours timeout for all the tests which usually take around 4 minutes to something sane would also have fixed the blocking.

[mwasilew]

@mattface the timeout PR is #15.
As for the original issue, it's either remoteproc test or the feature and it should be fixed. I can see there is a discussion in qcom-linux-testkit repository. IMHO this PR should be dropped.

[qcom-anilyada]

#15 is merged, which will resolve the issue without any reordering, for which this PR was raised.
Closing this PR.