ExecCmds="Automation" does not support queueing several Run requests.
Either RunFilter, RunAll or RunTest should be used.
You can set filter additionally to a Run request to further narrow down the test collection.
Also -testexit= is legacy and is not the best way to exit the editor after executing an Automation command.
Instead use:
-ExecCmds="Automation RunFilter Smoke;Quit"
Repro Rate: 3/3
This issue was reproduced in //UE5/Release-5.1 at CL 23058290.
This is confirmed to not be a regression since the error also occurs on the previous live version. This is regression tested on //UE5/Release-5.0 at CL 20979098.
There seems to be something wrong with not finding any workers. Check the attached log for more details. Running the command only with a single "Automation RunTests Project" or "Automation RunFilter Smoke" works perfectly fine, the issue only occurs when combining the two in a single command.
Steps:
Actual Result: The command is stuck in an endless loop and never starts any tests.
Expected Result: The Automation tool runs all Tests that match the "Project" Filter and then all tests that match the Smoke Filter and then exits.
There's no existing public thread on this issue, so head over to Questions & Answers just mention UE-170942 in the post.