Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Retry commands prints an error after success #2253

Open
1 task done
Fishbowler opened this issue Jan 17, 2025 · 1 comment
Open
1 task done

Retry commands prints an error after success #2253

Fishbowler opened this issue Jan 17, 2025 · 1 comment

Comments

@Fishbowler
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues and didn't find mine.

Steps to reproduce

This flow passes. It fails its first 2 iterations on the retry loop, then succeeds on the 3rd. But prints the error from the 2nd loop to stderr.

appId: com.example.notUsed
jsEngine: graaljs
---
- evalScript: ${output.number = 0}
- retry:
    maxRetries: 3
    commands:
        - evalScript: ${output.number++}
        - assertTrue: ${output.number == 3}
- assertTrue:
    condition: ${true}
    label: "Reached the end!"

Actual results

Screenshot:

Image

Plain text version:

Running on emulator-5554                                                        
                                                                                
 ║                                                                              
 ║  > Flow: flow                                                                
 ║                                                                              
 ║    ✅   Run ${output.number = 0}                                             
 ║    ✅   Retry 3 times                                                        
 ║         Warning:                                                             
 ║           Retrying the commands due to an error: Assertion is false: false is
 ║           true while execution (Attempt 2)                                   
 ║    ✅   Reached the end!                                                     
 ║                                                                              
                                                                                
Assertion is false: false is true

Expected results

Everything but that last line.

About app

No app used.

About environment

macOS 15.2, Maestro @ main, near 1.39.9

Logs

Logs 
09:47:07.327 [ INFO] MAESTRO.logSystemInfo: ---- System Info ----
09:47:07.328 [ INFO] MAESTRO.logSystemInfo: Maestro Version: Undefined
09:47:07.329 [ INFO] MAESTRO.logSystemInfo: CI: Undefined
09:47:07.329 [ INFO] MAESTRO.logSystemInfo: OS Name: Mac OS X
09:47:07.329 [ INFO] MAESTRO.logSystemInfo: OS Version: 15.2
09:47:07.329 [ INFO] MAESTRO.logSystemInfo: Architecture: aarch64
09:47:07.329 [ INFO] MAESTRO.logSystemInfo: Java Version: 17
09:47:07.612 [ INFO] MAESTRO.logSystemInfo: Xcode Version: 16.2
09:47:07.614 [ INFO] MAESTRO.logSystemInfo: Flutter Version: Undefined
09:47:07.617 [ INFO] MAESTRO.logSystemInfo: Flutter Channel: Undefined
09:47:07.617 [ INFO] MAESTRO.logSystemInfo: ---------------------
09:47:08.715 [ INFO] maestro.cli.command.TestCommand.runShardSuite: [shard 1] Selected device emulator-5554 using port 7001
09:47:09.158 [DEBUG] io.micrometer.common.util.internal.logging.InternalLoggerFactory.newDefaultFactory: Using SLF4J as the default logging framework
09:47:10.886 [ INFO] maestro.Maestro.invoke: Getting device info
09:47:11.090 [ INFO] maestro.Maestro.invoke: Got device info: DeviceInfo(platform=ANDROID, widthPixels=1080, heightPixels=2400, widthGrid=1080, heightGrid=2400)
09:47:11.178 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Define variables RUNNING
09:47:11.180 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Define variables metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(defineVariablesCommand=DefineVariablesCommand(env={MAESTRO_OTH_AUTH_TOKEN=WXNUY3ZsLTJJRU1DY1pmcEZVbU00XzBaRS1tYlhxZ2Y6dUstdFJQYUh1WHNVUlFmYlhGOFNON3FPWTJfN1l2d09KeFRuVHE3OVN1UjU4RU9yX1cyaURPSy1HY3l0Ym4xeA==, MAESTRO_API_KEY=5a18a2df-a835-40b3-bee5-a5c0a8425c48, MAESTRO_FILENAME=flow}, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:11.181 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Define variables COMPLETED
09:47:11.181 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Apply configuration RUNNING
09:47:11.182 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Apply configuration metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(applyConfigurationCommand=ApplyConfigurationCommand(config=MaestroConfig(appId=com.example.notUsed, name=null, tags=[], ext={jsEngine=graaljs}, onFlowStart=null, onFlowComplete=null), label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:11.183 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Apply configuration COMPLETED
09:47:11.183 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number = 0} RUNNING
09:47:11.184 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number = 0} metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(evalScriptCommand=EvalScriptCommand(scriptString=${output.number = 0}, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:11.530 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number = 0} COMPLETED
09:47:11.530 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Retry 3 times RUNNING
09:47:11.531 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Retry 3 times metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(retryCommand=RetryCommand(maxRetries=3, commands=[MaestroCommand(evalScriptCommand=EvalScriptCommand(scriptString=${output.number++}, label=null, optional=false)), MaestroCommand(assertConditionCommand=AssertConditionCommand(condition=Condition(platform=null, visible=null, notVisible=null, scriptCondition=${output.number == 3}, label=null), timeout=null, label=null, optional=false))], config=null, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:11.531 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number++} RUNNING
09:47:11.531 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number++} metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(evalScriptCommand=EvalScriptCommand(scriptString=${output.number++}, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:11.560 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number++} COMPLETED
09:47:11.561 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Assert that ${output.number == 3} is true RUNNING
09:47:11.573 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Assert that ${output.number == 3} is true metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(assertConditionCommand=AssertConditionCommand(condition=Condition(platform=null, visible=null, notVisible=null, scriptCondition=false, label=null), timeout=null, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:11.785 [ INFO] maestro.Maestro.takeScreenshot: Taking screenshot
09:47:11.786 [TRACE] maestro.utils.ScreenshotUtils.takeScreenshot: Taking screenshot to output sink
09:47:12.489 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Assert that ${output.number == 3} is true FAILED
09:47:12.491 [ERROR] maestro.orchestra.Orchestra.retryCommand: Attempt 1 failed for retry command
maestro.MaestroException$AssertionFailure: Assertion is false: false is true
	at maestro.orchestra.Orchestra.assertConditionCommand(Orchestra.kt:345)
	at maestro.orchestra.Orchestra.executeCommand(Orchestra.kt:275)
	at maestro.orchestra.Orchestra.executeSubflowCommands(Orchestra.kt:733)
	at maestro.orchestra.Orchestra.runSubFlow(Orchestra.kt:785)
	at maestro.orchestra.Orchestra.retryCommand(Orchestra.kt:593)
	at maestro.orchestra.Orchestra.executeCommand(Orchestra.kt:304)
	at maestro.orchestra.Orchestra.executeCommands(Orchestra.kt:196)
	at maestro.orchestra.Orchestra.runFlow(Orchestra.kt:130)
	at maestro.cli.runner.MaestroCommandRunner.runCommands(MaestroCommandRunner.kt:179)
	at maestro.cli.runner.TestRunner$runSingle$result$1.invoke(TestRunner.kt:62)
	at maestro.cli.runner.TestRunner$runSingle$result$1.invoke(TestRunner.kt:53)
	at maestro.cli.runner.TestRunner.runCatching(TestRunner.kt:160)
	at maestro.cli.runner.TestRunner.runSingle(TestRunner.kt:53)
	at maestro.cli.command.TestCommand.runSingleFlow(TestCommand.kt:356)
	at maestro.cli.command.TestCommand.access$runSingleFlow(TestCommand.kt:64)
	at maestro.cli.command.TestCommand$runShardSuite$2.invoke(TestCommand.kt:330)
	at maestro.cli.command.TestCommand$runShardSuite$2.invoke(TestCommand.kt:301)
	at maestro.cli.session.MaestroSessionManager.newSession(MaestroSessionManager.kt:103)
	at maestro.cli.session.MaestroSessionManager.newSession$default(MaestroSessionManager.kt:53)
	at maestro.cli.command.TestCommand.runShardSuite(TestCommand.kt:301)
	at maestro.cli.command.TestCommand.access$runShardSuite(TestCommand.kt:64)
	at maestro.cli.command.TestCommand$handleSessions$1$results$1$1.invokeSuspend(TestCommand.kt:269)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:585)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693)
09:47:12.492 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Retry 3 times metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(retryCommand=RetryCommand(maxRetries=3, commands=[MaestroCommand(evalScriptCommand=EvalScriptCommand(scriptString=${output.number++}, label=null, optional=false)), MaestroCommand(assertConditionCommand=AssertConditionCommand(condition=Condition(platform=null, visible=null, notVisible=null, scriptCondition=${output.number == 3}, label=null), timeout=null, label=null, optional=false))], config=null, label=null, optional=false)), logMessages=[], insight=Insight(message=Retrying the commands due to an error: Assertion is false: false is true while execution (Attempt 1), level=WARNING))
09:47:12.492 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number++} RUNNING
09:47:12.493 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number++} metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(evalScriptCommand=EvalScriptCommand(scriptString=${output.number++}, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:12.503 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number++} COMPLETED
09:47:12.503 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Assert that ${output.number == 3} is true RUNNING
09:47:12.511 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Assert that ${output.number == 3} is true metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(assertConditionCommand=AssertConditionCommand(condition=Condition(platform=null, visible=null, notVisible=null, scriptCondition=false, label=null), timeout=null, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:12.756 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Assert that ${output.number == 3} is true FAILED
09:47:12.757 [ERROR] maestro.orchestra.Orchestra.retryCommand: Attempt 2 failed for retry command
maestro.MaestroException$AssertionFailure: Assertion is false: false is true
	at maestro.orchestra.Orchestra.assertConditionCommand(Orchestra.kt:345)
	at maestro.orchestra.Orchestra.executeCommand(Orchestra.kt:275)
	at maestro.orchestra.Orchestra.executeSubflowCommands(Orchestra.kt:733)
	at maestro.orchestra.Orchestra.runSubFlow(Orchestra.kt:785)
	at maestro.orchestra.Orchestra.retryCommand(Orchestra.kt:593)
	at maestro.orchestra.Orchestra.executeCommand(Orchestra.kt:304)
	at maestro.orchestra.Orchestra.executeCommands(Orchestra.kt:196)
	at maestro.orchestra.Orchestra.runFlow(Orchestra.kt:130)
	at maestro.cli.runner.MaestroCommandRunner.runCommands(MaestroCommandRunner.kt:179)
	at maestro.cli.runner.TestRunner$runSingle$result$1.invoke(TestRunner.kt:62)
	at maestro.cli.runner.TestRunner$runSingle$result$1.invoke(TestRunner.kt:53)
	at maestro.cli.runner.TestRunner.runCatching(TestRunner.kt:160)
	at maestro.cli.runner.TestRunner.runSingle(TestRunner.kt:53)
	at maestro.cli.command.TestCommand.runSingleFlow(TestCommand.kt:356)
	at maestro.cli.command.TestCommand.access$runSingleFlow(TestCommand.kt:64)
	at maestro.cli.command.TestCommand$runShardSuite$2.invoke(TestCommand.kt:330)
	at maestro.cli.command.TestCommand$runShardSuite$2.invoke(TestCommand.kt:301)
	at maestro.cli.session.MaestroSessionManager.newSession(MaestroSessionManager.kt:103)
	at maestro.cli.session.MaestroSessionManager.newSession$default(MaestroSessionManager.kt:53)
	at maestro.cli.command.TestCommand.runShardSuite(TestCommand.kt:301)
	at maestro.cli.command.TestCommand.access$runShardSuite(TestCommand.kt:64)
	at maestro.cli.command.TestCommand$handleSessions$1$results$1$1.invokeSuspend(TestCommand.kt:269)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:585)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693)
09:47:12.758 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Retry 3 times metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(retryCommand=RetryCommand(maxRetries=3, commands=[MaestroCommand(evalScriptCommand=EvalScriptCommand(scriptString=${output.number++}, label=null, optional=false)), MaestroCommand(assertConditionCommand=AssertConditionCommand(condition=Condition(platform=null, visible=null, notVisible=null, scriptCondition=${output.number == 3}, label=null), timeout=null, label=null, optional=false))], config=null, label=null, optional=false)), logMessages=[], insight=Insight(message=Retrying the commands due to an error: Assertion is false: false is true while execution (Attempt 2), level=WARNING))
09:47:12.758 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number++} RUNNING
09:47:12.759 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number++} metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(evalScriptCommand=EvalScriptCommand(scriptString=${output.number++}, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:12.765 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Run ${output.number++} COMPLETED
09:47:12.766 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Assert that ${output.number == 3} is true RUNNING
09:47:12.771 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Assert that ${output.number == 3} is true metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(assertConditionCommand=AssertConditionCommand(condition=Condition(platform=null, visible=null, notVisible=null, scriptCondition=true, label=null), timeout=null, label=null, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:12.774 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Assert that ${output.number == 3} is true COMPLETED
09:47:12.775 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Retry 3 times COMPLETED
09:47:12.776 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Reached the end! RUNNING
09:47:12.783 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Reached the end! metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(assertConditionCommand=AssertConditionCommand(condition=Condition(platform=null, visible=null, notVisible=null, scriptCondition=true, label=null), timeout=null, label=Reached the end!, optional=false)), logMessages=[], insight=Insight(message=, level=NONE))
09:47:12.783 [ INFO] maestro.cli.runner.MaestroCommandRunner.invoke: Reached the end! COMPLETED

Maestro version

1.39.9ish

How did you install Maestro?

built from source (please include commit hash in the text area below)

Anything else?

No response
@linear Linear
Copy link

linear bot commented Jan 17, 2025

MA-2940 Retry commands prints an error after success

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Fishbowler