SEOS-2956: SCHED0021 test in native sel4test suite fails for the MiG-V 1.0 Evaluation Board

[FelixSchladt]

SCHED0021 test in native sel4test suite fails for the MiG-V 1.0 Evaluation Board

The SCHED0021 test in the native sel4test suite currently shows some non-deterministic behavior, as it either succeeds or fails "randomly". In case of a failure, the following output is produced:
{code:java}
[email protected]:1619 Finished in 53656us
Error: Check duration(53656756) < ((((2 * 5 * (1000UL * 1000UL)) * (4 + 1)) * (102)) / (100))(51000000) failed. at line 1624 of file /host/src/src/native/sel4test/src/apps/sel4test-tests/src/tests/scheduler.c
Test SCHED0021 failed
Failure: result == SUCCESS at line 291 of file /host/src/src/native/sel4test/src/apps/sel4test-driver/src/testtypes.c
Error: result == SUCCESS at line 219 of file /host/src/src/native/sel4test/src/apps/sel4test-driver/src/main.c
Starting test 3: Test all tests ran
Test suite failed. 2/3 tests passed.
*** FAILURES DETECTED ***
{code}
The exact time (here: 53656us) can vary between different runs.

Created by

sebastian eckl
Created: Friday, July 02, 2021 09:31 AM UTC+02:00
Updated: Thursday, August 12, 2021 03:12 PM UTC+02:00

Comments

sebastian eckl

In a first run, the test was executed 10 times, whereas the test succeeded 8 times and failed 2 times. In case of success, the duration varied between 36324us - 38160us. In case of a failure, the duration varied between 76041us - 76278us.

Further investigation revealed that within the test (src/native/sel4test/src/apps/sel4test-tests/src/tests/scheduler.c), seL4_Yield() is called, in order to cause all other threads of the test to execute before returning to the main thread again. Measurements showed that the seL4_Yield() call itself takes around 36us, the other threads execute very fast, making up only a negligible portion of the overall duration.

We therefore currently assume that the test suffers from a race condition, which could lead to calling seL4_Yield() two times. This would explain the duration in case of a failure, which is almost double the size of the duration in case of success.

created: Friday, July 02, 2021 09:44 AM UTC+02:00
updated: Friday, July 02, 2021 09:46 AM UTC+02:00

axel heider

Seems a well know issues that SCHED0021 has some problem in some environments, see also seL4/ci-actions#112

created: Friday, July 09, 2021 12:08 PM UTC+02:00
updated: Friday, July 09, 2021 12:10 PM UTC+02:00

sebastian eckl

Based on the latest rebase of SEOS-829 onto integration and merging parts of SEOS-2800, the SCHED0021 test always runs successfully when being executed alone, without executing the rest of the sel4test suite. In combination with the well-known issues mentioned by [~hg123912], we might close this ticket for now.

created: Thursday, July 29, 2021 08:42 PM UTC+02:00
updated: Thursday, July 29, 2021 08:42 PM UTC+02:00

Thomas Wittal

not fixed, external github issue linked

created: Thursday, August 12, 2021 03:12 PM UTC+02:00
updated: Thursday, August 12, 2021 03:12 PM UTC+02:00

Subtasks