rockpro64: sel4test freezes at BIND0001 when SMP is enabled

[canarysnort01]

When I run sel4test with SMP enabled on my pinebook pro, sel4test freezes while running BIND0001.

I found this issue while investigating seL4/seL4#184. See that issue for further analysis. This does not appear to be a kernel issue.

Steps to reproduce:

mkdir sel4test
cd sel4test
repo init -u https://github.com/seL4/sel4test-manifest.git -b master
repo sync
cd kernel
git checkout master
cd ..
mkdir build
cd build
../init-build.sh -DPLATFORM=rockpro64 -DSIMULATION=false -DSMP=on -DNUM_NODES=4 -DLibUtilsDefaultZfLogLevel=0
ninja
cd images
../../tools/seL4/cmake-tool/helpers/make-uimage /usr/bin/aarch64-linux-gnu-objcopy ../elfloader/elfloader arm64 sel4test-driver-image-arm-rockpro64-uboot

When run on the pinebook pro (rockpro64 platform), this resulted in a freeze while running this test:

Starting test 18: BIND0001
[email protected]:383 Starting process at 0x4001c8, stack 0x10011df0

Here's the full serial output for the test that failed: serial3.txt

[xurtis]

Can confirm the issue, managed to replicate internally. Thanks for reporting. I'll have a poke around and see if I can tell exactly how it's getting stuck.

[xurtis]

This appears to be an issue with the platform timer implementation. It seems that the timer doesn't get correctly cleared so the driver spins handling timer interrupts preventing the test thread from progressing.

[xurtis]

Well, I am genuinely astonished by this issue. For some reason, only on the SMP configuration of the rockpro64, the capability slot that is configured for the second timer IRQ handler of the platform timer somehow gets turned into a capability referring to the associated notification.

When the timer is acknowledged, rather than acknowledging and unmasking the IRQ, the notification is signalled as though the IRQ has occurred again.

[yyshen]

Well, I am genuinely astonished by this issue. For some reason, only on the SMP configuration of the rockpro64, the capability slot that is configured for the second timer IRQ handler of the platform timer somehow gets turned into a capability referring to the associated notification.

When the timer is acknowledged, rather than acknowledging and unmasking the IRQ, the notification is signalled as though the IRQ has occurred again.

How did this happen? I assume it is sel4test code? was the change introduced recently?

[xurtis]

The root of the issue is that the driver implementation for the rockpro64 timer is incorrect. I suspect the intent is for the secondary timer to get the IRQ after the first timer but the number it uses is the ID of the IRQ handler in the environment, not the IRQ number itself. At that point in the code, there is presently no way to determine which IRQ number was actually used for the first timer.

Additionally, this error presents the way it does due to the allocation error being ignored.

Suggested fixes should be to correct the rockpro64 timer so that it requests the correct IRQ for the secondary timer (this may involve modifying a few chunks of platsupport so that the IRQ found in the FDT is actually passed back to the caller that requested it) and for the error to allocate an IRQ to be treated as fatal.

[xurtis]

How did this happen?

The allocation of the IRQ handler fails but is ignored. The notification for the IRQ then gets created and ends up in the slot that would have been used for the IRQ handler.

[yyshen]

How did this happen?

The allocation of the IRQ handler fails but is ignored. The notification for the IRQ then gets created and ends up in the slot that would have been used for the IRQ handler.

Thanks for the explanation.