Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workaround for random kernel panic in github runner #52

Merged
merged 8 commits into from
Feb 24, 2025
Merged

Workaround for random kernel panic in github runner #52

merged 8 commits into from
Feb 24, 2025

Conversation

nirs
Copy link
Owner

@nirs nirs commented Feb 23, 2025

Add support for setting kernel command line parameters and add the workaround from socket_vment.
lima-vm/socket_vmnet#56

Also add apic=debug to get more info on that may help to resolve the qemu issue:
https://gitlab.com/qemu-project/qemu/-/issues/2832

Fixes #34

@nirs nirs force-pushed the debug branch 9 times, most recently from 3938b24 to 2fb267c Compare February 23, 2025 21:53
nirs added 4 commits February 24, 2025 00:19
Add new step for inspecting guest data, logging `uname -a` and
/proc/cmdline. Needed to make configurable kernel command line.
Change ssh log level to ERROR to silence warnings like:

    Warning: Permanently added '192.168.105.2' (ED25519) to the list of
    known hosts.
Use a dict with "image" key to specify the image url. This allows adding
related images such as kernel and initrd images. Use same format to
return the vm disk image.
Use disk["image"] to limit the number of temporary variable, since we
need to add similar code to create kernel and initrd images.

Make create_image() more generic so we can reuse it for creating kernel
and initrd images that do not need any conversion.

Extract clone() helper to make the code nicer to work with. We will need
to clone also the kernel and initrd images.
@nirs nirs force-pushed the debug branch 4 times, most recently from c573b33 to 8a8ffd7 Compare February 23, 2025 23:28
Add "kernel", "kernel_parameters", and "initrd" properties to image spec.
If kernel and initrd are specified the urls are downloaded to the cache
and the data is cloned to the vm directory. If kernel_parameters is
specified it is copied to the disk dict.

If the optional disk properties are specified we use them in the qemu
command. This will be used to debug the random kernel panics in the
github runner.

We can use these properties later for vfkit and krunkit it needed.
@nirs nirs force-pushed the debug branch 2 times, most recently from 4b352d2 to 57f7747 Compare February 23, 2025 23:48
To debug the random kernel panic on the github runner we need to add the
apic=debug kernel command line parameter.

    [    0.020000] Kernel panic - not syncing: IO-APIC + timer doesn't
    work!  Boot with apic=debug and send a report.  Then try booting
    with the 'noapic' option.

Trying first to boot with kernel and initrd and the default command line
parameters.
@nirs nirs force-pushed the debug branch 2 times, most recently from d22d5b7 to 3783795 Compare February 24, 2025 00:20
@nirs nirs changed the title Suport specifying kernel command line arguments Workaround for random kernel panic in github runner Feb 24, 2025
@nirs nirs marked this pull request as ready for review February 24, 2025 00:23
@nirs nirs merged commit 155393a into main Feb 24, 2025
5 checks passed
@nirs nirs deleted the debug branch February 24, 2025 00:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flaky qemu job:
1 participant