Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kraft rm --all ends up with an error #820

Open
razvand opened this issue Sep 16, 2023 · 7 comments
Open

kraft rm --all ends up with an error #820

razvand opened this issue Sep 16, 2023 · 7 comments
Assignees
@craciunoiuc
Labels
kind/bug Something isn't working priority/low Nice to have issues thatare not urgent, but can be nagging.

Comments

@razvand
Copy link
Contributor

razvand commented Sep 16, 2023

Describe the bug

After doing KraftKit testing, kraft rm --all end up with an error.

Steps to reproduce

I used KraftKit and, then, at some point I wanted to remove the created VMs. I used a normal account to remove those created by the privileged account.

I'm using Ubuntu 22.04. KraftKit version kraft 0.6.7 (b139ae2a52608f2e4a910225be0c981f671fe75f) go1.20.6 2023-09-14T09:00:05Z.

Expected behavior

kraft rm --all should remove all Unikernel images / VMs, not end up in error.

Which architectures were you using or does this bug affect?

x86_64

Which operating system were you using or does this bug affect?

linux/debian

Relevant log output

razvan@yggdrasil:~/projects/unikraft/maintainer-tools/workdir/apps/elfloader$ sudo kraft ps
NAME                      KERNEL                                 ARGS                                                                                 CREATED     STATUS  MEM   PLAT
agitated_joemartin        project://elfloader:kraftcloud-x86_64  /bin/python3.11 -m http.server 8080                                                  2 days ago  exited  0MiB  fc/x86_64
agitated_lucy             project://elfloader:kraftcloud-x86_64  /python -m http.server 8080                                                          1 day ago   exited  0MiB  fc/x86_64
charming_faben            project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
charming_frodo            project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
crazy_edgar               project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 http_server.py       2 days ago  exited  0MiB  fc/x86_64
eager_ayumu               project://elfloader:kraftcloud-x86_64  /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf                      2 days ago  exited  0MiB  fc/x86_64
elegant_loon              project://elfloader:kraftcloud-x86_64  /python http_server.py                                                               1 day ago   exited  0MiB  fc/x86_64
exciting_timmy            project://elfloader:kraftcloud-x86_64  /python -m http.server 8080                                                          1 day ago   exited  0MiB  fc/x86_64
focused_ahmeng            project://elfloader:kraftcloud-x86_64  /http_server                                                                         2 days ago  exited  0MiB  fc/x86_64
frosty_azy                project://elfloader:kraftcloud-x86_64  /http_server                                                                         1 day ago   exited  0MiB  fc/x86_64
funny_crystal             project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 http_server.py       2 days ago  exited  0MiB  fc/x86_64
gifted_sultan             project://elfloader:kraftcloud-x86_64  /http_server                                                                         2 days ago  exited  0MiB  fc/x86_64
gracious_bintijua         project://elfloader:kraftcloud-x86_64  /http_server                                                                         2 days ago  exited  0MiB  fc/x86_64
interesting_sultan        project://elfloader:kraftcloud-x86_64  /http_server                                                                         1 day ago   exited  0MiB  fc/x86_64
jolly_fanni               project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 --version            2 days ago  exited  0MiB  fc/x86_64
musing_ahmeng             project://elfloader:kraftcloud-x86_64  /http_server                                                                         2 days ago  exited  0MiB  fc/x86_64
musing_gargantua          project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
musing_panbanisha         project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  1 day ago   exited  0MiB  fc/x86_64
objective_davidgreybeard  project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
pedantic_washoe           project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
priceless_littlemama      project://elfloader:kraftcloud-x86_64  /python -m http.server 8080                                                          1 day ago   exited  0MiB  fc/x86_64
upbeat_crystal            project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11                      2 days ago  exited  0MiB  fc/x86_64
vigorous_ramu             project://elfloader:kraftcloud-x86_64  /python http_server.py                                                               1 day ago   exited  0MiB  fc/x86_64
xenodochial_jaccomacacco  project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  1 day ago   exited  0MiB  fc/x86_64
youthful_viki             project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 http_server.py       2 days ago  exited  0MiB  fc/x86_64
zealous_flint             project://elfloader:kraftcloud-x86_64  /http_server                                                                         1 day ago   exited  0MiB  fc/x86_64


razvan@yggdrasil:~/projects/unikraft/maintainer-tools/workdir/apps/elfloader$ sudo kraft rm --all
panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
kraftkit.sh/machine/network/bridge.(*v1alpha1Network).Get(0x0?, {0x0?, 0x0?}, 0xc000619680)
        /__w/kraftkit/kraftkit/machine/network/bridge/v1alpha1.go:450 +0x378
api%2ezip.(*MethodClient[...]).Do(0x1b22580, {0x1b2e308, 0xc00033ad80}, 0xc000619680)
        /go/pkg/mod/[email protected]/client.go:49 +0xe8
kraftkit.sh/api/network/v1alpha1.(*NetworkServiceHandler).Get(0x163fe00?, {0x1b2e308?, 0xc00033ad80?}, 0x6?)
        /__w/kraftkit/kraftkit/api/network/v1alpha1/network.zip.go:151 +0x2c
kraftkit.sh/cmd/kraft/rm.(*Rm).Run(0xc0000117d0, 0x0?, {0xc000120c10, 0x0, 0x0?})
        /__w/kraftkit/kraftkit/cmd/kraft/rm/rm.go:143 +0x1068
kraftkit.sh/cmdfactory.bind.func1(0xc000172000?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc000172000?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc000172000?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0x0?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc00059bd80?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.execute(0xc000172000, {0xc0007a1120, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:231 +0x30b
kraftkit.sh/cmdfactory.executeC(0xc000004600)
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:291 +0x205
kraftkit.sh/cmdfactory.Main({0x1b2e308?, 0xc00033ad80}, 0xc000004600)
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:322 +0x73
main.main()
        /__w/kraftkit/kraftkit/cmd/kraft/kraft.go:153 +0x549
@razvand razvand added the kind/bug Something isn't working label Sep 16, 2023
@github-project-automation github-project-automation bot moved this to 🧊 Icebox in KraftKit Roadmap Sep 16, 2023
@razvand
Copy link
Contributor Author

razvand commented Sep 16, 2023

I also get this error when using kraft rm:

razvan@yggdrasil:~/projects/unikraft/maintainer-tools/workdir/apps/elfloader$ sudo kraft rm agitated_joemartin
panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
kraftkit.sh/machine/network/bridge.(*v1alpha1Network).Get(0x0?, {0x0?, 0x0?}, 0xc000577440)
        /__w/kraftkit/kraftkit/machine/network/bridge/v1alpha1.go:450 +0x378
api%2ezip.(*MethodClient[...]).Do(0x1b22580, {0x1b2e308, 0xc0003cc000}, 0xc000577440)
        /go/pkg/mod/[email protected]/client.go:49 +0xe8
kraftkit.sh/api/network/v1alpha1.(*NetworkServiceHandler).Get(0x163fe00?, {0x1b2e308?, 0xc0003cc000?}, 0x6?)
        /__w/kraftkit/kraftkit/api/network/v1alpha1/network.zip.go:151 +0x2c
kraftkit.sh/cmd/kraft/rm.(*Rm).Run(0xc000483800, 0x0?, {0xc0004a3620, 0x1, 0x0?})
        /__w/kraftkit/kraftkit/cmd/kraft/rm/rm.go:143 +0x1068
kraftkit.sh/cmdfactory.bind.func1(0xc0007b8000?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc0007b8000?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc0007b8000?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0x0?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc0008b9d80?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.execute(0xc0007b8000, {0xc00033daa0, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:231 +0x30b
kraftkit.sh/cmdfactory.executeC(0xc000004600)
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:291 +0x205
kraftkit.sh/cmdfactory.Main({0x1b2e308?, 0xc0003cc000}, 0xc000004600)
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:322 +0x73
main.main()
        /__w/kraftkit/kraftkit/cmd/kraft/kraft.go:153 +0x549

@razvand
Copy link
Contributor Author

razvand commented Sep 17, 2023

I may have caused this by doing the following:

  1. Create a network interface with:

    sudo kraft net create ...

  2. Create a Unikraft instance:

    sudo kraft run ...

  3. Remove the network interface with:

    sudo ip link set dev kraft0 down
sudo ip link del dev kraft0

  4. Run the kraft rm command:

    sudo kraft rm --all

@craciunoiuc craciunoiuc self-assigned this Sep 20, 2023
@craciunoiuc
Copy link
Member

Trying to remove a used interface is protected again in kraft thankfully

➜  app-nginx git:(staging) ✗ sudo kraft net rm kraftt1
 E  interface still in use: kraftt1@if0 (02:b0:b0:d3:64:01, 172.46.0.2)

@craciunoiuc
Copy link
Member

Deleting also messes up the network list

➜  app-nginx git:(staging) ✗ sudo ip link set dev kraftt1 down
➜  app-nginx git:(staging) ✗ sudo ip link del dev kraftt1
➜  app-nginx git:(staging) ✗ curl 172.46.0.2
^C
➜  app-nginx git:(staging) ✗ sudo kraft net rm kraftt1
 E  getting bridge kraftt1 failed: Link not found
➜  app-nginx git:(staging) ✗ sudo kraft net ls
NAME             NETWORK           DRIVER  STATUS
br-4ee203c06b74  172.20.0.1/16     bridge  up
br-c0a894b2682d  192.168.96.1/20   bridge  up
docker0          172.17.0.1/16     bridge  up
kraftt1          172.46.0.1/24     bridge  up
virbr0           192.168.122.1/24  bridge  up

@craciunoiuc
Copy link
Member

I got a lot of errors running kraft rm --all on it but no crash

➜  app-nginx git:(staging) ✗ sudo kraft rm --all
peaceful_fanni
 W  could not get network information for kraftt1: could not get link kraftt1: Link not found
xenodochial_ferdinand

And on the the machine side:

 E  could not get network information for kraftt1: could not get link kraftt1: Link not found

@craciunoiuc
Copy link
Member

Might be because of firecracker, checking now

@craciunoiuc
Copy link
Member

Not that, not sure what caused the crash, I can't reproduce

I will add a fix that removes networks if they were already removed by something else

@craciunoiuc craciunoiuc mentioned this issue Sep 20, 2023
Draft
4 tasks
@craciunoiuc craciunoiuc added the priority/low Nice to have issues thatare not urgent, but can be nagging. label Feb 29, 2024
@craciunoiuc craciunoiuc moved this from In progress to In review in KraftKit Issue Priority List Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@craciunoiuc craciunoiuc

Labels
kind/bug Something isn't working priority/low Nice to have issues thatare not urgent, but can be nagging.
Projects
KraftKit Issue Priority List
Status: In review
KraftKit Roadmap
Status: 🧊 Icebox
Milestone
No milestone
Development

No branches or pull requests
2 participants
@razvand @craciunoiuc