Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix C3 hot reloading #2

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

anonymix007
Copy link

@anonymix007 anonymix007 commented Jan 2, 2025

TLDR: do not use tclone as it's freed on library unload; interface is any, typeid is vtable and can be easily overwritten with any_make.

This will still break if you change data layout or add new (unhandled) types, so it should be on par with C.

Here's an explanation of how I came up with that.

Let's try to see what's actually happening during the hot reload:

...
ERROR: 'Out of bounds memory access.'
  in std.core.builtin.print_backtrace (/home/user/.local/lib/c3/std/core/builtin.c3:69) [./build/libc3.so]
  in std.core.builtin.sig_segmentation_fault (/home/user/.local/lib/c3/std/core/builtin.c3:715) [./build/libc3.so]
  in __sigaction (source unavailable) [/usr/lib/libc.so.6]
  in plug.Parallel.poll (/home/user/projects/c3-tests/panim/plugs/c3/plug.c3:53) [./build/libc3.so]
  in plug_update (/home/user/projects/c3-tests/panim/plugs/c3/plug.c3:142) [./build/libc3.so]
  in main (/home/user/projects/c3-tests/panim/./panim/panim.c:285) [./build/panim]
  in __libc_init_first (source unavailable) [/usr/lib/libc.so.6]
  in __libc_start_main (source unavailable) [/usr/lib/libc.so.6]
  in _start (source unavailable) [./build/panim]
Illegal instruction (core dumped)

But why? Let's fire up the GDB:

$ gdb --args ./build/panim ./build/libc3.so
(gdb) r
...
Thread 1 "panim" received signal SIGSEGV, Segmentation fault.
0x00007ffff7a7cda1 in plug.Parallel.poll (urmom=0x555556ef3780, env=...) at plug.c3:53
53	    foreach (future: &urmom.futures) {
(gdb) p *urmom
$2 = {futures = {ptr = 0xaaaaaaaaaaaaaaaa, len = 12297829382473034410}}

Yeah, I can only agree with that. AAAAAAAAAAAAAAAA!

(gdb) b reload_libplug
(gdb) r
...
(gdb) p *((void **)state+2)
$1 = (void *) 0x555556ef3780

What's *((void **)state+2)? It's state.anim, the instance of Future interface (aka any)
Let's add watchpoint and see what updates it:

(gdb) watch *0x555556ef3780@2
Hardware watchpoint 2: *0x555556ef3780@2
(gdb) c
Continuing.

Thread 1 "panim" hit Hardware watchpoint 2: *0x555556ef3780@2

Old value = {1458517840, 21845}
New value = {1458517930, 21845}
0x00007ffff7a78e26 in std.core.mem.allocator.TempAllocator.reset (self=0x555556ef3630, mark=0) at temp_allocator.c3:84
84						self.data[mark : cleaned] = 0xAA;

Mistery solved, I guess.

But this is not the end yet. If code layout changes even a little bit, we'll be greeted with

ERROR: 'No method 'poll' could be found on target'
  in std.core.builtin.default_panic (/home/user/.local/lib/c3/std/core/builtin.c3:99) [./build/libc3.so]
  in plug_update (/home/user/projects/c3-tests/panim/plugs/c3/plug.c3:283) [./build/libc3.so]
  in main (/home/user/projects/c3-tests/panim/./panim/panim.c:285) [./build/panim]
  in __libc_init_first (source unavailable) [/usr/lib/libc.so.6]
  in __libc_start_main (source unavailable) [/usr/lib/libc.so.6]
  in _start (source unavailable) [./build/panim]
Illegal instruction (core dumped)

That's interesting... So, apparently, dynamic dispatch failed.

Let's patch vtables then. Shouldn't be too hard, right? Padme.jpeg

Since interface is basically an any containing a pointer and a typeid (which is actually just a pointer to a global vtable), we can just

macro @patch(Future *f, $Type) {
    usz *x = (usz *) f;
    *x[1] = (usz) $Type.typeid;
}

fn void plug_post_reload(void *old_state) @export("plug_post_reload")
{
    state = old_state;

    @patch(&state.anim, Parallel);
    Parallel *p = anycast(state.anim, Parallel)!!;

    foreach(&s: p.futures) {
        @patch(s, Seq);
        Seq *s1 = anycast(*s, Seq)!!;
        foreach(&l: s1.futures) {
            @patch(l, Lerp);
            Lerp *l1 = anycast(*l, Lerp)!!;
            (void) l1;
        }
    }
}

Can your C++ or (safe) Rust do that? I don't fucking think so.

The actual implementation in this PR is a bit different so that this code would be closer to where state.anim is actually allocated.

Of course, there are some limitations, so having language support would've been much better.
It's actually not that hard to implement by hooking into the allocator (so that it'll export hashmap of typeid * to String before unload and optionally import after reload).
I'll leave this as an exercise to the streamer.

EDIT: There is a TrackingAllocator which does almost that, but without the typeid* which needs to be tracked separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant