Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect pointer handling in unsafe blocks causes rendering problems with non-zero opt-level #68

Open
miabaka opened this issue Jan 14, 2025 · 4 comments

Comments

@miabaka
Copy link

miabaka commented Jan 14, 2025

Hi! I'm using your crate in my rewrite of an old game engine and currently rendering everything with SDL_Renderer (e.g. Canvas) instead of raw graphics APIs.

When compiling with non-zero opt-level, most graphics don't render, and I can't see relevant draw calls in RenderDoc.

When calling Canvas<T>::copy without specifying src and dst, it works fine. However, specifying one or both causes the objects to disappear.

It seems there's a mistake in the unsafe blocks where you're taking references to temporary objects. The compiler optimizes these out, leading to invalid pointers being passed to SDL_RenderTexture. I don’t know how this should actually behave in Rust, but I'm seeing this issue.

I tried to fix it by modifying code like this and it actually works:

// Before

let ret = unsafe {
    sdl3::sys::render::SDL_RenderTexture(
        self.raw(),
        texture.raw(),
        match src.into() {
            Some(ref rect) => &rect.to_ll(),
            None => null(),
        },
        match dst.into() {
            Some(ref rect) => &rect.to_ll(),
            None => null(),
        },
    )
};

// After

let src = match src.into() {
    Some(ref rect) => &rect.to_ll(),
    None => null(),
};

let dst = match dst.into() {
    Some(ref rect) => &rect.to_ll(),
    None => null(),
};

let ret =
    unsafe { sdl3::sys::render::SDL_RenderTexture(self.raw(), texture.raw(), src, dst) };

I don't have much time and resources to investigate this further and create a PR with fix so just leaving this issue here

@miabaka miabaka changed the title Incorrect pointer handling in unsafe blocks causes Rendering problems with non-zero opt-level Incorrect pointer handling in unsafe blocks causes rendering problems with non-zero opt-level Jan 14, 2025
@WaffleLapkin
Copy link

The current version is quite clearly UB:

        match src.into() {
            //                 vvvvvvvvvvvv ---- this creates a temporary object
            Some(ref rect) => &rect.to_ll(), // <-- temporary is dropped here
            //                ^ -- this creates a reference that is immediately coerced to a pointer
            None => null(),
        },
        // here the pointer is invalid, since the temporary has been "freed"

I found 4 cases of this:

https://github.com/revmischa/sdl3-rs/blob/08e59fdee3d959db50fd50f613d00e76d003976a/src/sdl3/render.rs#L1342-L1349
https://github.com/revmischa/sdl3-rs/blob/08e59fdee3d959db50fd50f613d00e76d003976a/src/sdl3/render.rs#L1405-L1417
https://github.com/revmischa/sdl3-rs/blob/08e59fdee3d959db50fd50f613d00e76d003976a/src/sdl3/render.rs#L1295
https://github.com/revmischa/sdl3-rs/blob/08e59fdee3d959db50fd50f613d00e76d003976a/src/sdl3/video.rs#L1639-L1642

Although it does make you think how many similarly subtle UB is also in here.

The change made by @miabaka is not UB, but subtly so: since rust 1.79.0 match "forwards" lifetime extension from let, so the temporary from &rect.to_ll() lives for the whole function if it's in a match in a let.

Depending on lifetime extension for unsafe code is completely bonkers™ -- lifetime extension rules are quite subtle and even the most experienced rust users can't always tell how the code will behave wrt them. Lifetime extension rules were developed specifically for safe code/references, where a mistake leads simply to a compilation error.

The correct way of fixing all of these UB cases is to assign .to_ll() value to a local variable first, and then use that to get a reference/pointer. Something like this (example for the copy function, but similar code can be used in all of the linked code):

        let src = src.into().as_ref().map(FRect::to_ll);
        let dst = src.into().as_ref().map(FRect::to_ll);

        let ret = unsafe {
            sys::render::SDL_RenderTexture(
                self.context.raw,
                texture.raw,
                src.as_ref().map_or(ptr::null(), ptr::from_ref),
                dst.as_ref().map_or(ptr::null(), ptr::from_ref),
            )
        };

@revmischa
Copy link
Collaborator

revmischa commented Jan 18, 2025

Thanks for the detailed info! Want to open a PR with the fixes?

@WaffleLapkin
Copy link

I don't (never used sdl myself; researched just out of curiosity)

@miabaka
Copy link
Author

miabaka commented Jan 19, 2025

I'll probably fix this and open a PR in the next couple of days

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants