Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use link_name for dynamic library loading #3101

Conversation

Molot2032
Copy link
Contributor

@Molot2032 Molot2032 commented Jan 28, 2025

Problems

Dynamic library codegen generates code trying to load symbols using the name exposed in bindings instead of the overriden link name or mangled name.
Note that libloading::Library::get does not mangle the name for you

  1. When generating bindings for a dynamically loaded library (behaviour available by calling dynamic_library_name on the builder), the symbol names used in the generated __library.get("XXX") calls aren't the mangled link name as one might expect, but instead the bindings name. This behaviour by chance works with C-style exports, but is incorrect with mangled symbols (e.g. C++ exports) or when a user overrides the name of an item.
  2. In addition, the link_name attribute is sometimes erroneously applied to methods in the generated struct, causing this warning:
    attribute should be applied to a foreign function or static
    this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
    

Example

If my function in C is defined like this:

COOLLIBRARY_API float CoolLib_coolFunction(void);

I might have something like this implemented on my Processor to lop off the prefix, since it feels redundant when the item is already namespaced by being a member of the generated dylib type.

    fn generated_name_override(
        &self,
        item_info: bindgen::callbacks::ItemInfo<'_>,
    ) -> Option<String> {
        if let ItemKind::Function = item_info.kind {
            let name = item_info.name.to_string();
            if name.starts_with("CoolLib_") {
                return Some(name[8..].to_string());
            }
        }
        None
    }

The line in the generated from_library will then look for the wrong symbol "coolFunction"

let coolFunction = __library.get(b"coolFunction");

when it should be this:

let coolFunction = __library.get(b"CoolLib_coolFunction");

When troubleshooting I messed around with generated_link_name_override, and noticed that it doesn't affect that generated code at all, and also reveals the second problem mentioned above about the link_name attribute -- the attribute is added if the symbol name would differ from the canonical one -- in this case because it's overriden.

Note that generated_link_name_override is, confusingly, passed the normal name (possibly overriden by the processor's generated_name_override), rather than the link name. So in my case even with the PR's changes I had to add the following to my Processor impl to re-add the prefix for to the symbol so that it can be found:

    fn generated_link_name_override(
        &self,
        item_info: bindgen::callbacks::ItemInfo<'_>,
    ) -> Option<String> {
        format!("CoolLib_{}", item_info.name.to_string()).into()
    }

I decided not to touch this last oddity in the PR in case I'm overlooking something, but would be happy to tack it on with go-ahead from a maintainer.

Changes in the PR

  • Fix problem 1: The symbol name used in the generated __library.get call will use the link_name for a function or var; i.e., the same as what is generated for link_name attributes. In order of precedence, this is:
    1. the overriden link name if it exists, or
    2. the mangled name if it exists, or
    3. the parsed name.
  • Some dynamic_loading_xx expectations tests were also victim to problem 1, so they've been adjusted. Some integration tests for dynamic loading would be handy here.
  • Fix problem 2: Methods in the generated library type's impl to call the inner dynamically-loaded functions will no longer be erroneously given the link_name attribute.

I have tested and confirmed that the changes generate fully working bindings on my own windows and linux environments with a test dynamic library and binary that loads it.

@Molot2032 Molot2032 force-pushed the dynamic_library_use_link_name_when_getting_symbol branch 2 times, most recently from 7ac4a39 to 6364365 Compare January 30, 2025 01:43
@Molot2032 Molot2032 marked this pull request as draft January 31, 2025 02:22
@Molot2032 Molot2032 force-pushed the dynamic_library_use_link_name_when_getting_symbol branch from 6364365 to b6d7a00 Compare February 1, 2025 00:18
@Molot2032 Molot2032 marked this pull request as ready for review February 1, 2025 00:46
@Molot2032
Copy link
Contributor Author

@emilio
@pvdrz
Requesting review 🙂

Copy link
Contributor

@emilio emilio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks sensible, thanks!

@emilio emilio added this pull request to the merge queue Feb 1, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to a conflict with the base branch Feb 1, 2025
@Molot2032 Molot2032 force-pushed the dynamic_library_use_link_name_when_getting_symbol branch from b6d7a00 to ed9962c Compare February 1, 2025 22:39
@Molot2032
Copy link
Contributor Author

@emilio
Fixed merge conflicts with commit c704b14, which touched some of the same function signatures.

@emilio emilio added this pull request to the merge queue Feb 2, 2025
Merged via the queue into rust-lang:main with commit 03d49b6 Feb 2, 2025
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants