LibraryImportAttribute, DllImportAttribute, and You: What's really going on, here? #3238
Replies: 6 comments 1 reply
-
So, building in Debug configuration and inspecting the high-level C# as decompiled by the JetBrains decompiler, here's what the LibraryImport class looks like: internal static class LibraryImportMethods
{
[LibraryImport("kernel32", SetLastError = true)]
[GeneratedCode("Microsoft.Interop.LibraryImportGenerator", "8.0.9.8001")]
[SkipLocalsInit]
[return: MarshalAs(UnmanagedType.Bool)]
public static bool Beep(int frequency, int duration)
{
Marshal.SetLastSystemError(0);
int num = __PInvoke(frequency, duration);
int lastSystemError = Marshal.GetLastSystemError();
bool flag = num != 0;
Marshal.SetLastPInvokeError(lastSystemError);
return flag;
[DllImport("kernel32", EntryPoint = "Beep")]
static extern int __PInvoke(int __frequency_native, int __duration_native);
}
} And here's the DllImport class: internal static class DllImportMethods
{
[DllImport("kernel32", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool Beep(int frequency, int duration);
} Right away, we can see some clear differences, but also some rather interesting similarities. First of all, there are additional attributes applied in the LibraryImport case. Those are easy enough to ignore, though the fact that SkipLocalsInit is used is a clue we could probably add that to the DllImport case just for a small efficiency boost. Then we see that the LibraryImport case has generated an entire wrapper method that takes care of a little housekeeping that we might otherwise be inclined to write ourselves, such as calling But then we see....A DllImport? Yep. You're reading that right. The LibraryImport code still uses DllImport, underneath, and it looks pretty darn similar to the DllImport case we wrote, doesn't it? Moving on to the DllImportMethods class, the decompiled high-level c# looks identical to what we wrote. The only difference in the DllImport part of these two is that one specifies the real return type of the native method (int) and the one we wrote in DllImport says bool instead. These types are blittable and don't need the explicit marshalling, because a bool in .net is already actually an integer, under the hood. There's no data type that is a single bit. That's not a thing. So far, it's looking like LibraryImport isn't actually doing anything useful, except for ensuring that the methods to get/set the native error codes are actually called with every invocation of the method. But why should we bother doing that every single time? Seems a bit inefficient, no? In any case, so far we haven't seen anything that LibraryImport actually does which should make any difference when it comes to trimming, so maybe we need to dig deeper? Next comment will take it down to what the tool calls "low-level c#", to pull the covers back a little more. |
Beta Was this translation helpful? Give feedback.
-
Ok, now let's dig down to decompiled code that is not as pretty. This is what the JetBrains decompiler calls "low-level c#", and is essentially the IL code turned into the lowest level constructs and method calls that are actually legal and necessary to represent the IL in the compiled assembly, and is a good half-way point between the written code and the IL, to help understand why the IL is what it is. First, the LibraryImport class: internal static class LibraryImportMethods
{
[LibraryImport("kernel32", SetLastError = true)]
[GeneratedCode("Microsoft.Interop.LibraryImportGenerator", "8.0.9.8001")]
[SkipLocalsInit]
[return: MarshalAs(UnmanagedType.Bool)]
public static bool Beep(int frequency, int duration)
{
Marshal.SetLastSystemError(0);
int num = LibraryImportMethods.<Beep>g____PInvoke|0_0(frequency, duration);
int lastSystemError = Marshal.GetLastSystemError();
bool flag = num != 0;
Marshal.SetLastPInvokeError(lastSystemError);
return flag;
}
[CompilerGenerated]
[DllImport("kernel32", EntryPoint = "Beep")]
internal static extern int <Beep>g____PInvoke|0_0(int __frequency_native, int __duration_native);
} And the DllImport class: internal static class DllImportMethods
{
[DllImport("kernel32", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool Beep(int frequency, int duration);
} "But @dodexahedron , those are the same as the last post!" Exactly. That's pretty important for the next step and is very revealing about the whole thing. If a piece of code is complex enough to result in more interesting IL, there's usually going to be a pretty big change at this level. But there wasn't, for either class. That's because this really is very very simple/basic code, and there aren't additional ways to break it down into even simpler C#. So.... Let's move on to the IL, to see what the final result is.... |
Beta Was this translation helpful? Give feedback.
-
Ok, so there's gotta be something different in the IL, then, if this really makes a difference... right? Well... Before we even look at the IL.... Why would that even be expected? Both implementations boil down to a DllImport with the same signatures, with the only significant difference being the wrapper method and the attributes on it. We could apply attributes ourselves, so that's not even much of a difference, and we could write our own wrappers (and usually do, anyway), so... What's it doing for us? Well, just for sake of argument, let's look at the raw IL from the compiled assembly... This may or may not format/color nicely, as I'm not sure how complete github's support for CIL is, but here it goes... First, the LibraryImport class, as before, leaving in a lot of the comments it generates, for ease of reading: .class private abstract sealed auto ansi beforefieldinit
LibraryImportMethods
extends [System.Runtime]System.Object
{
.method public hidebysig static bool marshal(bool)
Beep(
int32 frequency,
int32 duration
) cil managed
{
.custom instance void [System.Runtime.InteropServices]System.Runtime.InteropServices.LibraryImportAttribute::.ctor(string)
= (
01 00 08 6b 65 72 6e 65 6c 33 32 01 00 54 02 0c // ...kernel32..T..
53 65 74 4c 61 73 74 45 72 72 6f 72 01 // SetLastError.
)
// string('kernel32')
// property bool 'SetLastError' = bool(true)
.custom instance void [System.Runtime]System.CodeDom.Compiler.GeneratedCodeAttribute::.ctor(string, string)
= (
01 00 28 4d 69 63 72 6f 73 6f 66 74 2e 49 6e 74 // ..(Microsoft.Int
65 72 6f 70 2e 4c 69 62 72 61 72 79 49 6d 70 6f // erop.LibraryImpo
72 74 47 65 6e 65 72 61 74 6f 72 0a 38 2e 30 2e // rtGenerator.8.0.
39 2e 38 30 30 31 00 00 // 9.8001..
)
// string('Microsoft.Interop.LibraryImportGenerator')
// string('8.0.9.8001')
.custom instance void [System.Runtime]System.Runtime.CompilerServices.SkipLocalsInitAttribute::.ctor()
= (01 00 00 00 )
.maxstack 2
.locals (
[0] int32 V_0,
[1] bool V_1,
[2] int32 V_2,
[3] bool V_3
)
IL_0000: nop
IL_0001: nop
IL_0002: ldc.i4.0
IL_0003: call void [System.Runtime.InteropServices]System.Runtime.InteropServices.Marshal::SetLastSystemError(int32)
IL_0008: nop
IL_0009: ldarg.0 // frequency
IL_000a: ldarg.1 // duration
IL_000b: call int32 LibraryImportMethods::'<Beep>g____PInvoke|0_0'(int32, int32)
IL_0010: stloc.2 // V_2
IL_0011: call int32 [System.Runtime.InteropServices]System.Runtime.InteropServices.Marshal::GetLastSystemError()
IL_0016: stloc.0 // V_0
IL_0017: nop
IL_0018: ldloc.2 // V_2
IL_0019: ldc.i4.0
IL_001a: cgt.un
IL_001c: stloc.1 // V_1
IL_001d: ldloc.0 // V_0
IL_001e: call void [System.Runtime.InteropServices]System.Runtime.InteropServices.Marshal::SetLastPInvokeError(int32)
IL_0023: nop
IL_0024: ldloc.1 // V_1
IL_0025: stloc.3 // V_3
IL_0026: br.s IL_0028
IL_0028: ldloc.3 // V_3
IL_0029: ret
}
.method assembly hidebysig static pinvokeimpl ( "kernel32" as "Beep" winapi nomangle )int32
'<Beep>g____PInvoke|0_0'(
int32 __frequency_native,
int32 __duration_native
) cil managed preservesig
{
.custom instance void [System.Runtime]System.Runtime.CompilerServices.CompilerGeneratedAttribute::.ctor()
= (01 00 00 00 )
}
} In particular, the very last .method is the most interesting, since it's what we have found to be equivalent, so far, between the two. Here's the CIL for the DllImport class: .class private abstract sealed auto ansi beforefieldinit
DllImportMethods
extends [System.Runtime]System.Object
{
.method public hidebysig static pinvokeimpl ( "kernel32" winapi lasterr )bool marshal(bool)
Beep(
int32 frequency,
int32 duration
) cil managed preservesig
{
}
} Ok. A few differences, and otherwise they seem pretty similar, don't they? But what does it mean? Next comment will pick apart the CIL. |
Beta Was this translation helpful? Give feedback.
-
Something that may help a little if you're not familiar with CIL is the ECMA-335 Standard, which is the latest ECMA standard for CIL from 2012 and contains descriptions of most opcodes and metadata up to that point in time. Anyway... So, let's go line-by-line, same order as before, where LibraryImport is first, followed by DllImport: .method assembly hidebysig static pinvokeimpl ( "kernel32" as "Beep" winapi nomangle )int32
// and
.method public hidebysig static pinvokeimpl ( "kernel32" winapi lasterr )bool marshal(bool) This is all metadata, declaring a (you guessed it) method. Next keyword in each one is an accessibility modifier. The LibraryImport method gets only "assembly," which just means it's only accessible within this assembly. In c# terminology, that's internal. Interesting. So it's callable by other languages, potentially? Interesting. The DllImport case says public, which makes sense, since we declared it as public. Next is Next is static, which has basically the same meaning as it does in C#. It's something that doesn't need an instance of the Next, they both say Inside the parentheses, the syntax follows the above snippet. First is the location of the method, so the name of the library. Next is an optional name, typically only emitted by the compiler if the name in code does not match the name of the EntryPoint. The LibraryImport case used a fake name, so it had to specify this, whereas our DllImport case used the real name and was able to leave it out. Next is the platform-specific calling convention. In this case, it's winapi, which is a macro for the calling convention used in win32. Next the LibraryImport has The DllImport case has Yes, but only on the extern method. In the LibraryImport case, getting the last error code is explicitly handled in code. In the DllImport case, that code isn't there, and this keyword just makes the runtime do that part for you. So, same, right? Well, yes, but with a potential caveat, which is relevant if the DllImport case isn't wrapped with the same functionality, either in an actual wrapper/proxy method or directly in the callsite. This is a difference that will matter when AoT compilation is performed. The LibraryImport case will get and set error codes as we expect it to from the attribute, as we specified it, without any other work. The DllImport case, however, has no such explicit code and thus that won't happen, unless we did it explicitly. So, for this bit, the LibraryImport case has written that boilerplate for us already. Small but important difference that is, nevertheless, quite trivial to handle in the DllImport case by calling those methods when needed. Then, after the parentheses, is the return type of the method. Going to talk about that in more detail, since it is different, but for explainable reasons. For the LibraryImport case, we see int32, which is the most literal match to the native API method possible. That one has the potential to be more important and to require more care when writing code, if writing things with DllImport. In the case of this method, it's not a problem, because bool and int32 are required by the standard to be implemented and implicitly able to be marshaled by the runtime, on all platforms. See page 189 of the spec for information on marshaling, the entirety of which is this, as of that version of the spec: Even though platform-specific rules may be different for what these types mean, it isn't relevant here, because the method calls themselves don't cross platform boundaries. Long story short on that, this difference is only relevant for the LibraryImport case because it opted to do the conversion to bool itself, explicitly, using that That seems like a potential problem for types where the compiler couldn't possibly know the conversion rules for, though, right? Yes, but actually mostly no. Next comment will resume from the point of answering that question. |
Beta Was this translation helpful? Give feedback.
-
So I said "yes, but actually mostly no," in response to the question of if LibraryImport might behave improperly due to it needing explicit marshaling information for primitives like bool, whereas DllImport does not. Why yes? Why "mostly no?"
For #1, it's already the end of the road. Attributes in code are not relevant to runtime code without reflection, unless a source generator consumes them during compilation and emits other code because of them. For #2, Let's take a look at that attribute for the return of the methods.... The For LibraryImport, the return type's required marshaling behavior is not assumed, if the return type is not identical to that of the native API method. Specifying There are a few attributes you can use to influence marshaling code generated by the source generator:
Which attribute is used is up to you, since they end up causing the same behavior for identical inputs to the corresponding parameters of each. If there's a non-blittable type, such as one that contains reference fields or other non-blittable instance member fields, you will be required to provide a custom marshaller for that type, specified in any of the above attributes (preferably using Custom marshallers must implement ICustomMarshaler. The methods in that interface do what their names suggest. Also see this document for a lot more detail on ICustomMarshaler and how to use it. This brings up something about the way the source generator code is, as well... Note that it calls Marshal.SetLastPInvokeError(lastSystemError); before returning? Why's it doing that? That's because of this explanation from Microsoft Learn:
That is exactly what the compiler-generated code for LibraryInput does. Again, this is something that can be trivially done in the DllImport case by simply making the method calls shown above in a wrapper method like LibraryInput creates, or surrounding the call site, if such wrapper does not exist. As is plainly shown in the IL provided earlier, that's just two lines of code. Usually, MarshalAsAttribute is the preferred attribute for any element, as it is the most flexible and can be applied to return types, parameters, and even to fields of types that will be used for PInvoke, which avoids most cases that would otherwise necessitate an ICustomMarshaler to be supplied for the type. And the next lines?The next few lines, as should be obvious are just the rest of the signature of the method, but then we get to this, at the end of the two: cil managed preservesig That's the same for both, so clearly it can't be anything different, but here's what they're for: cil specifies the language to interpret the following body as. Since it's CIL code, this keyword is used. managed means the following method is managed, meaning the GC cares about things that happen in the method body. preservesig is for windows and is mostly only relevant specifically for COM interop, which isn't what we're doing. Next comment should be the last of this series, wrapping things up. |
Beta Was this translation helpful? Give feedback.
-
So... The part you probably actually care about: How does the code published as a native AoT binary (and the trimming that implies) behave?Well, on .net 8, on my system, I get an exe about 1.3MB in size that, when launched.... *drum roll please*... Plays a one-second tone at what I'm trusting is 800Hz, immediately followed by a one-second tone that sounds like it's an octave lower (half the frequency or 400Hz). So, what does that mean?Well... I've mostly already said everything that it means, as we've worked our way to this point. It means that DllImport, in a vacuum, is not obsolete, bad, or incompatible with trim, contrary to what documentation on Microsoft Learn and elsewhere around the net might say or at least make one (reasonably, the way they put it) assume. Rather, we learned the following, which are the important takeaways:
That's really it, for the basic case. And the fact that it is a code generator and writing a wrapper based on the signature you generated is exactly why LibraryImport has the restrictions and requirements it has, such as needing to be a But it must matter sometimes, right?Yes. Of course. Otherwise it most likely wouldn't exist and documentation wouldn't goad you in that direction. So, when does it matter and why should I bother using LibraryImport?It matters when there's any marshaling going on that you haven't explicitly written and that is required to actually make the call as written. Alright, so when is it required, then?Mostly, in these situations, and only if trimming is involved in the build process:
If you either aren't using/doing any of the above or if you have written code around every usage of a DllImport method that does need or does any of that (best is to just write a wrapper), then you don't need to use LibraryImport, even to produce trim-safe compiled assemblies. If it prevents me from having to care, why not just always use LibraryImport?Well... That's a valid question, in most cases. The answer comes down to the following:
And, if it's a question of replacing DllImport with LibraryImport, for a significant body of existing code, add on these questions, which are all subjective but helpful to guide you toward a yes or no:
Specifically for Terminal.Gui, what's the bottom line?Most PInvoke calls in the existing code are almost certainly already trim-friendly. The questionable ones are those which violate any of the bullets in the bullet list prior to the one immediately above. That's mostly (maybe only) Windows stuff. Some of those may perform the necessary stuff that would otherwise be trim-unsafe near/around their callsites, so they may actually be fine anyway. That will just require investigation. In any case, any time we make changes to PInvoke code, we should always try an AoT publish on the platforms targeted by that code and ensure that it actually works. That, unfortunately, requires running inside those environments. So, to automate it on github will require additional test runner configurations for Windows and MacOS environments, in addition to the *nix runner currently in use. That's probably a good thing for us to do, anyway, completely unrelated to this. Man, that was longYeah... I'm verbose. Sorry. Any questions, comments, corrections, or whatever are welcome. Footnotes
|
Beta Was this translation helpful? Give feedback.
-
So... I was thinking, while playing with CSWin32 and also browsing various Microsoft code on github related to PInvoke, with the initial intention of looking for "blessed" versions of stuff to be sure we're doing things as actually intended, straight from the horse's mouth.
But, as I was going through that code, I noticed some pretty glaring deviations from their documented recommendations, the biggest of which was...
Why don't they use LibraryImportAttribute in any of that code and why doesn't CsWin32 emit LibraryImportAttribute either? 🤔
They're very clearly aware of it, being...ya know...Microsoft...and most of the people working on it are on the relevant teams at MS who create and maintain these things anyway, so that was a big clue that the public documentation may not be telling the whole story, out of what I can only assume is an attempt to encourage consistency...
And then, what that does actually mean, with regards to LibraryImportAttribute itself? That's there for source generation. Attributes aren't relevant at execution. There must be SOME good reason for suggesting a change, right? Or it must be generating some kind of special code, right? Well... No... Source generators output c# code, which still has to follow the same rules written code has to follow, so it couldn't be that.... 🤔
So that made me suspicious that it's probably all smoke and mirrors and that there's not actually much happening that's actually any different.
So, I put together a simple console application, just to inspect what's actually going on.
Here is the entirety of the program, which I will dissect afterward:
This program declares two classes that each hold the same method from kernel32:
bool Beep(int,int)
, the ancient method that plays the specified tone for the specified duration. The method isn't important. What's important is what this all compiles down to.One class does it via LibraryImportAttribute, and the other does it via DllImportAttribute. Otherwise, they're identical, in source.
But we want to know what they end up as, after compilation.
Analysis to follow in additional comments, starting with a high-level decompiled c# representation of the two static classes.
Beta Was this translation helpful? Give feedback.
All reactions