What the hell is marshaling?

Question

Whenever I see signatures of P/Invoke full of Marshaling attributes I always remove one by one and testing if this causes any error, most of the time I end up with a signature with no attribute, no side effect.

What is Marshaling?

What are attributes for, when to use and when this is required?

Maybe that fits another question, but what comes to be the Calling Convention?

Newer versions of Visual Studio generate warnings about the absence of Marshaling definition, which side effect of not explicitly defining these attributes?

This affects the performance?

This concept is general in computing or is a pipeline specific memory of. NET?

Answer 1

What is Marshaling?

Do you know about serialization? It’s more or less the same thing. But the goal of it is to call some function from the other side so you just pass parameters or get the return in a RPC (Remote Process Call) process. Serialization has no specific function. Roughly speaking we can say that there is an embedded serialization. At least this is the universal way to use.

It is a more mechanism to shape the call, not necessarily need to serialize, just ensure that the call is in accordance with what is expected.

Can be used as IPC (Inter Process Call) too. O . NET uses both, although the remote part has been preferred by other mechanisms involving serialization. Aside from the obsolete mechanisms, I don’t know if there’s anything remote left in . NET.

In general the adopted process is to decompose an object into its most primitive types so that it can meet the need of the other side that does not understand exactly what it needs. Documentation.

It is common for the type used to be modified or interpreted in some way to conform to what it needs on the other side, for example a C API that has slightly different types of CLR in some cases.

Just as there is deserialization there is unmarshalling.

What are attributes for, when to use and when this is required?

It serves precisely to say how the object will be treated, how the objects will be used.

An example that can be given is an attribute that indicates that a structure will be used to communicate with C so it cannot go through some optimization that C# would do. Or indicate as a string will be formed. Example:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct MyPerson {
    public string first;
    public string last;
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 3)]
    public int[] vals;
    [MarshalAs(UnmanagedType.BStr)]
    public string str;
}

[DllImport("..\\LIB\\PinvokeLib.dll", CallingConvention = CallingConvention.Cdecl)]
internal static extern int TestStructInStruct(ref MyPerson person);
[DllImport("..\\LIB\\PinvokeLib.dll", CallingConvention = CallingConvention.Cdecl)]
internal static extern int TestArrayOfStructs2([In, Out] MyPerson[] personArray, int size);

I put in the Github for future reference.

There is also an immense amount of utilitarian methods to be able to make conversation between languages flow even considering their differences, for example by the fact of . NET use managed memory and in C not.

Documentation.

Maybe that fits another question, but what comes to be the Calling Convention?

It is the call form a function, for example how the parameters will be placed in the registers, the order, what should be copied, etc., is the way the compiler transforms the call of function into a native code so that everything fits. The call itself and the function being called must always have the same call convention to work. The attributes may be indicating something about this.

See some.

I said something related to What is ABI? Has some relation to API?.

Newer versions of Visual Studio generate warnings about the absence of Marshaling definition, which side effect of not explicitly defining these attributes?

If the call and the function to be called are incompatible there will be a break of the application or have an unwanted result without warning. I have no experience and I won’t try to answer better than that.

This affects the performance?

If you actually use the mechanism it undoubtedly does. Just because it’s prepared to use it doesn’t mean it does. Just because it indicates that something must be in a format means that there will be conversion.

This concept is general in computing or is a . NET-specific memory pipeline?

It’s general. The way . NET uses it is specific to it. It doesn’t mean that all languages use it.

Some use it without using the term, it’s very curious. I always used a language (it was Clipper, now it’s Harbour) that always did this without ever using the term. This one I can explain better because I know her well, but it won’t help much because other people don’t know her. But it’s not so different from the other languages of script market, such as PHP, Python, Ruby, Lua, etc.

These languages have bytecodes that form the algorithms organized into functions. When you talk to C for example, which is usually the basis of all these languages (it has different implementations), the parameters and the way of calling function is very different and it takes a marshalling to adapt the call. C functions are raw memory addresses and the functions of these languages are tables with bytecodes, and the parameters are more complex data structures to manage the dynamicity of the language, C does not understand this, need a translation engine. These languages require you to have a C function to handle this, in C# does not require you can call any C function directly, it takes care of solving.

It doesn’t have to be C all this communication, it’s just the most common. It needs to be a native language without greater demands (it may not be, but then it would have to have a glue, until C++ needs a glue in some cases, for that there is a CX).

It is possible for these languages to use something similar for some form of RPC as well.

Whenever I see signatures of P/Invoke full of Marshaling attributes I always remove one by one and testing if this causes any error, most of the time I end up with a signature with no attribute, no side effect.

I would need to see the specific situation, but you’re probably not actually using it, at least not in a way that causes problems. Maybe I didn’t need marshalling, but it is unlikely. It may be that you only need a specific way of calling. So these attributes are useful only in this case.

Some cases the marshalling can occur automatically by Runtime without being told anything by adopting a default, but there is risk of not being desired. If the type is blittable do not need to indicate anything because it is already what should be on the other side. It may be that they do it to be more explicit or to silence some tool.

I don’t know the specific scenario, nor am I an expert in the mechanism since I never had to use it effectively (I have a personal project that I would use, but I can’t find time to do).

The whole subject is quite complex. I think this answer serves more as a curiosity about the subject and know what it is. Not even close to picking up every detail.