Friday 12 March 2010

Dynamic method invocation without the TargetInvocationException wrapping

This post
is about

C#

Motivation

I find it useful that the Visual Studio debugger stops at the right place where an unhandled exception occurs. That makes it all the more annoying when it doesn’t. One of those cases where it doesn’t is when any of the methods on the call stack happen to have been called dynamically via Reflection. MethodBase.Invoke catches all exceptions thrown by the target method, wraps them in a TargetInvocationException, and then rethrows that. Consequently, the Visual Studio debugger stops at the place where the dynamic invocation happens, which means I can't see the context and call stack where the actual exception really happened.

I am, nota bene, aware of the fact that there is a good reason for having the TargetInvocationException wrapping. I am also aware of the fact that you can instruct Visual Studio, via “Exceptions” in the “Debug” menu, to stop the debugger when the exception happens irrespective of whether it is unhandled or not. Neither of those considerations mitigate the annoyance; the “Exceptions” dialog is very hard to use, especially for custom-defined exception types, which are not automatically listed — and even if it wasn’t, I simply shouldn’t have to do any of that because in most cases the fact that a method was invoked dynamically is irrelevant to the particular problem I might be debugging.

A partial solution

Any solution to this problem must not involve MethodBase.Invoke(). Nor, indeed, can it use Delegate.DynamicInvoke() because that performs the same exception wrapping. However, invoking a normal delegate of the right signature doesn’t do the wrapping, so we’ll start looking there.

public delegate string IntToStringDelegate(int integer);

public string InvokeMethod(MethodInfo method, object instance, int parameter)
{
    var delegate = (IntToStringDelegate) Delegate.CreateDelegate(
        typeof(IntToStringDelegate), instance, method);
    return delegate(parameter);
}

We’ve achieved part of our goal; we can now call any int-to-string method — even private ones! — and have the Visual Studio debugger report exceptions in the place where they occur. But the usefulness is limited by the fact that we can only call single-parameter methods that take an int and return a string. We can easily extend this to any single parameter type by declaring generic methods:

public void InvokeVoidMethod<TParam>(MethodInfo method, object instance,
        TParam parameter)
{
    var delegate = (Action<TParam>) Delegate.CreateDelegate(
        typeof(Action<TParam>), instance, method);
    delegate(parameter);
}

public TResult InvokeMethod<TParam, TResult>(MethodInfo method, object instance,
        TParam parameter)
{
    var delegate = (Func<TParam, TResult>) Delegate.CreateDelegate(
        typeof(Func<TParam, TResult>), instance, method);
    return delegate(parameter);
}

It is straightforward from here to extend this further by adding additional overloads that can take multiple parameters. But the attentive reader will surely see that this solution is still rather useless: it requires the caller to know the exact signature of the method at compile time. In most cases where you know the signature, you generally know the method itself, so you could actually just call it directly (unless, of course, it’s private). We want to cover the case where you know nothing about the method and all you have is an object[] containing the intended parameters for the method (which have the right types, but only at runtime), an object containing the instance on which you want to call the method, and the MethodInfo object describing the method to be called. In other words, we want to replace MethodInfo.Invoke() completely.

This poses several problems. We can’t use Func<> or Action<> because they only exist for up to four parameters. Of course you could declare more of them, but you can’t declare infinitely many of them. We also can’t invoke the delegate directly; the only method on the Delegate type that can take an object[] for the parameters is Delegate.DynamicInvoke(), which we know is unsuitable because it wraps our precious exceptions.

The full solution

We are going to create a whole new assembly. At runtime. We will dynamically declare a delegate type and we will write a method in IL. Sounds crazy? It sure is!

But let’s go slow. Let’s start from the beginning. What do we need? We know that we can’t invoke the delegate directly. We can’t even create the delegate directly. But we certainly need a method that we can invoke directly. Let’s think about what such a method would have to look like by creating an interface declaration. We certainly can’t make any assumptions about the parameter types, the return type, or the instance type on which we want to call the method. Everything has to be unspecific:

public interface IInvokeDirect
{
    object DoInvoke(object instance, object[] parameters, MethodInfo method);
}

Now that we have an interface, let’s think about what a class would have to look like that wants to implement this interface. Since we can theoretically have several different classes implementing this same interface, we can actually start making assumptions. So let’s examine an easy special case first. What would an implementation of this method have to look like if the method has one parameter of type “int” and a return type of “string” (like the IntToStringDelegate we mentioned earlier)? Answer: it would look almost exactly like the InvokeMethod() we had at the very beginning:

public delegate string IntToStringDelegate(int integer);

public class IntToStringInvoker : IInvokeDirect
{
    public object DoInvoke(object instance, object[] parameters, MethodInfo method)
    {
        var delegate = (IntToStringDelegate) Delegate.CreateDelegate(
            typeof(IntToStringDelegate), instance, method);
        return (string) delegate((int) parameters[0]);
    }
}

And now comes the magic. We don’t want to make any assumptions about the parameter types at compile time, but we certainly know the method parameter types at runtime (simply by looking at the MethodInfo object we’re given). Therefore, for any given MethodInfo object, we know exactly what the above delegate and class declarations would have to look like. We could, if we wanted to, generate the C# code at runtime. But that would be laborious: we’d have to make sure we output all the type identifiers in valid C# syntax (including arrays, generic types, etc.) and we’d have to make sure to include references to the client code that declares some of those types. So instead of generating C# code, we generate the actual delegate and class types directly — using System.Reflection.Emit.AssemblyBuilder. This works because the stuff in the System.Reflection.Emit namespace is clever enough to let us pass Type and MethodInfo objects into it that we don’t know anything about. Also, we can be more efficient by using only one assembly instead of generating a new assembly for each different method signature.

So our InvokeDirect() method will look roughly like this:

public static class DirectInvoker
{
    private static Dictionary<string, IInvokeDirect> invokers =
        new Dictionary<string, IInvokeDirect>();

    public static object InvokeDirect(MethodInfo method,
        object instance, params object[] parameters)
    {
        // Put some exception throw statements here to ensure that method != null,
        // method.GetParameters().Length == parameters.Length, etc.

        // Create a dynamic assembly, or re-use the last one
        createAssembly();

        // Generate a string that identifies the method "signature" (the parameter
        // types and return type, not the method name). Different methods which
        // have the same "signature" according to this criterion can re-use the
        // same generated code.
        var sig = string.Join(" : ", method.GetParameters()
            .Select(p => p.ParameterType.FullName)
            .Concat(new[] { method.ReturnType.FullName }).ToArray());

        if (!invokers.ContainsKey(sig))
        {
            // Create a delegate type compatible with the method signature
            var delegateType = createDelegateType(method, sig);

            // Create a class that implements IInvokeDirect
            var classType = createClassType(method, sig, delegateType);

            // Instantiate the new class and remember the instance for future calls
            invokers[sig] = (IInvokeDirect) Activator.CreateInstance(classType);
        }

        // Invoke the interface method, which will call the target method.
        return invokers[sig].DoInvoke(instance, parameters, method);
    }
}

I’ll skip the boring parts (createAssembly() and createDelegateType() are straightforward) and get straight to createClassType(), where we construct our method by emitting IL code. We have to write IL code that is equivalent to the DoInvoke() method above, so let’s first see what this method looks like in IL. We’ll actually look at the IL for a single-line version of the method that doesn’t use a local variable, and I’ll make it two parameters instead of one so we can more easily spot the repeating bit:

public object DoInvoke(object instance, object[] parameters, MethodInfo method)
{
    return ((IntToStringDelegate) Delegate.CreateDelegate(
        typeof(IntToStringDelegate), instance, method))
        (
            (int) parameters[0],
            (string) parameters[1]
        );
}

Here’s the IL. I’ve abbreviated the type names to make it more readable, and I’ve added colours to show which part of the above method each bit corresponds to. Also, notice that “delegate(parameters)” is C# syntactic sugar for “delegate.Invoke(parameters)”, hence the call to an “Invoke” method at the end. Finally, the following IL is for the Release build; if you compile in Debug mode, you’ll see a few redundant extra operations such as nop.

.method public hidebysig static object DoInvoke(
    object instance, object[] parameters, MethodInfo method)
    cil managed
{
    .maxstack 8
    L_0000: ldtoken IntToStringDelegate
    L_0005: call Type Type.GetTypeFromHandle(RuntimeTypeHandle)
    L_000a: ldarg.0
    L_000b: ldarg.2
    L_000c: call Delegate Delegate.CreateDelegate(Type, object, MethodInfo)
    L_0011: castclass IntToStringDelegate
    L_0016: ldarg.1
    L_0017: ldc.i4.0
    L_0018: ldelem.ref            // Notice this one uses unbox.any instead of castclass
    L_0019: unbox.any int32    // because int is a value type
    L_001e: ldarg.1
    L_001f: ldc.i4.1
    L_0020: ldelem.ref            // Notice this one uses castclass instead of unbox.any
    L_0021: castclass string     // because string is a reference type
    L_0026: callvirt string IntToStringDelegate.Invoke(int, string)
    L_002b: ret
}

You can straightforwardly translate this into a series of calls to ILGenerator.Emit(). The only gotchas you need to be aware of are to use unbox.any for value types and castclass otherwise; and to use ldc.i4.s and ldc.i4 appropriately to handle cases where you have more than 9 parameters. Finally, if the return type is void, you have to add an extra ldnull before the ret (just like in C# where you have to add an extra “return null” after the delegate call).

The complete code

To save you some frustration trying to get the boring bits to work, here is the complete implementation.

Caveat

This solution doesn’t work for invoking constructors. Delegate.CreateDelegate() expects a MethodInfo, not a MethodBase (ConstructorInfo derives from MethodBase). If you can think of a way to extend my solution that it would cover constructors too, please e-mail me.

7 comments:

  1. I like your solution, smart :) It's pretty annoying when I have to dig trough exception stack to find place of original exception… it would be nice to have debugger option to do this, but since there isn’t one, this makes life quite easier :)
    Not sure now much will I use it in released applications, I’ll probably use some intermediate static method (I’ll add it to DirectInvoke class) that will decide on some global flag if to use standard invoke or your way… but trough development, I’ll use this a lot :)
    I have an idea how to make constructors work, is your email your blog username at gmail?

    ReplyDelete
  2. Previously i was using this:

    http://weblogs.asp.net/fmarguerie/archive/2008/01/02/rethrowing-exceptions-and-preserving-the-full-call-stack-trace.aspx

    It isn't a solution (debugger still breaks on outer exception), but is nice to know if you have to use standard invoke or exception information redirection in some other case.

    ReplyDelete
  3. Absolutely brilliant, works excellently. Thanks!

    ReplyDelete
  4. Just fixed a slight bug, by the way. Everything was working great until I had to dynamically invoke a method that had an array as a Parameter. I was getting an unhelpful "Record not found on lookup." when emitting the opcode to cast the Delegate. Sure enough, it is because names can't include [ or ] in them. A quick fix in the sig generation to trim out the brackets, and I was back in business.

    ReplyDelete
  5. Thanx!

    Is it thread safe?

    ReplyDelete
  6. This is a wonderful bit of code Timwi. Thanks so much for sharing, it will make users of an execution framework I developed so happy.

    ReplyDelete
  7. On the edge of my understanding... But gives me a stacktrace contining the Method that caused the exception, but no line or column. Bettter than nothing. Thanks
    John

    ReplyDelete