Tuesday 2 March 2010

Ensure code quality using a post-build event

This post
is about

C#

Motivation

While writing code, have you ever run into a situation where you got this uneasy feeling that there’s something you need to remember for later? Something where you need your code to remain consistent, but it’s not something the compiler can check for you? I’ll give an example so you know what I’m talking about. Suppose you declare an enum, something like this:

public enum Animal
{
   Cat,
   Sheep
}

And then you have a method that uses a switch statement on the enum, something like this:

public static string GetAnimalSound(Animal animal)
{
   switch (animal)
   {
       case Animal.Cat:
           return "meow";
       case Animal.Sheep:
           return "baah";
       default:
           throw new InvalidOperationException("Unrecognised animal: " + animal);
   }
}

Of course, you don’t want that InvalidOperationException to happen in your production code. But if you add a new value to your enum and forget to update that method, that is exactly what’s going to happen. And you knew this while writing the code; you knew that the compiler won’t remind you of this method when you add an enum value. In some simple cases, you can add a comment to the enum to remind yourself, but that still doesn’t stop you from simply missing that comment — and it breaks down if the enum is in a library and the method is in client code.

Solution

A more robust solution is to run an automated validity check and cause the build to fail if the check doesn’t pass. In other words, you cannot run the code while it is inconsistent. First, let’s write the validity check itself.

#if DEBUG
    private static void PostBuildCheck()
    {
        // Ensures that all Animal enum values are covered by Tools.GetAnimalSound()
        foreach (var animal in Enum.GetValues(typeof(Animal)))
            Tools.GetAnimalSound((Animal) animal);
    }
#endif

As you can see, this will simply call GetAnimalSound() on each Animal value and discard the result. The net effect is that the method will throw an InvalidOperationException if any of the enum values is not covered by the method; otherwise it returns normally.

Now we need something that calls this method. We don’t want to call the method directly; we want to be able to add PostBuildCheck() methods to various types as we see fit and not have to remember to enlist them somewhere. So we go through all types in the assembly and call all those methods via reflection:

#if DEBUG
    public static int RunPostBuildChecks(Assembly assembly)
    {
        bool anyError = false;
        foreach (var ty in assembly.GetTypes())
        {
            var meth = ty.GetMethod("PostBuildCheck", BindingFlags.NonPublic | BindingFlags.Static);
            if (meth != null && meth.GetParameters().Length == 0 && meth.ReturnType == typeof(void))
            {
                try
                {
                    meth.Invoke(null, null);
                }
                catch (Exception e)
                {
                    var realException = e;
                    while (realException is TargetInvocationException && realException.InnerException != null)
                    {
                        realException = realException.InnerException;
                    }
                    Console.Error.WriteLine(string.Format("Error: {0} ({1})",
                        realException.Message.Replace("\n", " ").Replace("\r", ""),
                        realException.GetType().FullName));
                    anyError = true;
                }
            }
        }
        return anyError ? 1 : 0;
    }
#endif

Things to note here:

  • We wrap all this code inside #if DEBUG. This way, the code will be absent from the Release build, lest all of you scream and shout that you’d be shipping redundant code.
  • The call to Invoke() has the habit of wrapping all exceptions in a TargetInvocationException. We unravel this to get to the real exception (in our example, the InvalidOperationException).
  • We output the error message to Console.Error and prefix it with the word "Error:". We will explain later why this is important.
  • We return 0 in case of success, 1 in case of failure. Maybe you can already guess where this is going.

Now obviously something needs to call this method. Add the following small piece of code to the beginning of your project’s Main() method:

#if DEBUG
    if (args.Length == 1 && args[0] == "--post-build-check")
        return RunPostBuildChecks(Assembly.GetExecutingAssembly());
#endif

And finally, edit your project properties, and on the Build Events tab, add the following post-build event command line:

    "$(TargetPath)" --post-build-check

Now add a value to the original enum and try to build your project. You should see an error message in the Error Window. The reason this works is because if the Main() method returns 0, the post-build event is considered to have succeeded; otherwise it is considered to have failed, and each line that was output to Console.Error is shown in the Error Window if it starts with "Error:" or "Warning:".

Enhancements

At the moment, the error message you get is not very useful. For one, it only says "Unrecognised animal", but it doesn’t tell you where the error occurred. The good news is, the original exception contains a stacktrace that we can use to find out where it was thrown. Instead of the simple call to Console.Error.WriteLine in RunPostBuildChecks(), use the following code:

    var stackTrace = new StackTrace(e, true);
    var stackFrame = stackTrace.GetFrame(0);
    Console.Error.WriteLine(string.Format("{0}({1},{2}): Error: {3} ({4})",
        stackFrame.GetFileName(),
        stackFrame.GetFileLineNumber(),
        stackFrame.GetFileColumnNumber(),
        realException.Message.Replace("\n", " ").Replace("\r", ""),
        realException.GetType().FullName));

This will cause your console output to look something like this:

Visual Studio correctly recognises this as a filename, line number and column number, allowing you to simply double-click on the error in the Error Window and taking you directly to where the exception occurred in the PostBuildCheck() method.

Advantages of this method

Here are some reasons why I like this idea:

  • You don’t have to create or maintain a new project. All the code is right there in the project itself.
  • Because it’s in the same project, and because you can place the verification code in any class (or struct), it’s trivially possible to access all the types, fields and methods in your project, including the private fields and methods of the class you’re testing.
  • A failing check tells you the place in the code where it is failing. It’s as good as a compiler error.
  • Everyone in your team who builds the project will run the tests. If someone in your team submits a change to revision control that violates a test, it is as inexcusable as submitting a change that doesn’t compile.
  • In theory you can put all your unit tests into a post-build check like this, removing the need for a separate unit-testing framework. In some situations this may be undesirable because it might make the build take too long, but you can do it in simple cases.

Further enhancements

There are some more ways in which I’ve enhanced this idea for myself:

  • The RunPostBuildChecks() method is actually in a library so I can re-use it in several projects.
  • Using exceptions allows each PostBuildCheck() method to return only one error. Therefore, I decided to declare an interface IPostBuildReporter, which has methods to output errors and warnings directly, and pass it as a parameter to the PostBuildCheck() methods. This way I can report any number of errors or warnings.
  • My implementation of IPostBuildReporter can search the source code for any string. Instead of outputting the location of the exception, I use that to output the location of the actual error. (This doesn’t make much sense in the above example because the location of the exception is the location of the error, but imagine a different example where a post-build check verifies that a certain set of classes in your project are all marked with the [Serializable] attribute or something like that. You want the error to point to the class that is missing the attribute, which could be anywhere in the source.)
  • To this end, I pass the $(SolutionDir) as a command-line parameter in the post-build event, allowing the IPostBuildReporter implementation to know where to look for the source code.

No comments:

Post a Comment