Cerrio

Wanted to let people know about a project am working on. Cerrio is a new type of Complex Event Processing System. We don't have a lot of info on the web yet but hopefully that will change that soon. The basic idea is to make dealing with real time data easy. Right now you can watch some videos about how to use the system, but hopefully in a the next couple months we will have a way for people to play around with it themselves.

SocketAsyncEventArgs Sending Timeout

Microsoft .Net 3.5 introduced a new way to do Async Socket operations the SocketAsyncEventArgs class. It was specifically designed for high performance application and over it works great. For the applications I have used it for it has significantly reduced latency for our operations.


But I do have one pet peeve about it. There is no way (that I have found) to timeout if a send operation is taking to long. The specific case I am running up against is that occasionally my data consumer will encounter an error where it will stop processing data out of the TCP buffer but not disconnect. The server keeps sending and everything looks normal on the server side until the TCP buffers fill up. Once that happens the next call to SendAsync will just end up in the abyss. The completed event of the SocketAsyncEventArgs will never be fired.


This ends up being a problem. I ended up having to add some monitoring logic on top of the socket sending to make sure I know when calls aren't returning. but this seems like something that should be handled by the framework. Hopefully this will get remedied soon.

IL Generation, Optomizations, Debuggers and AccessViolationExceptions

Recently I have been working with some code that dynamically generates IL. Before we get into the guts of the issue I was running into let me say that yes I know that generating IL on the fly is a terrible idea, it not something that I like doing. But in this case it is necessary.

We'll start with the code:

namespace TestApplication1

{

    class Program

    {

        static void Main()

        {

            string methodName = "TestMethod";

 

            AssemblyBuilder asmBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(

                new AssemblyName("DynamicAssembly"), AssemblyBuilderAccess.Run);

 

            ModuleBuilder moduleBuilder = asmBuilder.DefineDynamicModule("DynamicModule");

            TypeBuilder typeBuilder = moduleBuilder.DefineType("DynamicType");

 

            MethodBuilder method = typeBuilder.DefineMethod(methodName, MethodAttributes.Public);

 

            ILGenerator il = method.GetILGenerator();

 

            il.Emit(OpCodes.Newobj, typeof(TestClass).GetConstructor(Type.EmptyTypes));

 

            il.Emit(OpCodes.Callvirt, typeof(TestClass).GetMethod("StaticFunction", BindingFlags.Public | BindingFlags.Static));

 

            il.Emit(OpCodes.Pop);

 

            il.Emit(OpCodes.Ret);

 

 

            Type dynamicType = typeBuilder.CreateType();

 

            ConstructorInfo constructor = dynamicType.GetConstructor(Type.EmptyTypes);

 

 

            object o = constructor.Invoke(null);

            o.GetType().GetMethod(methodName).Invoke(o,null);

        }

    }

 

    public class TestClass

    {

        public static void StaticFunction()

        {

            Console.WriteLine("Static Hi");

        }

    }

}

Go ahead, paste into Visual Studio and hit F5, runs fine, StaticFunction is called, just like you would expect it to be.

Now instead of hitting F5 run it via Ctrl+F5:

System.Reflection.TargetInvocationException was unhandled
  Message="Exception has been thrown by the target of an invocation."
  Source="mscorlib"
  StackTrace:
       at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
       at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
       at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
       at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
       at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
       at TestApplication1.Program.Main() in C:\Documents and Settings\bwillard\Desktop\TestApplication1\Program.cs:line 38
  InnerException: System.AccessViolationException
       Message="Attempted to read or write protected memory. This is often an indication that other memory is corrupt."
       Source="DynamicAssembly"
       StackTrace:
              at DynamicType.TestMethod()
       InnerException:

What the heck? The same code ran, it should have yielded the same results. Well it turns out that running with a debugger attached causes the code that is generated not to be optimized (this is independent of weather you compiled with the optimize code flag or not). It is the same as if you had put the following code in the previous sample:

Type debuggableAttribute = typeof(DebuggableAttribute);

ConstructorInfo daCtor = debuggableAttribute.GetConstructor(new[] { typeof(DebuggableAttribute.DebuggingModes) });

CustomAttributeBuilder daBuilder = new CustomAttributeBuilder(daCtor, new object[]

                                                                          { DebuggableAttribute.DebuggingModes.DisableOptimizations |

                                                                            DebuggableAttribute.DebuggingModes.Default});

asmBuilder.SetCustomAttribute(daBuilder);

Mystery #1 solved, but why doesn't the code work when not run in a debugger. This is a fairly straight forward bug. You can't Callvirt on a static method, the line should read:

il.Emit(OpCodes.Call, method);

Took me a while to figure those two things out.

I'll leave you with one more bonus tip about emiting code. If you are getting the exception:

System.Reflection.TargetInvocationException was unhandled
  Message="Exception has been thrown by the target of an invocation."
  Source="mscorlib"
  StackTrace:
       at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
       at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
       at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
       at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
       at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
       at TestApplication1.Program.Main() in C:\Documents and Settings\bwillard\Desktop\TestApplication1\Program.cs:line 49
       at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)
       at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
  InnerException: System.InvalidProgramException
       Message="JIT Compiler encountered an internal limitation."
       Source="DynamicAssembly"
       StackTrace:
            at DynamicType.TestMethod()
       InnerException: 

What it means in this case is that there is an extra object on the stack when the function returned. That is why I need the last "pop" in my sample. Because I am calling a static function it doesn't consume the TestClass object that I created and that is now at the top of the stack.

Adding Search Paths to a Domain

Recently I was trying to load an assembly, call it assembly A, into my AppDomain and execute code in the assembly. This is usualy pretty easy thing to do, the problem comes from the fact that assembly A depends on another assembly B that is in the same directory as A.


If I was creating a new AppDomain this would be easy I could just add the search path to the AppDomainSetup when I create the domain. But in this case I wanted everything to be in one app domain so I couldn't do that.


The solution (that I found, there might be an easier way in which case please let me know) is to leverage closures and the AssemblyResolve event:

public static void LoadPackage(string dllPath, string className)

{

    string dllFullPath = Path.GetFullPath(dllPath);

    var assembly = Assembly.LoadFile(dllFullPath);

    var package = assembly.CreateInstance(className);

    if (null == package)

    {

        throw new Exception("Couldn't load type " + className + " from " + dllPath);

    }

    var asembliesInDirectory = new Dictionary<string, string>();

    foreach (string file in Directory.GetFiles(Path.GetDirectoryName(dllFullPath), "*.dll"))

    {

        try

        {

            asembliesInDirectory.Add(AssemblyName.GetAssemblyName(file).FullName, file);

        }

        catch{}

    }

    AppDomain.CurrentDomain.AssemblyResolve += (sender, eventArgs) =>

    {

        if (!asembliesInDirectory.ContainsKey(eventArgs.Name))

        {

         return null;

        }

        return Assembly.LoadFile(asembliesInDirectory[eventArgs.Name]);

    };

}

The AssemblyResolve event gets called when the CLR find an assembly it needs to load. Handling this event lets you add your own assembly resolving logic into the process. This code will check the directory you loaded your first dll from any time the CLR can't find a dll. The reason for the dictionary is purely for performance, it is expensive to scan the directory every time you are trying to load something. So this just pre-caches all the assembly names for easy lookup.

Programaticly Configuring NServiceBus

I just started implementing a PubSub system for our company using NServiceBus. Overall if has been super easy to setup. The samples are a great starting point and everything is pretty self explanatory.


However as I was trying to roll it out into production I realized that using the standard Spring configuration wasn't going to work for us. I wanted the name of the Msmq input queue to be specified programatilcy instead of via a config file. I knew it had to be possible but I had a really tough time figuring out how to do it. All the samples I could find were about how to set things up with Spring. So I though I would share how I did it.


The standard example shows:

NServiceBus.Configure.With()

    .XmlSerializer()

    .MsmqTransport()

        .IsTransactional(false)

        .PurgeOnStartup(false)

    .UnicastBus()

        .ImpersonateSender(false)

        .LoadMessageHandlers()

    .CreateBus()

    .Start();

So to dynamically set the MsmqTransoprt values you can write the following:

Configure configure = NServiceBus.Configure.With().SpringBuilder();

MsmqTransport transport = configure.Configurer.ConfigureComponent<MsmqTransport>(ComponentCallModelEnum.Singleton);

transport.InputQueue = "DynamicQueueName";

transport.NumberOfWorkerThreads = 1;

transport.ErrorQueue = "DynamicErrorQueue";

transport.MaxRetries = 5;

configure

    .XmlSerializer()

    .UnicastBus()

        .ImpersonateSender(false)

        .LoadMessageHandlers()

    .CreateBus()

    .Start();

This works for me, I have no idea if that is the best practice or if there is a better way to set these values programaticly. Hopefully this helps you and saves you from some the headache I went through (it always seems pretty simple once you get it figured out). Please let me know if you know a better way to do this.