Share |

Framework.SuperPool

From Matrix Platform

Jump to: navigation, search

Documentation Framework.SuperPool



Contents


Overview

What is the super pool?

The super pool is a framework for decoupled communication and management of components. It relies internally on a Message Bus system, but fully hides its complexities away from the user – no messages are ever visible to the user of the super pool. Instead, all operations are performed using syntax similar to that of a direct call to an object. Here is a very basic example:

   client.CallSync<IMyInterface>(otherId).MyMethod();

is similar to:

   other.MyMethod();

However there are 2 major differences:

  • The super pool call is done entirely trough an Id, so we do not hold a direct reference to the other object. This allows for full decoupling between the object that orders the execution and the object that receives it.
  • The super pool call can be made to remote objects.


Another significant difference is that the majority of super pool calls are geared toward asynchronous operations. Although there is full support for both synchronous and asynchronous modes, a most notable advantage the framework has over similar solutions is its flexible and coherent ability to perform multiple operations at the same time trough placing asynchronous requests and receiving results.


The invocation model of the super pool is based around the usage of interfaces; they are a mandatory part of the communication process. Interfaces are the only part that an object exposes to the super pool, and the only part that the super pool uses to communicate to the object.


Besides synchronous communication, the super pool has a few more “services” that it provides its member components with:

  • Service discovery, allows establishing what component provides what service (i.e. implements what interface)
  • Remote invocation, allows executing operations on remote components, trough TCP/IP connection and a server-client model of connection.
  • 7 different modes of invocation (synchronous and asynchronous, addressed to one or many recipients or non-addressed, direct or indirect)

What is the super pool from a technical perspective?

The super pool is a thread pool, message bus, remote invocation framework (allowing to cross a class, application domain, application or computer boundary) and a component container all combined and designed to work together. The actual implementation has all of the above elements as separate standalone items, so each one can be used on its own or replaced or improved. However the Super Pool is the top level API, and all the complexities of these underlying layers are hidden away – they operate “under the hood” of the system. This allows a smooth coding experience, very similar to that of directly working with the objects (direct referencing), while maintaining the full decoupling coding standards.


The Matrix Platform

The super pool is only one of the few systems that the Matrix Platform provides. The Matrix Platform is an Open Source (LGPL) software development platform, entirely written in C# .NET 2.0. Its goal is to encompass many commonly used functionalities in a single software package and allow the user to take this for granted when staring a new system.


Introduction

Why you need the super pool?

First and foremost the Super Pool introduces natural asynchronous communication environment into your solution that can be fluently spread over different components, threads, processes or even computers or networks. The following diagram demonstrates it:

The a-synchronicity is not mandatory, and the Super Pool can be used as synchronous communication framework too, however it is strongly advisable to make use of the multi-threading capabilities.


A modern .NET solution needs a few things, to make sure it will be able to keep growing over time and remain sustainable; here are a few:


  • Thread pool

It takes care of running many things at the same time; with the advancement of CPU technology the average number of threads in modern hardware will only grow so it is becoming critical to make your applications be able to perform multiple tasks in parallel.


  • Message transport

It allows components (or services) to talk to each other fluently. Even if you are not building a distributed application, messages are a great solution for designing complex applications, since they force a level of "independence" (or decoupling) between the parts of your application. Decoupling is crucial for a component based solution.


  • Component container

It holds references to and manages the components, also provides them will additional support services, for ex. “service discovery” functionality. The super pool brings all of the above together (and more) under one roof. It is a great foundation for building coherent software. It is also surprisingly small - the standalone DLL being only 195KB in size.


A few words on design

The super pool is a solution that tries to encompass the flexibility of a messaging system with the ease of use of direct referencing. It was designed from scratch, to be a flexible, dynamic, fast and distributable framework.


What is so special about it is it allows fine grained control of each call a component makes to other components (classes) of the solution. Since the components in the super pool are detached from each other, they can execute their tasks independently; when needed obtain results from each other in a synchronous or asynchronous fashion.


Along with the decoupling, comes the power of message based communication. It allows executing everything not only locally, but also remotely – to other parts of the solution that are connected using the TCP protocols, and this may be hosted anywhere. This is done trough no change in the execution syntax whatsoever, so the ordering component has no knowledge where the receiving component is located. This gives ultimate freedom to where each component is executing, and gives a natural environment for distributed applications.


The predecessor of the super pool was a purely message based communication solution, that combined both messaging and invocation control. The super pool extends this by adding the ability to directly use interface calls, event subscriptions etc. This is much more user friendly as compared to the direct work with messages, that requires to construct, send and receive the actual messages in your application code. Such a great improvement allows to omit large portions on tedious repetitive code. Still full access to the underlying messaging infrastructure is also available, so it can be used directly where needed. The message infrastructure is a very good solution on its own and although it has been developed especially to fit the performance needs of the super pool it can be used on its own very successfully.


Lastly, this framework has been built with the idea that multi-threading must be introduced to a solution in top to bottom fashion, meaning first calls between components must be done in parallel prior to parallelizing bottom level functionalities. Certainly access to a-synchronicity on every level is most beneficial.


Quick operation overview

The following diagram gives an overview of the structure of a super pool setup. The configuration shown consists of 2 super pool instances (named “A” and “B”), connected with a TCP.IP connection. This allows all components to communicate fluently with each other, fully disconnected trough the usage of messages, and with full support of asynchronous tasks, trough the usage of thread pools.



All of the elements of this configuration come as standard parts of the framework. No additional code is needed to run it.

A “Client” instance connects the component and the super pool

In order to have maximum flexibility, a class that becomes an active part of the super pool (aka. Component) does not need to have anything special. There are no requirements to implement or inherit a specific “client” class or interface. The entire communication between a Component and the pool is done trough an instance of the SuperPoolClient class we call “Client”. Although it is not mandatory to have this instance as part of your component class, it is often the best place to keep it there, so that it is easily accessible to all the methods of your class. The framework suggests that each component should have one corresponding client, but other configurations are also possible.



Let’s look at an example – here is a simple class that wants to be an active the super pool component, and implements a simple interface:

    [SuperPoolInterface]
    public interface ISomeInterface
    {
        void ReceiveSomeInfo(string info);
        void DoSomeWork();
    }
 
 
    public class MyComponent : ISomeInterface
    {
        public SuperPoolClient Client { get; set; }
 
        /// <summary>
        /// Constructor.
        /// </summary>
        public MyComponent()
        {
            Client = new SuperPoolClient("MyClient", this);
        }
 
        /// <summary>
        /// Send a request to "other" to do some work.
        /// </summary>
        public void RequestSomeWork(ClientId otherId)
        {
            Client.Call<ISomeInterface>(otherId).DoSomeWork();
        }
 
        #region ISomeInterface
 
        public void ReceiveSomeInfo(string info)
        {
            // ... someone send us some info.
        }
 
        public void DoSomeWork()
        {
            // ... doing work.
        }
 
        #endregion
    }

This class can order to have work done to another component, through its RequestSomeWork method. It can also receive work requests from other components, in its DoSomeWork() method. Thus is allows both – to be active and passive element of an operation.

An example: the client in action

Now that we have our client, let’s see how to put it to work.

First we need to create the super pool and simply add the clients to it:

    // Create the pool.
    Super.Core.SuperPool pool = new Super.Core.SuperPool("MyPool");
 
    // Create component 1 and 2.
    MyComponent component1 = new MyComponent();
    MyComponent component2 = new MyComponent();
 
    // Add them both to the pool (using their client instances).
    pool.AddClient(component1.Client);
    pool.AddClient(component2.Client);

The following diagram shows what happens when a simple asynchronous call is made from one component to another, along with the source code executing on both locations:



What the last line does is instruct the pool to do an asynchronous invocation of the Method() method from the I2 interface, on this object (so for the call to succeed, the this object must implement the I2 interface).

If however, you have no idea on whom of the clients implements what interface (and are too lazy to ask the super pool), you can simply invoke it like this (with no specific recipient client specified):

    client2.Call<I2>().Method();

This will make the system promptly invoke all clients that have implemented this interface and it will do it asynchronously and in parallel. So imagine you have a 4 core CPU, and with to perform calculations on each of the cores. All you need to do is create 4 calculation components (let assume they implement ICalculator) and do an unaddressed call:

    client2.Call<ICalculator>().Calculate();

The fact that the super pool handles starting and management of threads, means your actual components do not need to. The component must only be aware of the fact multiple threads may enter its methods, and so lock its resources where needed to protect from collisions or other mishaps. Note: there are quite a few demonstrations, samples and unit tests that show the operation of the framework inside the full Matrix Platform solution source code.

Functionality Overview

Coupling

According to coupling terminology, the default type of type coupling used is either “Content coupling” (tight, direct usage class references) or “External coupling” (loose, usage of references trough common interface). The current version super pool provides a “Message coupling” (lowest coupling possible) based model, combined with “External coupling” (aka. loose) model for improved usability. The resulting coupling is very loose, with no direct references between two communicating elements – the only thing in common being the Interface that defines the communication. Future versions of the framework may also provide a more primitive “Message coupling” based mode of operation, where no common interface is used, but instead calls are done trough a “soft” binding mini-framework. This mode will be suitable in environments where sharing a common interface is not desired, or when required to do communication with other non-CLR languages versions like Java or even Win32 based C++; also consuming Web services is an option. The changes introduced in the latest version of the .NET DLR system provide a very good starting point for designing these new models of operation.


Subscription

Besides invocation, the Client instance allows to subscribe to events that come from any element on the pool. This is one of two callback mechanisms, the other one being simple implementation of a callback interface, which gets invoked by the event generator. Here is a sample on how to perform subscription to an event:

    Client.Subscribe<IPoolInterface>(id).Event += new MyDelegate<string>(InterfaceImplementor_Event);

This will perform subscription to the Event event, of the IPoolInterface, that is expected to be implemented by the element with Id “id”. It is also possible to subscribe to an event, raised by any element – to do this use the SubscribeAll<>() method of the Client instance.


Asynchronous Results

The framework provides a way to receive one or many results asynchronously, trough the usage of a delegate passed in the “Call()” method. The delegate will be invoked every time a result has been received from the requesting call. Also in case an exception occurred during the execution of the call on the executing side, this exception will be provided here (note: make sure to user serializable exceptions when executing calls with remote execution).

Here is an example of asynchronous results consummation:

    AsyncCallResultDelegate delegateInstance =
       delegate(ISuperPoolClient client, AsyncResultParams parameters)
       {
           if (parameters.Result != null)
          {// Do something with result.
             string resultString = parameters.Result.ToString();
           }
        };
 
    Client1.Call<ITestInterface>(Client2.Id, delegateInstance, 152).AsyncResultMethod(1500);

Fail safety and error tolerance

One of the strong advantages of using the framework is its very high level of fault tolerance. This means requesting calls to components that are no longer available or connected is seen as a fairly trivial event, something common in live dynamic systems distributed. By default the system will only log the error (if Diagnostics system is available and logging is enabled) and return a result, in case the call is a synchronous one.


To allow for a more flexible approach, there is also a special call type named “CallConfirmed”. This type of call will try to (synchronously) make sure that the receiving party received the call, although it will not wait for the result of the call to become available. There is also a timeout assigned, as with other synchronous operations, to indicate the maximum time interval to wait for confirmation from the receiver, that the call was received.


The current implementation of the framework relies heavily on using the diagnostic system for fault detection and diagnostics. Although the super pool is perfectly capable of running with no diagnostics system, and is actually a bit faster this way, catching errors may prove more difficult in this way. Future versions will possibly extend in this area to allow for other options of diagnostics, besides the default system.


Execution modes, models and strategies (7 major modes of operation)

The super pool aims to make asynchronous execution easy. When an asynchronous call is made, it arrives at its destination and enters a queue of operations waiting to be executed. The execution is done on a thread pool, and is controlled trough an “execution strategy”. The strategy encompasses the thread pool to allow more flexibility and also to allow sharing thread pools. Both the thread pool and execution strategy are interchangeable, so if a given task requires specific thread management, these two can be designed or set up to provide that needed functionality.

The following diagrams describe the different models of invocation. Those are in the heart of the super pool framework ability, so it is important to understand what each does:









A few words on implementation techniques

The implementation of the super pool contains a few fairly advanced techniques. It utilizes both hot-swapping (see specialized article) and IL code generation (both Dynamic Method for event capturing and a full on Type Builder for interface dynamic proxy classes). It does all that to be as fast as possible in runtime. Also the source code has been structured neatly, with attention to detail and separating each part based on its responsibilities, so that reading and maintaining it is easy to do.


Flexibility

By its very nature the framework is very flexible. It allows swapping and replacing elements of the solution, moving them around and adding or removing them on runtime.

It is also very flexible when it comes to its own behavior; let’s look at an example of this:

Say you want to have a client of the super pool that executes its tasks not on the default .NET framework thread pool, but on a custom one.

All you need to do is assign your client with one of the custom predefined execution strategies like this:

    /// <summary>
    /// Creates a new client, assigns it with a custom execution strategy and adds it to the super pool.
    /// </summary>
    /// <param name="superPool"></param>
    public void Demonstrate(Matrix.Framework.Super.Core.SuperPool superPool)
    {
        SuperPoolClient client = new SuperPoolClient("Client", this);
        client.SetupExecutionStrategy(new CustomExecutionStrategy());
 
        superPool.AddClient(client);
    }

The purpose of the execution strategy is to distribute and control the execution that occurs upon a given client. For ex. if you wish to only allow one single execution thread to enter your client at a certain moment (or in some other way throttle the amount of executions), you can place this restriction in your execution strategy. Here is a very simple, yet fully functional custom execution strategy that relies entirely on the .NET Thread Pool to execute the items as they come.

    public class CustomExecutionStrategy : MessageBusClientExecutionStrategy
    {
        protected override void OnExecute(Envelope envelope)
        {
            // Process the incoming request, we will simply execute in on default thread pool.
            WaitCallback del = delegate(object state)
            {
                Client.PerformExecution((Envelope)envelope);
            };
 
            ThreadPool.QueueUserWorkItem(del, envelope);
        }
    }

There are 2 pre-provided execution strategies in the framework:


  • One is based on the .NET framework thread pool (much like the above sample) called FrameworkThreadPoolExecutionStrategy


  • The other using the custom thread pool implementation called ThreadPoolFastExecutionStrategy; the custom thread pool was designed for maximum speed, and increased flexibility over the default implementation and so is the execution strategy; it is useful if you want to achieve absolute maximum performance trough fine tuning the system, if you wish to have a dedicated thread pool for a single client or if you wish to control the thread apartment of the executing threads (needed when performing COM calls)


Is it dynamic?

Fully dynamic, all the components of the system can be added and removed on runtime. In order for the call/invoke operations of the system to be as fast as possible, hot swapping is often used for storage of the components references. What does this mean? It means adding or removing clients to and from the system is relatively slow (for ex. it takes about 20ms to add a new client), but this allows for all other operations to be blistering fast, since no locking is applied when they are executed.


Performance overview

The framework is rather fast. On a standard dual core machine the current pool implementation can do around 0.5 million indirect decoupled asynchronous calls per second. The number for direct decoupled synchronous calls is about 2.5-3 times as much. This should easily suffice in the speed requirements of the vast majority of applications where component framework be applied. Remember, this is calls that are done from one component to another, not one class to the other. A component typically has multiple classes.

The super pool provides inter-component communication, not inter-class. If you have a very call intensive solution to build, you may want to consider where you place your component / class / module boundaries. If higher speed is needed, you can also use the advanced functionality of the framework to obtain direct references to other local components (and use them for direct calls); however do so with caution, since this will interfere with a fully decoupled design.

Due to its nature the framework "opens up" your application to extensive multi-threading usage (each call executed on its own thread), the wave of highly-multi core processors (4 or more) become available will be put to good use and the intrinsic value of your solution based on the super pool architecture will natively grow further.

Since the default Super Pool remote transport connection is a TCP.IP with binary serialization, it is fast (compared to other ways of remote invocation) and can transport more than 10K messages (calls) per second, depending allot on machine and network speed.


Assemblies

This diagram shows how the assemblies of the framework stack up to each other.


An overview of the assemblies in the platform


Configuration

The super pool does not require any external source of configuration, and it also runs nicely with its defaults values. All configuration of the system is done trough code. This allows for better flexibility – with configuration done trough code you can easily extend it to have it done trough a configuration file, or whatever other form of configuration you choose to have.


The framework is designed to run smoothly “out-of-the-box”, the most notable configuration required is the port settings, when building a Server-Client setup; here is an example how to do that (you can see this in detail in the Matrix.Framework.SuperPool.Demonstration project). The example takes a few steps:


1) Create server side super pool, the server will listen at port:19452; the last parameter controls access control (in case a user name & password are required for a client to connect):

    ServerMessageBus messageBus =  new ServerMessageBus("Server", 19452, null);
    _pool = new SuperPool(messageBus);


2) Create client side super pool

    IPEndPoint endPoint = new IPEndPoint(IPAddress.Loopback, 19452);
    ClientMessageBus messageBus = new ClientMessageBus(endPoint, this.ClientName, null);
    // Initialize the super pool with this message bus.
    _pool = new SuperPool(messageBus);

3) (Optional) Configure an execution strategy for precise control over execution on each client.


The purpose of the execution strategy is to define the type and count of execution threads, speed and order of the calls coming in to a component. It can also be used to perform “throttling” so that the component does not get overloaded. To achieve these tasks, one can use a pre-existing or a completely custom execution strategy; in this sample we use one of the existing options.

Assigning an execution strategy is entirely optional, since the system defines a default execution strategy for each client, however we do it to show the extended configuration capabilities of the framework.

    _poolClient.SetupExecutionStrategy(new FrameworkThreadPoolExecutionStrategy());

This will assign the default Framework thread pool execution strategy to this client, meaning tasks that come to it will be performed using this strategy. It relies on using threads from the .NET Framework Thread Pool. The other default option is using the ThreadPoolFastExecutionStrategy, that relies of a fast custom implementation of a thread pool, providing more advanced control over execution. See the Technical details section for more information on this topic.


Technical Details

Standalone Dll

You can use the framework by referencing all of the dlls of the solution. However, if you wish to keep your solution compact, and with minimal number of external references, you can use the Standalone Dll. It combines all the parts that are required to run the Super pool into one single Dll file, for the purpose of convenience. The only functionality that it misses is the diagnostics capabilities that are optionally provided by the framework.


Parameters

By default parameters of methods are transported by reference, even when sending to many receivers simultaneously, which means multiple threads may end up accessing the same parameter at the same time. Parameters are serialized when sent over TCP.IP connection. By default the .NET framework binary serializer is used, so make sure to have any custom types compatible with it, should they be part of calls to remote component.


Serialization models

The serialization is handled by the Message Bus sub-framework, however its operation also concerns the types of classes used in the Super Pool communication, since some types of communication require that the parameters be serialized (most notably when sending to a remote node trough TCP). Currently the serialization relies entirely on the default .NET framework binary serialization model. Future versions may also provide built-in support for Protobuf-net and JSON.net frameworks. You can also provide your own serializer by implementing the ISerializer interface and assigning the instance to the MessageBus constructor.


Communication Interface Requirements

The only requirement to an interface so that it be considered a “service” in the super pool framework, is to have it market with the [SuperPoolInterface] attribute. This is needed to evade automatic subscription of unwanted interfaces. The next version of the framework will also provide a way to go around this requirement using a special configuration feature.

Also it is important to note, the framework does not support ref and out method parameters, so make sure to omit those from super pool consumed interfaces. There are a few reasons for this, most notably the problems that arise from having a method provide multiple results, other than its actual result.


Exception handling

The super pool allows the caller to examine an exception that has been generated by an execution receiving method. If an exception has occurred, it is stored in the responding call data.


Exceptions

It is advisable exceptions thrown to be serializable so they can be delivered back to caller, in case the caller is connected to a remote location.


Direct calls and execution order

Unlike all other types of calls, a Direct Call executes synchronously and instantly. This may result in having a Direct Call execute before a previously placed call, since all other types wait on a queue and a thread to be free to execute them. This issue is only possible where Direct Calls are mixed with other types of calls.


Features Not Supported

The current version of the framework does *not* support ref, and out parameters. In case a call is made to such a method, a 'NotImplementedException will be generated. There are currently no plans to provide support for these parameters, since they interfere additionally with the result from executing a method; when using a distributed execution framework, results occur in many different forms and scenarios (for ex. asynchronous execution will typically provide none), so providing support for out and ref parameters will significantly increase complexity both of implementation and of usage. The usage of *params* parameters are supported.


Application Scenarios

Super pool and .NET 4.0

The .NET 4.0 brings multiple improvements and new features in the area of asynchronous programming. The Super pool is fully capable of taking advantage of those. For ex. the standard .NET thread pool has been optimized to work faster on multi-core machines. To take advantage of this optimization, simply compile the Super pool with .NET 4.0 support and use the default .NET Thread pool as execution strategy.


Super pool as Event Aggregator (Event Broker)

The super pool can also operate in the role of an event aggregator. Although the actual implementation details vary, the same operations can be performed as in an aggregator.


Super pool as .NET Remoting replacement

The super pool can also serve as a replacement for the .NET Remoting framework in the cases where the Remoting is used as object oriented solution. The super pool has the advantages of being highly optimized and fully open source. It also provides multiple different types of invocation for a more detailed level of invocation control.


A Distributed Execution Framework with excellent bridging and RMI capabilities

Can serve as a bridge between separate modules, applications, even distributed across computers; future plans include addition of Java bridging support, so that .NET calls can be executed to JAVA and vice versa.


Component Service Discovery – A service locator system

The super pool has a functioning service locator system. To access it, the component can use the Client.Resolve() methods, or directly query the SuperPool instance (trough GetInterfaceImplementors() method). These allow to obtain identifiers of objects that implement the requested services (i.e. interfaces).


A communication framework and an IoC container

The super pool shares allot of common responsibilities with an IoC container, still there are allot of differences. Most notably, the super pool first responsibility is communication, where an IoC container is geared more towards organization of components. Where the IoC container typically relies on a “boot-strapper” piece of code to instruct what component uses what instance to perform its operations, the super pool leaves this decision to the actual modules, thus allowing a greater flexibility.

A critical difference is the super pool provides full decoupling of one module to another, meaning during normal usage, no module will store an actual reference to another object.

Also the super pool is designed to have a flat, more “natural” learning curve, as opposed to an IoC where the shift of taking away essential control from a component is rather severe and can be easily confusing at times. Finally, due to its “communication first” design, the super pool is more suited toward building active dynamic systems, where components come and go all the time and where access to some of the parts of the system may be unreliable.


The v.2 of the Matrix Platform shall be extended with additional IoC features, providing full support for this mechanism.


One Framework instead of five

As seen in this article, the super pool is a universal and versatile base for development. Finally, using one framework to cover a wide array of tasks offers the great advantage of offering a much reduced learning curve. Without the need to worry about integrating the separate instruments required for a successful application, one can truly focus on delivering the actual productivity and features that one is after.

modified on 6 July 2010 at 14:26 ••• 40,242 views