Improving Message Throughput in Akka.NET with Routers

One of the things I cover in my new Pluralsight course is the awesome feature of routers in Akka.NET.

Routers, with very little code/config change, allow us to spread messages across multiple instances of actors. This means we can process messages concurrently and thus improve the overall throughput of messages in the actor system.

Routers distribute message to a set of actor instance “routees”.

Router Types

Routers can be grouped into two categories: Group routers and Pool routers.

Group routers distribute messages to actors that have already been created in the actor system, this means that a Group router does not supervise its routees.

A Pool router on the other hand does create and supervise its routees.

Routing Strategies

Both Pool and Group routers use routing strategies to determine which routee(s) a message should be routed to. Not all routing strategies are available in both Group and Pool routers.

One of the routing strategies is the Round-Robin strategy. This strategy will route incoming messages in turn to each routee in order, then return back to the first routee and continue around all routees.

An Example

As an example, consider the following console application that simulates some work being done with a Thread.Sleep(200).

using System;
using System.Diagnostics;
using System.Threading;
using Akka.Actor;

namespace ConsoleApplication1
{
    public class CompletedSomeWorkMessage
    {
    }

    public class DoSomeWorkMessage
    {
        public DoSomeWorkMessage(int workItem)
        {
            WorkItem = workItem;
        }

        public int WorkItem { get; private set; }
    }

    public class WorkerActor : ReceiveActor
    {
        private readonly IActorRef _counterActorRef;

        public WorkerActor(IActorRef counterActorRef)
        {
            _counterActorRef = counterActorRef;

            Receive<DoSomeWorkMessage>(
                message =>
                {
                    Console.WriteLine("Working on {0}", message.WorkItem);

                    // simulate some work
                    Thread.Sleep(200);

                    _counterActorRef.Tell(new CompletedSomeWorkMessage());
                });
        }
    }


    public class WorkCounterActor : ReceiveActor
    {
        private int _workLeft;

        public WorkCounterActor(int workToBeDone)
        {
            _workLeft = workToBeDone;

            Receive<CompletedSomeWorkMessage>(
                message =>
                {
                    _workLeft--;

                    if (_workLeft == 0)
                    {
                        // all work is done so shutdown actor system
                        Context.System.Shutdown();
                    }
                });
        }
    }

    internal class Program
    {
        private static void Main(string[] args)
        {          
            const int totalWorkToBeDone = 20;

            var system = ActorSystem.Create("MySystem");
            var counter = system.ActorOf(Props.Create(() => new WorkCounterActor(totalWorkToBeDone)));
            var worker = system.ActorOf(Props.Create(() => new WorkerActor(counter)));


            var timeTaken = Stopwatch.StartNew();

            for (int i = 1; i <= totalWorkToBeDone; i++)
            {
                worker.Tell(new DoSomeWorkMessage(i));
            }

            system.AwaitTermination();

            timeTaken.Stop();


            Console.WriteLine("Took {0}ms", timeTaken.ElapsedMilliseconds);
            Console.ReadLine();
        }
    }
}

Running this application results in the following screenshot:

Akka.NET console application screenshot with no router

Notice in the preceding screenshot that to process 20 work items took about 4 seconds.

We can increase the concurrency in the system by adding a Round Robin Pool router.

The following modified code shows the creation of 5 worker actors as part of a pool.

var worker = system.ActorOf(Props.Create(() => new WorkerActor(counter)).WithRouter(new RoundRobinPool(5)));

Now when we execute worker.Tell(new DoSomeWorkMessage(i)) the message will be routed to one of five automatically created WorkerActor instances.

Running the program now results in an execution time of about 1.5 seconds rather than the non-router version that took about 4 seconds.

Akka.NET console application screenshot with a round robin router

To learn more about the different types of routers and other ways to improve message throughput check out my Pluralsight course: Improving Message Throughput in Akka.NET or check out the Akka.NET documentation.

Akka.NET Dispatchers and User Interface Thread Access

Akka.NET actors typically run in their own (non-UI) threads. This means by default accessing UI objects (for example setting some text in a text box) will cause a thread access exception.

For example the following code shows an actor trying to set the text of the button. The actor is not running in the UI context but the Button is so we’re trying to cross threads which causes the exception shown below.

Screenshot of Visual Stuid showing thread access exception in debugger

<Window x:Class="WpfApplication1.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="350" Width="525">
    <StackPanel>
        <Button Click="Button_OnClick">Say Hi</Button>
        <TextBox Name="GreetingTextBox"></TextBox>
    </StackPanel>
</Window>

public class HelloWorldActor : ReceiveActor
{
    public HelloWorldActor(TextBox textBox)
    {
        Receive<string>(s => textBox.Text = s);
    }
}
public partial class MainWindow : Window
{
    private ActorSystem _actorSystem;
    private IActorRef _actor;

    public MainWindow()
    {
        InitializeComponent();

        _actorSystem = ActorSystem.Create("MySystem");

        _actor = _actorSystem.ActorOf(Props.Create(() => new HelloWorldActor(GreetingTextBox)));
    }

    private void Button_OnClick(object sender, RoutedEventArgs e)
    {
        _actor.Tell("G'day");
    }
}

The thread on which an actor runs can be configured, either in code or in the app/web HOCON config.

We can modify the actor props in code as follows:

_actor = _actorSystem.ActorOf(Props.Create(() => new HelloWorldActor(GreetingTextBox))
                                   .WithDispatcher("akka.actor.synchronized-dispatcher"));

Now the HelloWorldActor will run on the UI thread and be able to access the text box as the following screenshot shows:

screenshot of WPF application running and updating the TextBox from another thread

 

MVVM and INotifyPropertyChanged

When using MVVM and a view-model, rather that passing the UI element to the actor, the view-model can instead be passed. The actor now updates the view-model which the textbox is databound to. In this situation the data-binding mechanism/INotifyPropertyChanged will take care of marshalling changes to the UI thread. This means that the actor no longer needs to be configured to run on the UI thread. The following code shows these changes:

<Window x:Class="WpfApplication1.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="350" Width="525">
    <StackPanel>
        <Button Click="Button_OnClick">Say Hi</Button>
        <TextBox Text="{Binding Greeting}"></TextBox>
    </StackPanel>
</Window>

using System.ComponentModel;
using System.Runtime.CompilerServices;
using System.Windows;
using Akka.Actor;

namespace WpfApplication1
{    
    public partial class MainWindow : Window
    {
        private ActorSystem _actorSystem;
        private IActorRef _actor;
        private ViewModel _viewModel ;

        public MainWindow()
        {
            InitializeComponent();

            _viewModel = new ViewModel();

            _actorSystem = ActorSystem.Create("MySystem");

            _actor = _actorSystem.ActorOf(Props.Create(() => new HelloWorldActor(_viewModel)));
            
            DataContext = _viewModel;
        }

        private void Button_OnClick(object sender, RoutedEventArgs e)
        {
            _actor.Tell("G'day");
        }
    }

    public class HelloWorldActor : ReceiveActor
    {
        private readonly ViewModel _viewModel;

        public HelloWorldActor(ViewModel viewModel)
        {
            _viewModel = viewModel;

            Receive<string>(s => _viewModel.Greeting = s);
        }
    }

    public class ViewModel : INotifyPropertyChanged
    {
        private string _greeting;

        public string Greeting
        {
            get { return _greeting; }
            set
            {
                _greeting = value;
                OnPropertyChanged();
            }
        }

        public event PropertyChangedEventHandler PropertyChanged;

        protected virtual void OnPropertyChanged([CallerMemberName] string propertyName = null)
        {
            var handler = PropertyChanged;
            if (handler != null) handler(this, new PropertyChangedEventArgs(propertyName));
        }
    }
}

You can read more about dispatchers in the Akka.NET docs and learn more about using Akka.NET with WPF in my Pluralsight course: Building Reactive Concurrent WPF Applications with Akka.NET.

EDIT: fixed up typo in source, changed to _actor.Tell("G'day");

Using Predicates in Akka.NET Receive Actors

When using the Receive actor API in Akka.NET, we can take advantage of overloads of the Receive method that allow us to specify predicates.

A message will now be handled if it is of the correct type and the predicate is met.

Take the following actor that sends emails with a different message if a customer is “high value”:

class WelcomeEmailSender : ReceiveActor
{
    public WelcomeEmailSender()
    {
        Receive<SendNewCustomerWelcomeEmail>(message => HandleMessage(message));
    }

    private void HandleMessage(SendNewCustomerWelcomeEmail message)
    {            
        if (message.IsHighValueCustomer)
        {
            SendEmail("Dear high value customer...", message.EmailAddress);
        }
        else
        {
            SendEmail("Yo! ...", message.EmailAddress);
        }
    }

    private void SendEmail(string greeting, string emailAddress)
    {
        Console.WriteLine("{0} to {1}", greeting, emailAddress);
    }
}

In the preceding code, the if statement is deciding what action is taken – the welcome message to be sent.

Using an overload of the receive method, we can add a predicate that replaces this if statement:

class WelcomeEmailSender2 : ReceiveActor
{
    public WelcomeEmailSender2()
    {
        Receive<SendNewCustomerWelcomeEmail>(
            message =>
            {
                SendEmail("Dear high value customer...", message.EmailAddress);
            },
            handleIf => handleIf.IsHighValueCustomer);

        Receive<SendNewCustomerWelcomeEmail>(
            message =>
            {
                SendEmail("Yo! ...", message.EmailAddress);
            }, 
            handleIf => handleIf.IsHighValueCustomer == false);
    }

    private void SendEmail(string greeting, string emailAddress)
    {
        Console.WriteLine("{0} to {1}", greeting, emailAddress);
    }
}

In the preceding code the predicate (represented by the lambda: handleIf => handleIf.IsHighValueCustomer) replaces the if.

In these example we only have two possible predicates because we are looking at the boolean “IsHighValueCustomer”. The following code shows predicates being defined for non-boolean (string) values:

class SomeActor : ReceiveActor
{
    public SomeActor()
    {
        Receive<string>(message => Console.WriteLine("Knock knock..."), handleIfMessage => handleIfMessage.Equals("Tell me a joke"));
        Receive<string>(message => Console.WriteLine("Hello There!"), handleIfMessage => handleIfMessage.Equals("Say Hi"));
        Receive<string>(message => Console.WriteLine("Woof"), handleIfMessage => handleIfMessage.Equals("Bark like a dog"));
    }
}

If we were to send this actor the following messages they would be received and handled:

IActorRef someActor = actorSystem.ActorOf<SomeActor>("Some");

someActor.Tell("Tell me a joke");
someActor.Tell("Say Hi");
someActor.Tell("Bark like a dog");

However, the following would not be received as it has no matching predicate (even though the message is still of type string):

someActor.Tell("Say bye");

To learn more about Akka.NET, check out the docs or my Pluralsight courses: Building Concurrent Applications with the Actor Model in Akka.NET and Implementing Logging and Dependency Injection in Akka.NET.

New Pluralsight Course: Implementing Logging and Dependency Injection in Akka.NET

If you’ve already watched my Akka.NET fundamentals Pluralsight course and want to learn more about using DI and logging, check out my new Implementing Logging and Dependency Injection in Akka.NET course.

Course Description

Akka.NET makes building concurrent and distributed applications easier. As with other architectures, Akka.NET based systems need effective logging for the monitoring and diagnosing of systems when deployed to production. Just because we use Akka.NET to get the benefits of the Actor Model, it doesn’t mean that best practices for object construction and dependencies such as dependency injection should be ignored. By the end of the course, you’ll understand how to implement effective logging in your Akka.NET system and how to use dependency injection to ensure the services your actors depend on are still provided in a loosely coupled and configurable way.

Switchable Actor Behaviour in Akka.NET

One of the things that actors can do is change their behaviour in response to an incoming message. This doesn’t however mean that we end up with a single “god actor” that has too many responsibilities. Whilst this may also seem like a potentially dangerous thing to do it is actually a very poweful concept. Because an actor instance only processes one message at a time, we don’t have to worry about this behaviour switch affecting the currently processing message, in fact the change in behaviour applies to the next message processed.

There’s a number of benefits to using switchable behaviours including the potential reduction/simplification of the amount of code we need to write and the ability for our brains to conceptualize what the the actor does more readily.

As an example, consider the following code that represents an actor that can respond in one of two ways to an incoming message depending on its current (boolean) state:

class LightSwitchActor : ReceiveActor
{
    private bool _isTurnedOn;
    public LightSwitchActor()
    {
        Receive<FlipTheSwitchMessage>(message => HandleMessage(message));
    }

    private void HandleMessage(FlipTheSwitchMessage message)
    {
        if (_isTurnedOn)
        {
            // do some processing when currently turned on and message received

            // flip the switch
            _isTurnedOn = false;
        }
        else
        {
            // do some processing when currently turned off and message received

            // flip the switch
            _isTurnedOn = true;
        }
    }
}

In the preceding code, the state of the boolean field _isTurnedOn determines how the actor behaves in response to an incoming FlipTheSwitchMessage.

This actor could be refactored so that the isTurnedOn field is removed and the different behavioural states become more explicit:

class LightSwitchBehaviourActor : ReceiveActor
{
    public LightSwitchBehaviourActor()
    {
        TurnedOff();   
    }


    private void TurnedOn()
    {
        Receive<FlipTheSwitchMessage>(
            message =>
            {
                // do some processing when currently turned on and message received                            

                // flip the switch
                Become(TurnedOff);
            });
    }

    private void TurnedOff()
    {
        Receive<FlipTheSwitchMessage>(
            message =>
            {
                // do some processing when currently turned off and message received                                  

                // flip the switch
                Become(TurnedOn);
            });
    }
}

Note the use of the Become() method which allows us to change how we respond to the next message that gets processed.

Whilst in these simple examples there are only two states (TurnedOn and TurnedOff) an actor could describe a greater number of potentially more complex behaviours.

For more information check out the Akka.NET docs or my Pluralsight course: Building Concurrent Applications with the Actor Model in Akka.NET.

Actor Models Come to .NET Developers with Akka.NET

Akka.NET is a port of the Java/Scala Akka framework.

In some ways, the emergence of Akka.NET could be seen as .NET truly “coming of age” - in the sense that other .NET technologies for building web, phone, services, apps etc. are great whereas the distributed .NET story has not been as fully realised – until now with Akka.NET and also Orleans.

In Akka.NET there are two primary constructs: actors and messages.

Actors do work and messages allow actors (and the outside world) to communicate. Actors don’t expose their internal state to other actors, all communication is done with messages.

Messages can be built-in .NET types such as a string or int, or can be custom types containing multiple data items.

Actors perform the work of the system and can essentially do one of four things:

  • Receive and respond to incoming messages
  • Send messages to other actors
  • Create new actors
  • Change behaviour in response to an incoming message

Actors have an incoming mailbox and they process one message at a time, then move onto the next message, and so on. If there are no messages to process the actor sits around being lazy, it only reacts to incoming messages.

The ability of actors to change their behaviour allows the simplification of actor code and also allows the building of finite state machines.

Actors can also be distributed across multiple processes/machines. Akka.NET makes this easier by providing “location transparency”, essentially not having to worry about changing our code just because we want to send a message to an actor in a different process. Akka.NET goes even further and can actually deploy instances of actors automatically to remote machines when they are created by simple modification of config – again requiring no code changes.

Akka.NET also supports the idea that systems can be “self-healing”. Through a hierarchy of supervision, parent actors supervise the child actors that they create. If a child actors errors, the parent supervising it can decide how to handle the failure. We can accept the default supervision strategy or create our own as necessary.

For more information on Akka.NET check out the home page, follow the project on Twitter, or check out my Pluralsight course: Building Concurrent Applications with the Actor Model in Akka.NET.

Better User Experiences and More Robust Applications Pluralsight Course

My new Pluralsight course was just published which shows how to use the open source Polly library.

Polly is a great library by Michael Wolfenden for configuring the automatic retrying of operations if exceptions occur. Transient errors such as the network being slow/unavailable for a small amount of time can normally result in errors surfacing to the user and/or into log files that need looking at.

By using one of the exception handling strategies that Polly provides, these transient exceptions can automatically be retried, preventing user errors and/or log entries.

There are four strategies:

  • Retry Forever
  • Retry
  • Wait and Retry
  • Circuit Breaker

Polly is configured in a fluent way, for example to use the Wait and Retry strategy to retry 3 times waiting 2, 4, and 6 seconds between retries:

Policy.Handle<some exception>()
      .WaitAndRetry(new[]
          {
            TimeSpan.FromSeconds(2),
            TimeSpan.FromSeconds(4),
            TimeSpan.FromSeconds(6)
          })
      .Execute(some action);

Check out the Polly GitHub repo, the NuGet package or the “Better User Experiences and More Robust Applications with Polly” Pluralsight course.

FeatureToggle 3.2 Released

FeatureToggle logo image

A minor release of my open source FeatureToggle library has just been released to NuGet.

Version 3.2 was a result of user requests to be able to configure toggles that get their values from sql server to be configured using the <connectionStrings> section of the .config file.

A Sql based toggle can now be configured a number of different ways

Using appSettings only

This is the original method:

<appSettings>
   <add key="FeatureToggle.SaveToPdfFeatureToggle.ConnectionString" value="Data Source=.\SQLEXPRESS;Initial Catalog=FeatureToggleDatabase;Integrated Security=True;Pooling=False"/>
   <add key="FeatureToggle.SaveToPdfFeatureToggle.SqlStatement" value="select Value from Toggle where ToggleName = 'SaveToPdfFeatureToggle'"/>
</appSettings>

Using connectionStrings Entry by Convention

<appSettings>
    <add key="FeatureToggle.SaveToPdfFeatureToggle.SqlStatement" value="select Value from Toggle where ToggleName = 'SaveToPdfFeatureToggle'"/>
</appSettings>

<connectionStrings>
    <add name="FeatureToggle.SaveToPdfFeatureToggle" connectionString="Data Source=.\SQLEXPRESS;Initial Catalog=FeatureToggleIntegrationTests;Integrated Security=True;Pooling=False" />    
</connectionStrings>

 

Using connectionStrings Custom Named Entry

This option allows multiple toggles to share a named connection if you have more than one sql toggle in the application.

<appSettings>
    <add key="FeatureToggle.SaveToPdfFeatureToggle.SqlStatement" value="select Value from Toggle where ToggleName = 'SaveToPdfFeatureToggle'"/>
    <add key="FeatureToggle.SaveToPdfFeatureToggle.ConnectionStringName" value="myAwesomeConnection" />
</appSettings>

<connectionStrings>
    <add name="myAwesomeConnection" connectionString="Data Source=.\SQLEXPRESS;Initial Catalog=FeatureToggleIntegrationTests;Integrated Security=True;Pooling=False" />    
</connectionStrings>

Thanks

Thanks to the following people for contributions and ideas:

Improving Test Code Readability and Assert Failure Messages with Shouldly

Shouldly is an open source library that aims to improve the assert phase of tests; it does this in two ways. The first is offering a more “fluent like” syntax that for the most part leverages extension methods and obviates the need to keep remembering which parameter is the expected or actual as with regular Assert.Xxxx(1,2) methods. The second benefit manifests itself when tests fail; Shouldly outputs more readable, easily digestible test failure messages.

Failure Message Examples

The following are three failure messages from tests that don’t use Shouldly and instead use the assert methods bundled with the testing framework (NUnit, xUnit.net, etc):

  • “Expected: 9  But was:  5”
  • “Assert.NotNull() Failure”
  • “Not found: Monday In value:  List<String> ["Tuesday", "Wednesday", "Thursday"]”

In each of the preceding failure messages, there is not much helpful context in the failure message.

Compare the above to the following equivalent Shouldly failure messages:

  • “schedule.TotalHours should be 9 but was 5”
  • “schedule.Title should not be null or empty”
  • “schedule.Days should contain "Monday" but does not”

Notice the additional context in these failure messages. In each case here, Shouldly is telling us the name of the variable in the test code (“schedule”) and the name of the property/field being asserted (e.g. “Total Hours”).

Test Code Readability

For the preceding failure messages, the following test assert code is used (notice the use of the Shouldly extension methods):

  • schedule.TotalHours.ShouldBe(9);
  • schedule.Title.ShouldNotBeNullOrEmpty();
  • schedule.Days.ShouldContain("Monday");

In these examples there is no mistaking an actual value parameter for an expected value parameter and the test code reads more “fluently” as well.

To find out more about Shouldly check out the project on GitHub, install via NuGet, or checkout my Better Unit Test Assertions with Shouldly Pluralsight course.

FeatureToggle Version 3 Released

FeatureToggle logo image

Version 3 of my open source feature toggle library is now released to NuGet.

Version 3 introduces some breaking changes and new features.

Breaking Changes

  • The WPF value convertor package has been removed for version 3 as it’s trivial to implement a version in individual applications and reduce the maintenance overhead for the porject
  • Version 3 removes support for Windows Phone 8.0 (you can still use version 2.x if you need this support)

New Features

Platforms Support

In addition to .NET framework 4 full, version 3 also supports:

  • Windows Phone 8.1 Silverlight apps
  • Windows Universal Apps 8.1 (Store and Windows Phone 8.1 WinRT)
  • Windows Store 8.1 apps

Future versions will add support for Windows 10 apps.

New Toggle Types

The EnabledOnOrAfterAssemblyVersionWhereToggleIsDefinedToggle (a little verbose I know) allows a feature to become enabled once a specified assembly version number is reached. The assembly version used is that of the assembly inside which the toggle is defined, rather that the executing assembly. The configuration value for this toggle would be (for example) 0.0.2.1.

Future version will add support for comparing the executing assembly version.

New Decorators

A new type of feature that version 3 introduces is the idea of toggle decorators. These are classes that “wrap” one or more toggles and provide additional features.

The DefaultToEnabledOnErrorDecorator and the DefaultToDisabledOnErrorDecorator allows a default value to provided if there is an error evaluating the wrapped toggle value, for example with a configuration error. These decorators should be used with caution as your application may end up in an unknown state if you forget to configure something properly. In some cases you may want this behaviour rather than the application failing.

Assuming that a feature toggle called MyFeatureToggle has been defined, to use the decorator the following code would be used.

IFeatureToggle printFeatureThatDefaultsToDisabled = new DefaultToDisabledOnErrorDecorator( new MyFeatureToggle() );

bool isPrintEnabled = printFeatureThatDefaultsToDisabled.FeatureEnabled;

These decorators also allow you to specify an optional logging action that gets called if the wrapped toggle throws an exception:

new DefaultToDisabledOnErrorDecorator(wrappedToggle, ex => Console.WriteLine(ex.Message));

 

The CompositeAndDecorator allows multiple toggles to be wrapped, only if all these wrapped toggles are enabled will the decorator return true.

IFeatureToggle andToggle = new CompositeAndDecorator(new AnEnabledFeature(), new ADisabledFeature());

var isEnabled = andToggle.FeatureEnabled; // returns false

 

The final new decorator is the FixedTimeCacheDecorator. This allows a toggle that gets its value from a slow source (such as remote database, http, etc.) to have its value cached for a fixed period of time. To use this you specify the wrapped toggle and a TimeSpan.

IFeatureToggle cachedToggle = new FixedTimeCacheDecorator(cachedToggle, TimeSpan.FromSeconds(10));

 

Check out the release notes on GitHub. If you find any problems or bugs or want to suggest other features, please create an issue on GitHub.