Introduction to the Circuit Breaker pattern

Posted on March 6th, 2008

---

Download the code (14kb)

I was reading the book Release It! about a month ago and ran across this pattern for creating code that has the ability to essentially shut itself off.  It just so happens that I had a use for this type of process and it helped me to create my final design.  What I was working on was a component that used a third party webservice for batch processing.  In this batch we would send over anywhere from 50-1500 web request one right after the other.  As such if we had any problems on the service end we could have thrown 1500 or even more error messages depending on how it was coded.  Of course we did not want that as it could potentially cause even worse problems by letting our app continue to try and process our data even if the service was down.  So instead of needlessly killing each of our apps I took a queue from the book and implemented the circuit breaker pattern. 

Now you might ask what is the circuit breaker pattern?  Well lets take a look at a real world scenario that everybody is familiar with.  Every home today has a main power box that is the central point at which power enters the home.  Now depending on how old the home is you could either have circuit breakers, fuses, or even worse a piece of metal jammed between the connections (I’d hope not).  Now fuses and circuit breakers all work the same way and for the same reason, to regulate the amount of load that is allowed over the electrical wire.  This is important because if there were no limit then it would cause an electrical short somewhere and possibly a fire.  It happened a lot before the fuse was created.  So, the use of a fuse or circuit breaker is meant to make sure we don’t put too much of a load on the circuit or wire.  Once a threshold is met then the circuit is tripped or the fuse is blown.  Thus stopping the flow of power.  Now if we look at our code base we want the same thing to occur.  So if I were to start sending records to the service in question and they start constantly failing as long as we have setup a threshold for error counts or failures we can then trip our circuit so to say and stop the flow of data to this service. 

Well you might ask how can we do this?  In our code bases is really not all that difficult to implement once you get the concept down.  So let’s look at some code to create a small example application.  Now you don’t have to create an interface but I wanted to really show what you need to do to add this functionality to your existing code.

public interface ICircuitBreaker
{
    void BreakCircuit();
    void ResetCircuit();
    bool CircuitBroken { get; set; }
    int Threshold { get; set; }
}

Next I’m going to embed an entire class with the interface implemented.  To show how this works I have created a class that writes to a file.  If there is a problem writing to the file for a specified number of tries then it will fail gracefully vs throwing an exception.

public class FileWriterWithCB : ICircuitBreaker
  {
    private bool _circuitBroken;
    private int _threshold;
    private string _fileName;
    private int _batchSize;

    public FileWriterWithCB()
    {
      this._circuitBroken = false;
      this._threshold = 5;
      this._fileName = "CBTest.txt";
      this._batchSize = 50;
    }

    public string FileName
    {
      get { return this._fileName; }
    }

    public int BatchSize
    {
      get { return this._batchSize; }
      set { this._batchSize = value; }
    }

    public void WriteToFile()
    {
      FileStream fs = new FileStream(this._fileName, FileMode.Open, FileAccess.ReadWrite);

      StreamWriter sw = new StreamWriter(fs);

      string data = "This is a test file write";

      sw.Write(data);

      sw.Flush();
      sw.Close();
      fs.Flush();
      fs.Close();

      sw = null;
      fs = null;
    }

    public void RunBatchWithCircuitBreaker()
    {
      int failureCount = 1;

      for (int x = 0; x <= this._batchSize; x++)
      {
        if (failureCount > this._threshold)
        {
          Console.WriteLine("Threshold reached breaking circuit.");
          this.BreakCircuit();
        }
        if (this.CircuitBroken)
        {
          Console.WriteLine("Circuit is Broken exiting code loop.");
          return;
        }

        try
        {
          // This should throw a FileNotFound Exception
          this.WriteToFile();
        }
        catch (Exception ex)
        {
          failureCount++;
          // Just write to the console to show that it is throwing the errors.
          Console.WriteLine(ex.Message);
        }

      }

    }

    #region ICircuitBreaker Members

    public void BreakCircuit()
    {
      Console.WriteLine("Circuit is being broken.");
      this._circuitBroken = true;
    }

    public void ResetCircuit()
    {
      Console.WriteLine("Circuit is being reset.");
      this._circuitBroken = false;
    }

    public bool CircuitBroken
    {
      get
      {
        return this._circuitBroken;
      }
      set
      {
        this._circuitBroken = value;
      }
    }

    public int Threshold
    {
      get
      {
        return this._threshold;
      }
      set
      {
        this._threshold = value;
      }
    }

    #endregion
  }

Ok, so let’s look at what this is doing.  First the class is used to write to a file.  There are two methods to write to the file.  One is a single write only the other is a batch writer for us to show the use of the circuit breaker pattern.  By default we set the system to perform 50 writes and the threshold is 5 errors, at which point we would expect the circuit to be proverbial broken.  In the attached code you will find a unit test that you can run to see the results for yourself. Needless to say when you run the codebase you should see that there are five errors thrown at which point the codebase effectively shuts down.  Try playing with the threshold.

Overall the pattern shown here can be very effective in integration scenarios but it’s especially effective in batch processes.  In the integration scenario you could have your app try to connect to say an external service and if it fails you want it to return a specified value vs crashing or throwing an exception.  This can be easily accomplished by adjusting how you use the circuit breaker.  All you would need to do is set your boolean variable if your external service is down.