Building Our Own ESB: Publish / Subscribe (Part 1)
Let's Get it Started
If you only have a few services, an ESB (Enterprise Service Bus) might be overkill. That said, as the number of services you have deployed grow, an ESB will look more and more appealing. One of the main functions of an ESB is to give your services a central place to connect. If you only have a handful of services, manually updating each of their configurations each time a service location changes can be quite time consuming and error prone. When you have hundreds of services, that really isn't an option. An ESB supports your services, handling the grunt work for you so that you can focus on more important things. One of the best ways to understand a complex system is to take it apart, piece by piece and rebuilt it. That's exactly what we are going to do.
Earlier, we talked about how events are an essential part of building flexible systems. Events have one major flaw: they can't cross application boundaries. For example, your CRM can't just add an event handler to a form on your web site and start receiving notifications. Of course, you could hard code something in your web site like this:
void OnCustomerInquiry(object sender, CustomerInquiryEventArgs e)
{
salesForce.Create(e.Customer);
}
In fact, many systems are coded like this and they work fine... most of the time. But then the day comes when Salesforce goes down and your site goes down with it. There is a better way: "Publish / Subscribe." Publish / Subscribe is just a fancy name for events in distributed systems. Unlike the quick and dirty event handler above, a good publish / subscribe infrastructure assumes that systems might be unavailable and can work around temporary failures by automatically resending messages when the external system comes back up.
Since events are an important part of building distributed systems, and publish / subscribe is how we handle events in a distributed system, it makes a lot of sense to build a common framework for handling this functionality. At a minimum, we will need a few things:
- A way to register available events
- A way to fire events (publish)
- A way to register event handlers (subscribe)
- A way to handle temporary failures
- A way to handle permanent failures
Since we are talking about building an ESB, we'll add one more into the mix:
- A way to support different communication protocols
The last part isn't a standard function of a basic publish / subscribe framework, but it is really useful when building enterprise systems. You might decide that all your systems are going to communicate with WCF via a TCP/IP binding or an MSMQ binding for various reasons, but you can't control the protocols of systems that you don't write. A lot of systems today have minimal support for web services (ie. SOAP 1.1 without WS-*), and you aren't going to find many systems on the market today with full blown support for every WCF binding under the sun. You need your system to be able to span these different protocols, or it will be useless when trying to communicate with any system that you don't have complete control over. If we want to make it really useful, we'll also support message transformation, but let's save that for later.
This week, we'll take a look at building a publish / subscribe framework as we investigate what happens behind the scenes in an ESB.