I love REST-based applications, especially when I am developing a new app and I need to get some data from another application. Getting a response from their team that their application has a REST API which is well documented and accessible via Swagger API is the best thing ever…besides having some sunny days after a freezing winter in NYC.
You take a look at the Swagger page, figure out the REST endpoints you need to hit to get the data you need and you are good to go! But as you start to work on integrating these REST calls in your new shiny application, you need to build yet another integration. You realize you have done this a few times already before. You have worked on numerous existing and new applications that need to either share data with other apps or consume data from them. You have built these one-to-one integrations so many times that it’s a piece-of-cake for you now.
Though while it’s easy, it does take your valuable time and resources. You have to communicate with the other team, read the Swagger documentation, create JIRA tickets, have sprint meetings, test the API, implement failure logic (retries/time-outs) in case the endpoint is not available or is taking too long, and most annoyingly, poll for data at regular intervals.
Wouldn’t it be nice to not go through so much trouble when adding new microservices? You might still have to create JIRA tickets and have sprint meetings but many of the other steps can be avoided.
There are many usecases where REST is just the ideal way to build your applications/microservices. However, increasingly, there is more and more demand for applications to become real-time. If your application is customer-facing, then you know customers are demanding a more responsive, real-time service. You simply cannot afford to not process data in real-time anymore. Batch processing (in many modern cases) will simply not be sufficient.
RESTful services, inherently, are polling-based. This means they constantly poll for data as opposed to being event-based where they are executed/triggered based on an event. RESTful services are akin to the kid on a road trip continuously asking you “are we there yet?”, “are we there yet?”, “are we there yet?”, and just when you thought the kid had gotten some sense and would stop bothering you, he asks again “are we there yet?”
Additionally, RESTful services communicate synchronously as opposed to asynchronously. What does that mean? A synchronous call is blocking, which means your application cannot do anything but wait for the response. Alternatively, an asynchronous call is non-blocking providing your application with the freedom to continue processing. This can lead to much faster applications and improve customer experience.
Of course, there are some calls that must be synchronous. For example, if your app is responsible for opening a bank account, you must run some security/background checks before approving the account. However, other services, such as the one responsible for updating customer address, don’t need to provide a response immediately.
Advantage of being real-time
Embrace yourself for some nostalgia. Remember those pre-covid days when we used to be able to fly? My favorite example is Delta’s mobile app. If you have flown Delta recently, then you know what I am talking about. Delta has one of the best apps of all the airlines I have flown. As soon as I am eligible for early check-in, I get a push notification with confirmation. As I pass the security, the app keeps me informed of the plane’s status. If the gate gets changed last-minute, the information is updated in real-time and not after some regular interval. (Because it would do me no good to know my gate changed from C-1 to B-10 30 mins later if my flight is about to leave in 20 mins). If you check-in your bag right before boarding the flight because the plane is full, your app will tell you your bag just got scanned. And finally, as soon as you land, you will be notified which baggage carrousel to pickup your bag from!
The key here is not the information is made available to the customer but it is made available in real-time as I move through the airport to my flight. That’s what makes a difference to the user experience and while it may not seem like a differentiator, I have definitely picked Delta over other airlines simply due to their excellent mobile app and user experience.
To be able to provide such an experience, applications need to implement event-driven architecture using event broker (such as Solace PubSub+) and streaming APIs.
With such an architecture, your applications will publish data to your event broker in real-time and your broker will be responsible for routing those events to any downstream subscriber(s). For example, as soon as I check-in at the airport, an event can be published to a well-defined topic: delta/customer/{customerID}/{airportID}/checkedIn. That’s all that specific application needs to do. It doesn’t need to worry about calling some REST endpoint, waiting for it to reply back, figuring out failure/retry logic etc. And, in future, if it needs to publish the same event to another app, it doesn’t need to worry about integrating with another REST endpoint. In fact, it doesn’t need to do anything. The downstream subscriber simply subscribes to the topic and gets the event.
Once the event has been published, it is the router’s job to ensure it is delivered to all the downstream subscribers interested in this event. This is known as the pub/sub messaging pattern. Solace PubSub+ broker supports guaranteed messaging which means zero message loss. If your subscribers are online, they can subscribe using wildcards such as delta/customer/> or delta/customer/*/JFK/> and the event(s) will be pushed to them. If one or more subscribers are not online for some reason, events will be enqueued in queues with ordering preserved. Whenever the subscriber comes online, it will simply consume those messages and pick up where it left off.
These are just some of the benefits of implementing event-driven architecture. There are plenty more such as scalability, agility, high-availability, and efficiency (via pub/sub). However, as with anything, make sure to do a deep-dive, document all the requirements and see if event-driven architecture makes sense for your usecase.
Coexistance of REST APIs and streaming APIs
As I mentioned earlier, REST APIs are not going anywhere. There are many usecases that don’t need the additional complexity of an event broker and streaming APIs. For example, many user-facing applications should be REST-based as it is the standard for web-based applications. For example, if I am accessing a website, I am simply submitting a GET request. If I am entering some user information via a form, it’s a POST request. If I am scanning a QR code to submit a payment, it is just another RESTful endpoint behind the scenes. The user will submit a request over REST which most likely goes to some API gateway and is then routed to a broker. In this case, our publisher is REST-based. However, our downstream subscribers can choose to use REST, streaming APIs or both!
This is another benefit of implementing event-driven architecture via Solace PubSub+ event broker. It supports a variety of APIs and open protocols which means your applications can choose which language (i.e. Java, Python, C etc) or protocol (i.e MQTT, AMQP, SMF etc) to use.
Thus, a modern architecture will most likely not be solely REST-based or event-driven but instead a combination of the two. As your architecture evolves from REST-based synchronous communication to real-time asynchronous communication, there will be some components of it that will continue being REST-based.
To support such a hybrid architecture, you will need an event broker that supports REST out-of-box without any plugins or any proxy services. It will need to handle protocol translation from REST to other open protocols such as MQTT and AMQP reliably.
If your usecase does lend to full evolution from REST to streaming then having a broker which supports REST protocol natively will make it much easier for you to migrate your services. You can select one service at a time to support streaming and others can continue using REST.
In a subsequent post, I will demo a hybrid architecture such as the one shown above. All you need to know for now is that as you consider moving to an event-driven architecture, you don’t and shouldn’t necessarily get rid of all your RESTful services. Your evolved architecture might be a hybrid one!
One reply on “RESTful applications in an event-driven architecture”
Himanshu,
This is fantastic content !! Very easy to consume (no pun intended), with the real world example and a delineation on when EDA is needed or not as well as REST or Synch v Asynch. Nice !!