Intro to Message-Driven and Event-Driven Architectures in Azure
Fundamentals of Message-driven and Event-driven Architectures
In this post, we're going to discuss what message-driven and event-driven architectures (EDAs) are. But before we delve into that, let's first take a quick look at the traditional way of designing software architecture, understand its flaws, and explore why there was a need for something else.
Traditional Way of Designing Software Architecture
Before we explore EDA, let's take a quick look at the traditional and predominant way of designing software architecture. We'll examine its limitations and understand why the need for a different approach to software architecture was necessary.
Synchronous Client/Server (Request-Response) Architecture
Using this architecture approach, we send a request to some endpoint, usually an API, to perform an action, like booking an apartment. Being synchronous means that the client sits idle and does nothing until the server finishes working on the request and responds with either success or an error. Below, you'll find a fictional example of an apartment booking app, similar to Airbnb or Booking.com, where the client books an apartment in Paris synchronously.
The Request-Response architecture is fundamental to the world of networking in general, as it is based on the client-server model. It represents the oldest architecture still heavily in use today.
It functioned exceptionally well for the simpler applications of the past, and it continues to be effective for tasks that are not overly complex and remain largely monolithic. However, it starts to become a bottleneck as applications evolve to comprise many smaller apps that communicate with each other in a distributed manner, such as microservices.
Furthermore, the synchronous nature of this architecture means that the entire business logic must complete before the client can proceed with any other actions. For instance, in the above example, the server is likely performing tasks such as:
Validating input data
Checking availability
Notifying the apartment owner
Processing payments
Updating numerous tables in the database(s)
Sending emails with booking details to both the client and the owner
Returning success to the client
This process is time-consuming, and several issues can arise:
Payment processing and email dispatching involve external services. If these services are slow, they can significantly increase client wait times and negatively impact user experience.
If any step in the process fails, the entire transaction fails. For example, a payment might fail not due to incorrect credit card details, but because of an issue with the VISA network. There's no provision for retrying—failure means the client must attempt to book again by starting from scratch.
The architecture's reliance on server availability means that if the server goes down, no bookings can be made. This makes the server a single point of failure, rendering the application useless until the server is back online.
Message-Driven Architecture
Asynchronous message-driven architecture cures most of the synchronous client/server architecture's limitations. It decouples the participants from each other, making the whole business process and user experience smoother, resilient, and easier to scale.
This architecture allows applications to scale to millions of users with ease; something that wasn't that easy to do using the traditional approach. Let's take a quick look at messaging and see how it differs from the traditional approach above.
Below is the same fictional example we’ve already seen, but this time using messaging instead of the traditional request/response model.
As seen by the animation above, the client and the app server(s) are now decoupled from each other. They both rely on a middleman, called a message broker, to help them interact with each other.
There are cases where this architecture is implemented without a message broker, for example using Azure Queue Storage, but usually, for complex distributed apps, there is one. A message broker provides additional services like routing, filtering, transformation, etc.
Within the message broker, there is a structure called a queue that's used for point-to-point communication between a producer and a subscriber of a given message. What this means is that the message gets delivered to a concrete consumer(s) that expect that particular message.
Message example
Following the booking example above, here’s a fictional message that your mobile phone app could send to the fictional booking app:
{
"apartmentId": "123456789",
"customerId": "987654321",
"customerName": "John Doe",
"customerEmail": "johndoe@example.com",
"locationId": "551123",
"checkInDate": "2024-04-15",
"checkOutDate": "2024-04-20",
"numberOfGuests": 2,
"specialRequests": "Late check-in, baby crib",
"paymentStatus": "Pending",
"totalPrice": 1500.00,
"currency": "USD"
}
Message Queue Components
Message producers: Apps and services that send commands for other services to execute by using a message queue.
Message queue: A data structure used to temporarily store messages from producers and share them with consumers when requested.
Message consumers: Apps and services that receive commands from message producers through the message queue.
Benefits of Message Queues
Loose coupling between systems: Since systems no longer depend directly on each other but rather on a queue, it's now feasible for a system to be offline temporarily and still receive requests once it's back online, as these requests will have been stored in the queue. This level of independence is not achievable with the traditional synchronous approach, where the recipient must always be available.
Scalability & Reliability: In the event of a traffic surge, you can always add more recipients to the consumer pool and remove them when they're no longer needed. Message queue implementations, especially those provided by major cloud providers, are capable of handling a significant amount of load.
Dead Lettering: If a message can't be processed, it isn't deleted; instead, it's moved to a separate queue that contains all the messages that couldn't be processed for some reason. This allows for the possibility of retrying them with specific logic designed for erroneous message handling, or analyzing them using analytics tools to understand what's happening with your system and why message consumption is failing.
An important distinction between event-driven and message-driven architectures, as we'll explore shortly, is that in message-driven systems, once a message is marked as delivered by the consumer(s), it is usually removed from the queue permanently. This approach aligns with the message-driven model's emphasis on sending commands (as outlined in the Command design pattern), such as registering a user, booking an apartment, sending an email, etc.
In the Azure ecosystem, you can leverage Azure Service Bus to build a robust message-driven system capable of accommodating needs ranging from small to enterprise-grade requirements.
What are the Benefits of Azure Service Bus?
PaaS: Since Azure Service Bus is a PaaS (Platform as a Service) offering, you don't need to worry about scaling, hardware failures, backups, logging, patching systems, or maintaining any infrastructure-related aspects. Everything is managed for you by Azure.
Load balancing: This is an implementation of load balancing that allows multiple competing consumers (in our case, applications) to read from the queues simultaneously.
Topics and subscriptions: This feature enables clients to subscribe to specific messages, as seen in the Publish/Subscribe (Pub/Sub) model.
Transactions: Execute several operations within the scope of a single atomic transaction, similar to what is available in SQL.
Scale: Azure Service Bus is capable of handling extremely large volumes of messages and can dynamically scale in response to demand.
What is Event-Driven Architecture?
Event-driven architecture, as the name implies, is based on events.
Events are immutable, historical facts that have occurred within a system at a certain point in time. Examples of such events include: user registered, payment processed, and notification received.
Just as you cannot change history, you cannot alter events once they are published, which is what makes them so powerful in the realm of event-driven architecture.
Similarities
There are a few similarities between the two architectural designs, so let's explore them.
Asynchronous communication via middle-man: Both architectures rely on a middleman, leading to loose coupling.
Scalability & Reliability: Both architectures can be used to build systems that are scalable and reliable, capable of handling significant amounts of traffic.
Differences
Events could stay forever: Unlike messages, events are factual, integral parts of the system’s history. This is why they are often stored indefinitely and can be consumed either now or at a later point by some new system—or both. The concept here is that you can reconstruct the system’s state from day 0 to the present using the history of events.
Events don’t care who consumes them: In contrast to messages, which are intended for specific consumer(s), with events, you, the producer, simply publish them without concern for who might consume them later. It could be a single consumer today, and a hundred more could emerge tomorrow.
As illustrated in the animation above, each time an event occurs within the system (a factual occurrence), an event is published. This event is stored indefinitely and consumed only by the services that are interested in it.
For instance, using the example above, when John Doe submits a review, an event is generated. Once this event is published, the REVIEW-CHECK SERVICE is notified and consumes the event, enabling it to inspect the review for any vulgar or inappropriate content.
Similarly, the event attracts the attention of the NOTIFICATION SERVICE; whenever a review is posted, this service could inform the owner of the apartment, the author of the review, or an administrator whose task is to manually inspect the review (though ideally, this should be automated using AI).
This principle is applied to all user actions from above.
To construct an event-driven system in Azure, we could utilize Azure Event Grid.
If you found this useful, please consider sharing it on social media and subscribing. Insights like this one take a lot of time to produce and your support motivates me to keep going. 🙂