Sergeev Ilya Ivanovich (iisergeev_2@edu.hse.ru)
Higher School of Economics, Faculty of Computer Science
A “domain event” [1] is an event, leading to changes in the state from the perspective of the specific domain. Due to the active growth of the software market, projects start to be dedicated to more complex domains on average, having hundreds of events and related desired behaviors. Currently the most commonly used approach for creating extensible and performant systems with complex domains is the “micro service” [2] architecture. A “micro service” is an independent subsystem, corresponding to a specific domain aggregate and interacting with the rest of the system using unreliable channelds. Each microservice shall encapsulate a specific bounded context, allowing teams to work autonomously while maintaining a clear understanding of how their service interacts with others. This modular approach enhances scalability and maintainability, making it easier to evolve systems over time. However the creation of new independent subsystems inevitably leads to the issues with their interactions, especially when considering domain events.
This essay examines the topic of different asynchronous communion patterns from the perspective of domain events.
Despite the new approaches of building complex systems with independent and extensible components solving many organizational problems it also leads to the appearance of new issues. Introducing new elements with the property of “independence” to a system causes problems with asynchronous communications and especially with those dedicated to the application’s domain – Domain Events. Most often the issues appear with: Delivering events to all the services, depending on them. It is crucial for a new component dependent on an event to be possible to introduce to the system. Maintaining the system performance in the conditions of significant amounts of messages which may carry large payloads and require unpredictable computational resources. It is also necessary for an extensible software system to be tolerant to the unpredictability of its modules.
Natively software systems communicate directly and synchronously one to each other due to the way a domain model may be thought and expressed. Regretfully, this approach is not acceptable because it quickly leads to the appearance of the previously listed issues with the growth of the system or with any changes in the domain. Hence, an another method is required when modifying or scaling a software system.
In the following section solutions for the problems are proposed and compared.
The usage of dedicated message systems is proposed as a solution. Those systems shall be responsible for the specific task of asynchronously delivering domain events from the components producing them (called “Producers”) to the components, which consume and handle them (called “Consumers”). The approach implies the components to not communicate directly and
The message queue pattern implies having a dedicated message system, which provides a queue (the FIFO model), containing events. The producer generates messages and sends them to the message system, which is responsible for managing the queue and ensuring that messages are delivered to consumers in a reliable manner. Consumers then retrieve and process these messages at their own pace, independent of the producer’s lifecycle. This asynchronous interaction mitigates issues related to synchronous communication, such as tight coupling and increased latency.
Key features of the message queue pattern include:
1. Point-to-Point Communication, implying that messages are directed to specific consumers, which can lead to more controlled processing. 2. Message Persistence, implying that any message queue implementations offer durability options, ensuring that messages are stored until they are processed. 3. Load Balancing, implying that multiple consumers can process messages from the same queue, allowing for efficient resource utilization.
But at the same time pattern the following issues may become crucial if the pattern is applied incorrectly: 1. High coupling, implying that consumers must be aware of the queue's existence and structure, which can introduce some level of coupling. 2. High latencies, implying that there may be latency associated with retrieving messages from the queue.
The notifications processing systems may be considered as an example [3]. In scenarios where large volumes of messages need to be ingested and processed in real time, such as notifications aggregation or stream processing, message queues facilitate the efficient handling of data streams. By decoupling data producers from consumers, organizations can implement complex processing workflows that scale according to demand.
The pattern may be implemented seamlessly in a cloud with Amazon SQS [4] or Yandex Message Queue [5]. It may also be implemented locally using UNIX messages queues [6].
The pub/sub pattern is characterized by the model, where producers (publishers) send messages to a central broker without knowledge of the consumers (subscribers) that will ultimately receive these messages. In this model, publishers generate events or messages and publish them to specific topics or channels. Multiple subscribers express interest in these topics and receive notifications when relevant messages are published.
Key features of the pub/sub pattern include:
1. Decoupling, implying that Publishers and subscribers operate independently, promoting modularity and flexibility. 2. Broadcast Communication, implying that Messages are typically broadcasted to all subscribers interested in a particular topic.
But at the same time pattern the following issues may become crucial if the pattern is applied incorrectly: 1. Redundant design complexity, implying that Implementing a pub/sub system can introduce complexity in managing topics and subscriptions. 2. Message Losses implying that if a subscriber is offline when a message is published, it may miss that message unless additional mechanisms for persistence are employed.
This decoupling between producers and consumers is a defining feature of the pub/sub pattern. It enables multiple subscribers to react to the same event independently, allowing for greater flexibility in how systems respond to changes in state or information. Additionally, publishers can operate without needing to know how many subscribers exist or their identities, promoting a more modular architecture.
The applications of the pub/sub pattern are vast and varied, spanning numerous domains and use cases [7]. For example, event-driven architectures, where events trigger specific actions across different services. As an example, in an online retail environment, when a customer places an order (the event), multiple services may need to react: inventory management systems may update stock levels, payment gateways may initiate transactions, and notification services may inform customers about their order status. The pub/sub pattern facilitates this interaction seamlessly.
The pattern may be implemented using the RabbitMQ [8] or Apache Kafka [9] message broker on a remote service. It also may be implemented locally, using language capabilities, like RxJS [10] for JavaScript Frontend applications.
Considering the given analysis the preferable solution should be based on a particular problem examining. The key factors to contemplate are the use cases, performance requirements, and architectural goals.
The pub/sub pattern excels in scenarios requiring real-time updates and high scalability among multiple subscribers. This may be essential for the systems having several actors and dependent use cases for every use case, performed by the user. For instance, it’s common for an online-marketplace to involve warehouses, sellers, banking, analysis and dozens of other domain services and internal subsystems into a single purchase process; all of them may process a single event or a small portion of coupled events, containing customer and good data.
In contrast, the message queue pattern is more suitable for applications needing reliable message delivery and controlled processing among consumers. The most well-known example of such a system is an email service, providing a point-to-point interaction between the sender and the recipient. Another example to consider is the class of systems having service, which take a significant amount of computational resources – in this case the asynchronous way of communicating is crucial for the responsiveness of the whole system, and a message queue may help separating the problematic service and the rest of the system.
However, it may be observed that the message queue pattern does not contradict with the pub/pattern and those may be incorporated into one system at the same time. For instance, the system may communicate with its components using the pub/sub pattern in general, but use the message queue pattern specifically to deliver a particular event without performance issues. For this reason, popular message systems such as Apache Kafka [9] or RabbitMQ [8] provide both of these capabilities.
The essay examines the topic of delivering domain events using the pub/sub and the message queue patterns. As a result it is important to make a specific choice based on the specific domain. The pub/sub pattern is more preferable for a system having a significant count of processings related to an event in it, whilst the message queue pattern may be a better solution for performance problems. However, the patterns should be combined with the expansion of the requirements to a system in order to address both of the cases.
[1]. Vaugn Vernon. “Implementing Domain Driven Design”, Addison Wesley Professional, February 6 2016. URL: https://www.oreilly.com/library/view/implementing-domain-driven-design/9780133039900/ (date of viewing 23.12.2024) [2]. Sam Newman. “Building Microservices” | URL: https://samnewman.io/books/building_microservices_2nd_edition/ (date of viewing 23.12.2024) [3]. C. Esposito, F. Palmieri and K. -K. R. Choo, “Cloud Message Queueing and Notification: Challenges and Opportunities,” in IEEE Cloud Computing, vol. 5, no. 2, pp. 11-16, Mar./Apr. 2018, doi: 10.1109/MCC.2018.022171662. [4]. Message Queuing Service - Amazon Simple Queue Service | Amazon Web Services Building Microservices | URL: https://aws.amazon.com/sqs/ (date of viewing 23.12.2024) [5]. Yandex Message Queue | Amazon Web Services Building Microservices | URL: https://yandex.cloud/ru/services/message-queue (date of viewing 23.12.2024) [6]. Configuring message queues on UNIX and Linux | IBM | URL: https://www.ibm.com/docs/en/cics-tg-multi/9.1.0?topic=environment-configuring-message-queues-unix-linux (date of viewing 23.12.2024) [7]. Lanying Shi, Hongming Qiao, Chengwei Yang, Yiquan Jiang, Bingyi Long, “Research on highly reliable distributed message queue system,” Proc. SPIE 13229, Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024), 132291R (7 August 2024); https://doi.org/10.1117/12.3038104 [8]. RabbitMQ: One broker to queue them all | RabbitMQ | URL: https://www.rabbitmq.com/ (date of viewing 23.12.2024) [9]. Apache Kafka | Kafka | URL: https://kafka.apache.org/ (date of viewing 23.12.2024) [10]. RxJS | RxJS | URL: https://rxjs.dev/ (date of viewing 23.12.2024)