Anti-pattern: EDA message notifications at enterprise level

Event Driven Architecture (EDA) is a software architecture style. The wikipedia article on EDA [] states the following:

Event-driven architecture (EDA), is a software architecture pattern promoting the production, detection, consumption of, and reaction to events. An event can be defined as "a significant change in state". For example, when a consumer purchases a car, the car's state changes from "for sale" to "sold". A car dealer's system architecture may treat this state change as an event whose occurrence can be made known to other applications within the architecture. From a formal perspective, what is produced, published, propagated, detected or consumed is a (typically asynchronous) message called the event notification, and not the event itself, which is the state change that triggered the message emission. Events do not travel, they just occur. However, the term event is often used metonymically to denote the notification message itself, which may lead to some confusion. This is due to Event-Driven architectures often being designed atop message-driven architectures, where such communication pattern requires one of the inputs to be text-only, the message, to differentiate how each communication should be handled.

As at July 2019 this wikipedi article has 6 instances of the word "trigger" but none of them are in relation to a database trigger. There are 8 instances of the word "message" and the association with message-driven architectures is made explicit.

There are lots of examples of people praising EDA at the enterprise level. For example:

• Jack van Hoof's blog post EDA versus CEP (and SOA) (2008).
• Udi Dahan's blog post SOA, EDA, and CEP a winning combo (2008)
• Gartner press release Gartner Says CIOs Must Define an Event-Centric Digital Business Strategy (2017)
• Telmo Subira Rodriguez's blog post Understanding Event-Driven Architectures (EDA): the paradigm of the future (2018).
• Jerry Mathew's blog post SOA vs. EDA: Is Not Life Simply a Series of Events? (2019)

It's important to make the following distinction:

• event sourcing: this is where the base representation in the database involves the recording of temporal events. In a relational events database the DB schema is designed to record predicates about the events that are important to the business in a logical system. Such databases are inherently temporal databases. Mature businesses have a tendency to use event sourcing because it offers so many benefits such as the historical record - which for example might be needed for auditing. For example events for opening and closing bank accounts, transfers between accounts, raising invoices, refunding a policy etc. Since events are typically immutable they allow for certain interesting optimisations - such as for synchronoisation of replicated databases. This is because insert operations on sets commute (can be applied in different orders and give the same result).
• event messaging: event notification messages are raised by event producers and are used to trigger processing by event consumers. The events are not regarded as an underlying representation of the data in the manner of event sourcing. It is not expected that there will be queries over sets of event messages. Instead their purpose is to be delivered, received and processed by the consumers. If they persist in the messaging systm, it is typically only to ensure at least once delivery, and it is assumed events that have been delivered to all consumers can eventually be deleted.

At the enterprise level event messaging emphasises applications or services publishing and subscribing over topics using some kind of messaging system.

An event message driven approach is great for GUI frameworks, I/O competion events, timers and so forth - i.e. for events related to transient state machines in a single running process. However, it's an anti-pattern at the enterprise level (in a distributed system) when used for events related to state changes of the databases.

It is better to manage business events in an events database, using the power of the RM, i.e. first order logic, in a fully fledged DBMS, rather than leaving it to some half baked messaging system, that raises more questions than answers (e.g. about atomic commitment, duplicates, idempotency, missing messages, late subscribers, etc).

A DBMS should provide facilities to trigger asynchronous, exactly once in order processing on state changes on views. These are called triggers and they make an event messaging system redundant.

Using messages to achieve logical independence and loose coupling between producer and consumer is cumbersome by comparison. If a DBMS doesn't allow applications to define triggers then application developers are forced to implement them manually. This is bad because it's not easy to make it robust. A typical approach is the following:

• A messaging system is used, such as RabbitMQ
• Appropriate message types related to the state changes of the database are defined
• Typically EOIO processing of the events is required, which often involves application code being concerned with:
  • persisting messages on the producer allowing for message replay
  • purging old messages on the producer
  • persisting messages in a messaging system so it guarantees at least once delivery
  • assigning message ids on messges to allow for duplicate detection
  • persisting information about what messages have already processed by consumers
  • detecting duplicates or ensuring idempotent processing of messages by consumers
  • using dead letter queues
  • allowing for schema changes of the message types (given that the messages persist on both the messaging broker and the producers)

This is more complicated and less efficient than when the DBMS supports application defined triggers.

See the Sales,invoice,shipping example for a more detailed discussion of Udi Dahan's example.