In this post, we are going to look at some key differences between Apache Kafka and Traditional message brokers (e.g JMS, ActiveMQ).
1. Design
Apache Kafka : client-centric, with the client taking over many of the functions of a traditional broker, such as fair distribution of related messages to consumers, in return for an extremely fast and scalable broker.
Traditional message brokers : broker-centric, where the broker is responsible for the distribution of messages, and clients only have to worry about sending and receiving messages.
2. Scalability
Apache Kafka : horizontally scalable by adding more partitions
Traditional message brokers : not possible to scale horizontally
3. Performance
Apache Kafka : does not slow down with addition of new consumers
Traditional message brokers : both queue and topic performance degrades as the number of consumers rise on a destination
4. Publish options
Apache Kafka : unifies both publish-subscribe and point-to-point messaging under a single destination type— topic
Traditonal message brokers : provides both publish-subscribe and point-to-point messaging for publishing messages
5. Journal
Apache Kafka : each topic has a journal and there can be multiple topics.
Traditional message brokers : messages from all queues are stored in the same journal
6. Message delivery failures
Apache Kafka : client is responsible for replaying failed messages
Traditional message brokers : broker is responsible for re-delivery of messages
7. Message structure
Apache Kafka : message is a key-value pair. Payload of the message is sent as the value. Key, on the other hand, is used primarily for partitioning purposes and should contain a business-specific key in order to place related messages on the same partition.
Traditional message brokers : message consists of metadata (headers and properties) and body (payload)
8. Message integrity
Apache Kafka : includes checksums to detect corruption of messages in storage and has a comprehensive set of security features
Traditonal message brokers : no such option provided out-of-the box
9. Message retention
Apache Kafka : messages are not deleted on the broker once consumed.
Traditional message brokers : broker would either delete a successfully processed message or re-deliver an unprocessed or failed one (assuming transactions were in play)
10. Replay of messages
Apache Kafka : since Kafka retains messages, it is possible for clients to retrieve any message, providing option for re-processing of all messages.
Traditional message brokers : not possible, since, broker does not re-deliver messages that have been processed successfully by client