Using Apache Kafka
Apache Kafka is a messaging system based on the publish subscribe model. A Kafka cluster consists of one or more servers called brokers. Kafka keeps messages categorized by "topics" Producers produce messages and publish the messages to topics. Consumers subscribe to specific topic/s and consume feeds of messages published to the topic/s. The messages published to a topic do not have to be consumed as produced and are stored in the topic for a configurable duration. A consumer may choose to consume the messages in a topic from the beginning. Apache ZooKeeper server is used to coordinate a Kafka cluster. The Kafka architecture is illustrated in Figure 12 1.