Kafka Version 1.0 – The new improved Apache Kafka is now available, and ready for business use.
Kafka is a message broker, originally developed by LinkedIn. LinkedIn donated it to Apache and it became open source. It is lightweight, fast, and scalable. Traditional message brokers only manage the interchange of messages and data streams between languages. Kafka, on the other hand, provides low-latency, high throughput, fault-tolerant publish and subscribe pipelines. With streams of data, it can publish and subscribe them like a messaging system. It can process streams of data efficiently, in real-time and store it safely in a distributed and replicated cluster. The release of the new version Kafka 1.0 indicates that this platform is now ready for businesses.
Improvements
The performance improvement in the new version includes exactly-once semantics, the TLS and CRC32C implementations with Java 9 support are significantly faster, as is the shutdown. The JBOD provides better support, too.
Exactly-Once Semantics
Exactly-once semantics means that messages will be delivered only once to the end consumer, even if there is a problem at some place in the distribution system. A broker may crash, there may be a network problem, and someone sends the message again, but with exactly-once semantics, the end consumer will only receive the message once. This is the best possible outcome, but it is difficult for applications to achieve, as the messaging system and the application producing and consuming the messages need to be able to cooperate. Version 1.0 of Kafka has achieved this.
Kafka Streams
The latest version of Kafka also has an improvement to the Streams API. Kafka Streams provides a client library, and it is here that applications and micro-services are built, and the Kafka clusters store the input and output data. Internal methods have been hidden in the builder API for Streams, and there has been an addition to the new API which shows the state of active tasks at runtime. In addition to this, a new cogroup API can deal with partitioned aggregates, and there are fewer StateStores and less moving parts in your code.
JBOD
Kafka is now able to tolerate disk failures in an improved manner. The storage configuration on JBOD (Just a Bunch of Disks) was previously not recommended. Kafka 1.0 will be able to handle any failure of disks more efficiently. One disk failure will not bring the broker down rather it will continue serving the log files on the disks that are functional.