June delivered a packed month: 4.3.0 landed, 4.2.1 followed as a bugfix release, a significant milestone in the community is worth celebrating, and we recorded an episode I’ve been looking forward to for a while.
The latest episode of Get Kafka-Nated features Omnia Ibrahim, Software Engineer at Apple and Apache Kafka Committer. Omnia is one of the co-authors of KIP-1279, a proposal to fundamentally rethink how cross-cluster replication works in Kafka, a problem space that anyone who has wrestled with MirrorMaker will know all too well. We dig into the KIP itself, the experience of shepherding a major proposal through the Apache process, and what it’s actually like to work on Kafka at Apple’s scale. If you’ve been following the MirrorMaker replacement story, this is the conversation to catch up on.
Watch the full recording here.
A special mention this month: Kafka Monthly Digest: May 2026 by Mickael Maison is the 100th edition of his Kafka Monthly Digest, a series that has run continuously since the days of Kafka 1.0.1 back in 2020. Congratulations Mickael!
Kafka Share Groups and Parallelizing Consumption — Part 3: Client-local parallelism is the latest in Jack Vanlightly’s empirically-driven series on share groups, using his new Dimster benchmarking tool to compare share groups vs consumer groups across a range of workloads. Worth reading end to end if you’re evaluating share groups seriously.
Prototyping For Free, Scaling For Cheap with Aiven Dev Tier is a piece I wrote about building a distributed community event search tool using Free and Dev Tier Kafka. It’s a practical illustration of where Kafka earns its place in a small-scale pipeline, and how a one-click tier upgrade unlocked horizontal scaling without a single code change.
Release Update
The big news is that Kafka 4.3.0 is out. Mickael Maison released it on May 23 after the RC2 vote passed, making it the largest release in recent memory with 25 KIPs implemented across brokers, clients, Connect, and Streams. Highlights include KIP-1066 (a new mechanism to cordon log dirs and brokers for scaling and decommissioning), KIP-1258 (OAuth client assertion support for the client_credentials grant type), and KIP-1240 (new configs to control delivery limits and record count locks for share groups). Full release notes and the announcement are on the Apache Kafka blog. For those not yet ready to move to 4.3.0 check out 4.2.1 which was released by PoAn Yang on June 2nd. It fixes 26 issues so if you’re not ready for the upgrade it’s definitely worth a look!
There’s already a 4.3.1 in progress, with Bill Bejeck as release manager. The reason for the quick turnaround is a significant memory leak in Kafka Streams (KAFKA-20616) that can lead to OutOfMemory errors in 4.3.0. If you’ve upgraded and run Streams, keep an eye on the dev mailing list for the release announcement.
Looking ahead, 4.4.0 planning has started, and Omnia Ibrahim (this month’s episode guest!) has volunteered as release manager.
New KIPs
Twenty-one KIPs were filed in May, here are a few I think are worth watching:
KIP-1333 proposes removing the current ~2GB log segment size limit, which is stored as an int today and becomes a real constraint in high-throughput or long-retention scenarios. The proposal would allow sizes up to Long.MAX_VALUE.
KIP-1338 proposes adding Mapped Diagnostic Context (MDC) data to all Kafka component logs, which would make log analysis and correlation significantly easier for anyone running Kafka in production.
KIP-1352 proposes event-centric aggregations for the Kafka Streams DSL, basing them on record count and proximity to an anchor event rather than time windows. A nice addition for anyone doing pattern-matching style stream processing.
Active Discussions
KIP-1279 (Cluster Mirroring) remains in active discussion on the dev mailing list and didn’t make it into 4.3.0, as covered last month. It’s being targeted for 4.4.0. If you want a deeper understanding of what the KIP actually proposes and why the architectural questions are hard, the episode with Omnia Ibrahim is the place to start.
Thanks for reading, and keep an eye out for the next edition next month for further updates from the world of Apache Kafka.
Sources & Further Reading
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.