Revision 248¶

Charmed Apache Kafka has been updated from Apache Kafka 3.9 to 4.1, bringing in the latest major-version changes. For more detail on the upstream changes, see the official Apache Kafka 4.0 release notes and 4.1 release notes.

Apache ZooKeeper has now been fully removed as a dependency. For guidance on deploying Charmed Apache Kafka in production without ZooKeeper, see the deployment documentation.

Cruise Control is now available as a stable feature for partition rebalancing. To learn how to rebalance partitions with Cruise Control, follow the tutorial.

Kafka Connect is also now generally available. You can find instructions for using Kafka Connect through the API in the how-to guide.

MirrorMaker 2.0 Kafka Connect integrators have been released to stable as well, enabling cluster migration and replication workflows. See the cluster documentation for guides on using MirrorMaker 2.0 with Kafka Connect.

Kafka Connect integrators are available for PostgreSQL, MySQL, S3, OpenSearch and MongoDB charms. For a hands-on example, see the tutorial on using Kafka Connect Integrators.

Karapace is now available as a stable, drop-in replacement for Schema Registry. For details on managing topic schemas with Karapace, refer to the schemas and serialisation guide.

Kafka UI is now stable and can be used for cluster administration tasks. See the Kafka UI documentation for usage guidance.

OAuth support is now included, along with integration with Canonical Identity Platform. To enable OAuth, follow the OAuth setup guide.

Charmhub | Deploy guide | Upgrade instructions | System requirements

Warning

This is a major release, and in-place upgrades of Charmed Apache Kafka applications across major versions are not supported.

See the documentation for how to migrate data between Charmed Apache Kafka clusters using MirrorMaker.

Features¶

DPE-7363 - feat: remove zookeeper & use Kafka 4 snap #339
DPE-7147 - feat: adapt mTLS support to DA150 spec #344
DPE-7142 - feat: update configs #345
DPE-7141 - feat: KRaft JBOD support #349
DPE-4428 - feat: Add CONF, BIN, DATA, LOGS env-vars to /etc/environment #359
DPE-7284 - feat: add support for secret-based user management #360
DPE-7413 - feat: extend SSL mapping validation #354
DPE-7657 - feat: add broker_active check #375
DPE-7526 - feat: use multiple TLS interfaces for internal/external #353
DPE-7522 - feat: add HA during CA rotation #378
DPE-7110 - feat: add support for Juju spaces #385
DPE-8031 - feat: add Terraform charm module #408
DPE-8312 - feat: auto-balance #414
DPE-9107 - feat: add machines support to TF module #463

Improvements¶

DPE-7616 - cicd: add HA tests for controller #371
DPE-7150 - chore: refresh v3 #380
DPE-7138 - chore: switch to kafka-python #401
DPE-7583 - chore: split metadata.log.dir from log.dirs #365
DPE-7839 - chore: refactor config-changed #409
DPE-8311 - refactor: remove deployment mode logic from the TF module #418
DPE-8529 - feat: migrate to data_interfaces V1 #423
DPE-7790 - chore: rename internal users, use hyphen #390
DPE-9033 - chore: update TF provider to v1.0+ and TLS to 1/stable #442
DPE-8940 - chore: add OAuth integration tests and docs #439

Bug fixes¶

DPE-5702 - chore: Active Controllers set to == 0 #325
DPE-6987 - fix: add readiness check for TLS handler #335
DPE-6436 - fix: use pathops for proper cleanup of TLS artefacts #334
DPE-7846 - fix: secrets not set issue in RelationState update #387
DPE-7987 - fix: race condition in internal TLS setup #399
DPE-4546 - fix: juju remove-unit app/leader breaks TLS #400

Compatibility¶

Principal charms support the latest LTS series 24.04 only.

Charm	Revision	Hardware architecture	Juju version	Artefacts
Charmed Apache Kafka	248	AMD64	Juju 3.6+	Distribution: 4.1.1-ubuntu4. Snap: revision 67.
Charmed Apache Kafka Connect	33	AMD64	Juju 3.6+	Distribution: 4.1.1-ubuntu4. Snap: revision 48.
Charmed Karapace	21	AMD64	Juju 3.6+	Snap: revision 16.
Charmed Kafka UI	6	AMD64	Juju 3.6+	Snap: revision 3.

Apache Kafka release notes: 4.0.0, 4.1.0.