File system paths

In the following table, we summarise some of the most relevant file paths used in Charmed Apache Kafka:

Environment Variable	Path	Description	Permission
BIN	/snap/charmed-kafka/current/opt/kafka	Binary files for the Charmed Apache Kafka distribution. Note that this is a read-only Squashfs file system.	read-only
CONF	/var/snap/charmed-kafka/current/etc/kafka/	Configuration files used by Apache Kafka daemon process. These files are generally written and managed by the charm.	Owned by _daemon_ Managed by charm
LOGS	/var/snap/charmed-kafka/common/var/log/kafka/	Application Logging files generated by the Apache Kafka daemon process. These files are written by the workload, but they may be read by other components to provide monitoring (for example, Grafana or other charms).	Owned and managed by _daemon_
DATA	/var/snap/charmed-kafka/common/var/lib/kafka/	Raw data stored persistently by Apache Kafka during its operations. The files are written and managed by Apache Kafka only.	Owned and managed by _daemon_

All of the environment variables ($CONF, $BIN, $LOGS and $DATA) are written to /etc/environment on the charm units, and as such are automatically available for all SSH sessions.

For example, to list the files and directories in the $LOGS directory on a particular unit, simply do:

juju ssh kafka/0 sudo -i 'ls $LOGS'

Configuration

• $CONF/server.properties - the full configuration file for the broker and KRaft controller services

• $CONF/client.properties - a minimal configuration file for making client connections to brokers with SASL authentication and SSL encryption

  • For internal administrator usage only

    • --command-config $CONF/client.properties is often used as an argument when running Kafka CLI commands either directly, or via the snap commands

• $CONF/kraft-client.properties - the minimal configuration file for making client connections to KRaft controllers with SASL authentication and SSL encryption

  • For internal administrator usage only

• $CONF/peer-keystore.p12 + $CONF/peer-truststore.jks - the Java keystore and truststore used for inter-broker and broker-controller SSL encryption

  • Passwords to the keystore and truststore are stored in Juju secrets

• $CONF/client-keystore.p12 + $CONF/client-truststore.jks - the Java keystore and truststore used for client SSL encryption

  • Passwords to the keystore and truststore are stored in Juju secrets

System logs

• $LOGS/server.log - the broker service logs

  • Very useful for general debugging

• $LOGS/kafka-authorizer.log - Kafka ACL authorisation events

  • Very useful for debugging authorisation failures

• $LOGS/controller.log - the KRaft controller service logs

Apache Kafka binaries

• $BIN/bin/*.sh - general bash scripts provided from upstream Apache Kafka, with utilities for managing and interacting with the cluster

  • These files are typically accessible directly via Snap commands - e.g, kafka-topics.sh can be invoked by running charmed-kafka.topics

Message data and cluster metadata

• $DATA/data/* - the data storage directory where the raw Apache Kafka message data is persisted to disk

  • Each Juju mounted JBOD storage directory will have an integer identifier matching a subdirectory in $DATA/data/

  • Find these directories with juju status --storage kafka | grep data/