Skip Navigation

Working with a Cluster

Cluster Overview

Status, version, brokers, partitions and replication health at a glance, what you can do on the cluster, and the configuration KafkIO is using.

The Overview answers the first questions you have about a cluster: is it up, what version is it, how big is it, is any data at risk, and what can you do on it from KafkIO. It also shows the connection settings KafkIO is using for the cluster.

To open it, connect the cluster and click Overview under its name in the Kafka Clusters pane. It opens in the cluster's tab; to keep it open while you work elsewhere on the same cluster, give it a tab of its own with Duplicate Tab.

The Overview of the Storefront Dev cluster, in its main tab beside the Storefront Dev (Consumers) and Storefront Prod (eu-west-1) tabs. Cards show Status online, Version 3.9-IV0, Brokers 1, Active Controller 1, Topics 15, Partitions 156 of 156, URP 0 and In-sync Replicas 156 of 156. Under Features Enabled / Configured, Schema Registry, Kafka Connect, KSQL, Topic Deletion, ACL View and ACL Edit all say Yes. Below, KafkIO Cluster Configuration shows the cluster's settings as JSON, with a Search config field at the right. 1 2 3 4 5 6
The Overview of a connected cluster, with a duplicated Consumers tab and a second cluster open beside it.
  1. The cluster at a glance: status, version, size and replication health (below).
  2. Features Enabled / Configured: what you can use on this cluster from KafkIO (below).
  3. KafkIO Cluster Configuration: the cluster's saved connection settings, as JSON.
  4. Search config...: find text in that configuration.
  5. Refresh Overview: show the latest figures (see Refreshing).
  6. Duplicate Tab: open the Overview in a tab of its own, as was done here for Consumers (Storefront Dev (Consumers)).

The cluster at a glance

The eight cards at the top describe the whole cluster. Counts cover every topic KafkIO can see, including Kafka's internal topics (such as __consumer_offsets, which alone has 50 partitions by default) and the topics Schema Registry, Kafka Connect and ksqlDB keep their state in. So Topics is usually higher than the number of topics you created, and on a cluster where you may only see some topics, it counts only those.

CardWhat it shows
Status online when KafkIO's latest look at the cluster succeeded; offline when it couldn't reach it (the other cards then show zeros, Unknown and -1).
Version The cluster's Kafka protocol version, such as 3.9-IV0. KafkIO reads the brokers' inter.broker.protocol.version setting or, on newer KRaft clusters that don't set it, the finalised metadata.version. This is the version the cluster is running as, which after an upgrade can stay behind the broker software until the upgrade is finalised. Unknown when neither is available.
Brokers How many brokers are in the cluster. Details per broker are on the Brokers page.
Active Controller The ID of the node acting as controller: the quorum leader on a KRaft cluster, the elected controller broker on a ZooKeeper one. -1 if there's none.
Topics How many topics, internal ones included.
Partitions online of total: partitions that have a leader, out of all partitions. A partition without a leader is offline: nobody can produce to it or consume from it.
URP Under-replicated partitions: partitions where fewer replicas are in sync than the partition has.
In-sync Replicas in sync of total, counting replicas across all partitions. Equal numbers mean every replica has caught up with its leader.

URP and In-sync Replicas have a question-mark icon: hover over it for a short definition (shown while hints are on in Preferences).

Under-replicated partitions and other warning signs

On a healthy cluster, URP is 0 and both "of" cards show the same number twice. When they don't:

  • URP above 0 (and In-sync Replicas short of the total): some replicas have fallen behind or their broker is down. Data is still readable, but those partitions have less redundancy than intended, and producers using acks=all start failing if the in-sync count drops below the topic's min.insync.replicas. A broker restart or a partition reassignment causes a short-lived URP; a count that stays up points at a broker that is down, overloaded or short of disk. Check Brokers against the number you expect, then the Brokers page and the topic's Overview, which shows URP per topic.
  • Partitions online below the total: some partitions have no leader at all, usually because every broker holding a replica is down. Producing to and consuming from those partitions fails until a replica comes back.
  • Active Controller -1 or Status offline: KafkIO couldn't get the cluster's metadata. See Troubleshooting Connections.
Note: URP can only rise for partitions with more than one replica. On a single-broker development cluster, or for topics created with a replication factor of 1, it stays at 0 even when the only copy of the data is at risk.

Features enabled and configured

The second row tells you, with Yes or No, what you can do on this cluster from KafkIO. It's the quickest way to understand why a section in the cluster tree is greyed out or an action is refused.

CardYes means
Schema Registry A Schema Registry is set up in the cluster's configuration (Schema Registry Connection).
Kafka Connect At least one Kafka Connect cluster is set up (Kafka Connect Connections).
KSQL At least one ksqlDB cluster is set up (ksqlDB Connections).
Topic Deletion The brokers allow topics to be deleted (delete.topic.enable isn't false). With No, KafkIO refuses to delete topics (Topic deletion restricted).
ACL View The cluster has an authorizer and you're allowed to list its ACLs, so the ACL section works.
ACL Edit As for ACL View, and you also have Alter (or All) permission on the cluster, so you can create and delete ACLs.

The first three only say the connection is configured: KafkIO doesn't contact Schema Registry, Kafka Connect or ksqlDB to fill these cards, so Yes doesn't prove they're reachable. A wrong URL or password shows up when you open that section; use the Test buttons in the cluster's properties to check. The ACL cards depend on the credentials you connected with, so the same cluster can say No for one user and Yes for another (Limited permissions).

KafkIO cluster configuration

KafkIO Cluster Configuration shows the settings saved for this cluster (name, bootstrap servers, colour, security, and the Schema Registry, Kafka Connect and ksqlDB connections) as read-only JSON. It's a quick way to check what a connected cluster is using without disconnecting it to open Edit Cluster; to change anything, edit the cluster.

The KafkIO Cluster Configuration of Storefront Prod (eu-west-1) with password typed into the search field, which reports 1 of 1 found. Line 7 of the JSON, the jaasConfig of the SASL_PLAINTEXT security settings, is highlighted and shows the username alice and the password alice-secret in plain text. Password shown as entered 1 2
Searching the configuration of a SASL cluster for "password".
  1. Type to search; matching starts as you type and the first match is selected. Without the .* toggle at the right of the field, the search is for plain text and ignores case. With it on, the text is a regular expression, matched case-sensitively.
  2. Previous and next match (Enter in the field also goes to the next one; both wrap around), and which match is selected out of how many (Invalid regex if the expression doesn't compile).
Warning: only the truststore and keystore passwords are masked (as ******). The JAAS config, with any SASL password, client secret or connection string in it, the basic-auth user:password of Schema Registry, Kafka Connect and ksqlDB, and any extra HTTP headers are shown as entered. Mind this page when sharing your screen or taking screenshots. See How passwords are stored.

Refreshing

The Overview doesn't change while you look at it. KafkIO collects each connected cluster's figures when it connects and then every 30 seconds in the background (the same collection feeds the Health Log); click 5 Refresh Overview to show the latest. So a change you just made, such as a new topic, can take up to half a minute to show in the counts. If the figures can't be read at all, KafkIO shows Error retrieving overview. with the cause.

The Version, Topic Deletion and ACL View cards come from broker settings that KafkIO re-reads at most once an hour. After changing those on the brokers, disconnect and reconnect the cluster to see the change straight away. The configuration JSON is the one the cluster was connected with.