General Connection Settings

The Kafka General tab is the first tab of the Cluster Properties dialog, and the only one every cluster needs: it says what the cluster is called, where to find it, and what colour to show it in. Open it with Add Cluster, or Edit Cluster on a disconnected cluster (see Managing Cluster Configurations).

1 2 3 4 5

1. Name: required. See Name.
2. Bootstrap servers: required. See Bootstrap servers.
3. Check network reachability opens the Network Tool for these servers. See Checking reachability.
4. Color: optional; tick the box to use the picker. See Colour.
5. Sample Docker Compose, under the Docker note. See Kafka in Docker.

The two notes under the fields, Connections / timeouts and Docker, are reminders of the two things that most often stop a connection working. This page explains both, after the fields.

Name

The name is for you: KafkIO shows it in the Kafka Clusters list, on the cluster's tabs (a duplicated tab adds the section, as in Storefront Dev (Consumers)), in the status bar and in the tooltip of the coloured corner of windows opened for it. It isn't sent to Kafka and doesn't have to match anything on the cluster. It's required (Please enter a name for the cluster.), but it doesn't have to be unique.

Include the system and the environment, and the region if you have several, for example Storefront Prod (eu-west-1) rather than Kafka. Keep it short enough to read in the cluster list: long names are cut off there (hover over a coloured window's corner triangle, or look at the status bar, to see a name in full). You can rename a cluster at any time; its favourite topics and other settings stay with it.

Bootstrap servers

This is Kafka's bootstrap.servers setting (the note under the label says so), and KafkIO passes it to the Kafka client exactly as you type it.

Format and multiple brokers

Enter one or more brokers as host:port, separated by commas. Spaces after the commas are fine:

kafka1.stg.example.com:9092, kafka2.stg.example.com:9092, kafka3.stg.example.com:9092

• The port is required. Kafka has no default port: localhost on its own fails with ConfigException: Invalid url in bootstrap.servers: localhost. Use the port of the listener that matches your security settings (a cluster often has a plaintext port and a separate SSL or SASL one; your Kafka administrator or provider will tell you which is which).
• Host names or IP addresses both work. No http:// or other scheme: that's for the Schema Registry, Kafka Connect and ksqlDB URLs on the other tabs.
• If none of the names resolve (a typo, or a VPN that isn't connected), the test fails at once with ConfigException: No resolvable bootstrap urls given in bootstrap.servers.

You don't need to list every broker. The bootstrap servers are only used for the first contact: the client asks whichever one answers for the cluster's metadata, which lists every broker and the address to reach it at, and from then on it talks to the brokers directly. List two or three, ideally in different racks or zones, so that KafkIO can still connect when one of them is down. With a single address (a load balancer, or a managed service's bootstrap endpoint) that's taken care of for you.

Checking reachability

Check network reachability under the field opens the Network Tool with one row for every server in the field, so you can check them without typing them again. Click Check to test them all:

1 2 3

1. One row per bootstrap server, filled in from the field. You can edit, remove or add rows; it doesn't change the cluster's settings.
2. The results: for each host, its IP address, whether it answers a ping, and whether the port accepts a connection. Failures are in red.
3. Check runs the tests, waiting up to the Timeout (1000 ms by default) for each.

Read the three results in order:

The Network Tool doesn't block the dialog, so you can leave it open, fix the field and check again. It only tests the hosts you listed: a port that's open proves the network path to the bootstrap servers, not that the security settings are right or that the brokers' advertised addresses work (see the next section). A server entered without a port is tested on port 80.

Colour

Tick the box next to Color to give the cluster a colour, then choose one from the picker (it stays greyed out until the box is ticked). Untick it for no colour. KafkIO uses the colour for the cluster's row in the Kafka Clusters list, the Kafka icon on its tabs (duplicated ones included), the status bar, and a small triangle in the corner of windows opened for the cluster, such as the windows for producing a message or creating a topic (Cluster colours shows where).

The point is to make a mistake hard: pick a colour per environment and use it for every cluster in that environment, for example green for development, amber for staging and red for production. As the hint by the box says, pick a colour that contrasts with the background; very dark colours disappear in the dark theme and very light ones in the light theme.

Timeouts and misconfiguration

The Connections / timeouts note on the tab warns that Kafka clients report many problems as a timeout. That's worth understanding before your first failed connection, because the error rarely names the real cause.

When you click Test Connection (or connect), KafkIO creates a Kafka client with the bootstrap servers, the security settings and the timeouts from Preferences ▸ Kafka, and asks it to list the topics. If the client gets no usable answer from a broker, it doesn't know why: it keeps trying until the timeout runs out. With the default timeouts that takes about five or six seconds, and ends in this dialog:

1

1. Network Tool opens the Network Tool with the bootstrap servers, like the link on the tab.

That one message, Timed out waiting for a node assignment, came up in the KafkIO lab for each of these quite different mistakes:

The dialog says the same in short. Errors that aren't timeouts (a name that doesn't resolve, a wrong password, a certificate problem) come in an Error connecting dialog that usually names the cause directly; Show Details has the full exception. For more causes and how to work through them, see Troubleshooting Connections.

The note's second warning is about clusters with several brokers: a single broker with a wrong advertised address or listener doesn't always stop KafkIO connecting, because the other brokers answer. Instead, anything that needs that broker (the partitions it leads, for example) is slow or times out, which looks like an unreliable cluster rather than a configuration problem. Make sure every broker is configured the same way.

Kafka in Docker

Running Kafka in Docker on your own machine, with KafkIO on the same machine outside Docker, is the most common local setup, and the one most likely to connect and then time out.

Listeners and advertised listeners

A Kafka broker has two related settings, and both have to be right for a client outside the container:

listeners (KAFKA_LISTENERS in Docker)

The network interface and port the broker accepts connections on. Inside a container, bind it to 0.0.0.0 (all interfaces) so connections forwarded from the host through the container's port mapping arrive. Bound to the container's own name or to localhost, it only accepts connections from inside the container.

advertised.listeners (KAFKA_ADVERTISED_LISTENERS)

The address the broker tells clients to use, in the metadata it returns after the bootstrap. It has to be an address the client can reach. For KafkIO on the host, that's localhost (or 127.0.0.1) and the port mapped to the host.

The usual mistake is advertising the container's own host name, for example broker:9092. That name only resolves inside Docker's network, so KafkIO reaches the bootstrap server through the mapped port, is told to use broker:9092, can't resolve it, and times out, exactly as in the table above. The port mapping (ports: "9092:9092") also has to publish the advertised port.

If KafkIO runs somewhere else, for example on another machine or in another container, advertise an address that resolves there instead. When different clients need different addresses (containers on the Docker network and tools on the host), give the broker two listeners, one advertised with the container name and one with localhost, on different ports.

Using the Sample Docker Compose file

For a working starting point, click Sample Docker Compose under the Docker note (it's also in Tools ▸ Sample Docker Compose). It opens a window with a Docker Compose file for a single-broker Kafka, already set up for KafkIO on the host; Copy puts it on the clipboard. The lines that matter here:

    ports:
      - "9092:9092"
    environment:
      KAFKA_LISTENERS: PLAINTEXT_EXT://0.0.0.0:9092,CONTROLLER://broker:9093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT_EXT://localhost:9092

The broker listens on every interface on port 9092, publishes that port to the host, and advertises localhost:9092. Save the file as docker-compose.yml, run docker compose up -d in its folder, and add a cluster with Bootstrap servers set to localhost:9092 (or 127.0.0.1:9092) and no security. The whole file and how to run it are on the Sample Docker Compose page.

Related

• Managing Cluster Configurations
• Security & Authentication
• Troubleshooting Connections
• Network Tool
• Sample Docker Compose
• Kafka & Timeouts
• Cluster colours