The Kafka page of Preferences sets how long KafkIO waits for your clusters before it gives up: the Kafka brokers themselves, and the Schema Registry, Kafka Connect and ksqlDB servers next to them. It also has two options for how Avro messages are shown. The settings apply to every cluster; there's no per-cluster override.
The Kafka page
Choose File ▸ Preferences, then Kafka in the list on the left. As on every Preferences page, nothing changes until you click Save (see Opening Preferences).
- The Kafka page.
- Each timeout is typed in milliseconds. The arrows step by 250 ms, and the note under the label gives the maximum.
- Shortcuts: click one to fill in a common value.
- The same value in seconds or minutes, updated as you type, here for the API timeout, so you can check your zeros.
- Schema Registry timeouts.
- Avro display options: null fields
- and fully qualified names.
- Kafka Connect timeouts.
- ksqlDB timeouts.
Every timeout must be at least 1 ms and can't be left empty. The connect timeouts go up to 300000 ms (five minutes), the API and read timeouts up to 1800000 ms (30 minutes); the field won't take a larger number.
| Section | Setting | Default | Maximum |
|---|---|---|---|
| Kafka connection properties | Request/connect timeout (ms) | 6000 (6 s) | 5 min |
| API/read timeout (ms) | 20000 (20 s) | 30 min | |
| Schema Registry properties | Connect timeout (ms) | 2500 (2.5 s) | 5 min |
| Read timeout (ms) | 12000 (12 s) | 30 min | |
| Kafka Connect properties | Connect timeout (ms) | 2500 (2.5 s) | 5 min |
| Read timeout (ms) | 12000 (12 s) | 30 min | |
| ksqlDB properties | Connect timeout (ms) | 2500 (2.5 s) | 5 min |
| Read timeout (ms) | 12000 (12 s) | 30 min |
When changes take effect
KafkIO builds its connections to a cluster when you connect to it, so a cluster that's already connected keeps the timeouts it started with. To use new values, disconnect the cluster and connect again. A Test Connection in the cluster dialog always uses the values you last saved. The two Avro options are the exception: they apply from the next search, without reconnecting.
Kafka client timeouts
The two settings under Kafka connection properties are passed to the Kafka admin client that KafkIO uses to talk to your brokers: to test and open the connection, and for cluster information and admin operations such as listing topics, brokers and consumer groups, or reading and changing configuration. Searching and producing messages use their own consumers and producers, with Kafka's standard settings.
- Request/connect timeout (ms)
- Kafka's
request.timeout.ms: how long the client waits for an answer to a single request to a broker, including setting up the connection. If a broker doesn't answer in time, the request fails or is retried. - API/read timeout (ms)
- Kafka's
default.api.timeout.ms: the overall limit for one operation, such as "list the topics", including any retries. When it runs out, the operation fails with a timeout error.
Because one operation can include several requests, the request timeout must be less than or equal to the API timeout. If it isn't, both fields get a red error marker:
- The request timeout, longer than
- the API timeout. Fixing either field clears both markers.
Clicking Save in that state shows an error, "Please specify a valid request timeout (it must be less than or equal to API timeout)", and keeps the window open. When you raise the request timeout, raise the API timeout with it.
With the defaults, a cluster that can't be reached fails a connection test after about five or six seconds. Kafka clients report many different problems as a timeout, so read Timeouts and misconfiguration before you raise these values to cure a failing connection.
Schema Registry
Timeouts
These are standard HTTP timeouts for every request KafkIO makes to the Schema Registry: browsing and changing subjects and schemas, and fetching the schemas it needs to decode schema-encoded messages.
- Connect timeout (ms): how long to wait for the network connection to the registry to open (2.5 seconds by default).
- Read timeout (ms): how long to wait for the registry's response once connected (12 seconds by default).
A connect timeout usually means the registry can't be reached at all (a wrong URL, a firewall, a VPN that isn't up, or a proxy that's needed: see Network, Proxy & Trust). A read timeout means it was reached but answered slowly. Schema Registry connections are set up in Schema Registry Connections.
Avro display options
Two options change how KafkIO shows Avro messages it decodes with the Schema Registry. Both are on by default.
- Avro: show null fields in decoded messages: null fields appear as
"field": null. Off, they're left out, which shortens records with many optional fields but hides which fields exist. - Avro: show fully qualified names in decoded messages: a value of a union type
is labelled with the full name of its type, such as
com.storefront.Address. Off, it gets the short name,Address, unless two types in the union share a short name.
They apply from the next search after you save, and only to Avro: JSON Schema and Protobuf messages look the same either way. Deserialization shows the same message with the options on and off.
Kafka Connect timeouts
Standard HTTP timeouts for KafkIO's requests to the Kafka Connect REST API: listing connectors and their tasks, creating, changing, pausing and deleting them. Connect timeout (ms) (2.5 seconds by default) is how long to wait for the connection to open, Read timeout (ms) (12 seconds) how long to wait for an answer. They apply to every Connect cluster of every Kafka cluster. See Kafka Connect Connections and Kafka Connect.
ksqlDB timeouts
- Connect timeout (ms) (2.5 seconds by default): how long to wait for the connection to the ksqlDB server to open.
- Read timeout (ms) (12 seconds): as its information icon says, it's enforced only for certain operations. When KafkIO lists ksqlDB's streams, tables, topics, persistent queries or connectors, or creates or drops a connector, it waits at most the connect and read timeouts added together (14.5 seconds with the defaults) before reporting a timeout.
Statements you run in the ksqlDB editor aren't limited by the read timeout, so a long-running query or a push query isn't cut off. See ksqlDB Connections and ksqlDB.
Choosing timeouts
The defaults suit a cluster on the same network or a nearby cloud region. Change them when:
- Connections sometimes work and sometimes time out. A cluster on another continent, behind a busy VPN or a slow proxy, can need more time. Try a request timeout of 15 to 20 seconds with an API timeout of 60 seconds, then reconnect.
- Large clusters time out on big operations. Listing thousands of topics or consumer groups takes longer than a quick check: raise the API timeout first, since it covers the whole operation.
- A REST service is slow to answer (a registry with many subjects, a busy Connect cluster): raise that service's read timeout rather than its connect timeout.
- You want failures sooner. If you often open clusters that are offline, shorter timeouts make KafkIO give up faster.
To go back to the defaults, type them in from the table above, or reset the preferences (see Managing Preferences).