Security & Authentication

Most clusters outside your own laptop are secured: the brokers expect TLS, a login, or both. You tell KafkIO how on the Kafka Security tab of the Cluster Properties dialog (Add Cluster, or Edit Cluster on a disconnected cluster; see Managing Cluster Configurations). Every field on the tab is a standard Kafka client setting, and the small grey text under each label names it (security.protocol, ssl.truststore.location and so on), so the values your Kafka administrator or cloud provider gives you for any Kafka client go straight into the matching field.

The settings on this tab are for the brokers only. Schema Registry, Kafka Connect and ksqlDB have their own URLs, logins and TLS settings on their own tabs (see Schema Registry Connection).

Choosing a security type

Security type (security.protocol) is the first decision, and it has to match the listener behind the port in your bootstrap servers. A cluster often has several listeners on different ports (say, plaintext on 9092 and SASL on 9094), so ask which port goes with which type. The type decides which sections the tab shows:

Managed services almost always use SASL SSL. SASL plaintext sends everything, including a PLAIN password, unencrypted: fine on a development cluster, not over a network you don't control.

Settings in a hidden section are kept but not used: switching from SASL SSL to SSL doesn't lose your JAAS configuration, it just stops sending it.

SSL / TLS

With SSL or SASL SSL, the SSL/TLS section appears. For one-way TLS (the usual case: the broker proves who it is, and you log in with SASL or not at all) you need at most the first three fields; the rest is for two-way TLS.

1 2 3 4 5 6 7 8

1. Security type: SSL or SASL SSL shows this section.
2. Truststore file: the certificates you trust. Blank for default trust. See Truststores.
3. Truststore type: the file's format. See formats.
4. Truststore password: blank for none; hidden for PEM files.
5. SSL validation: check the broker's host name against its certificate. See Hostname validation.
6. Two-way SSL/TLS? shows the keystore fields 7 for a client certificate. See Mutual TLS.
7. The keystore: your client certificate and its private key.
8. Want to create a JKS? opens the JKS Converter. See Need a JKS?

Truststores

A truststore holds the certificates of the certificate authorities (CAs) you trust; KafkIO accepts a broker whose certificate was signed by one of them. Whether you need one depends on who signed the brokers' certificates:

• A public CA (Confluent Cloud, Amazon MSK, Azure Event Hubs, Aiven and most managed services): leave Truststore file blank. Blank for default trust means the certificate authorities built into the Java runtime that comes with KafkIO, which include the public ones.
• Your organisation's own CA, or a self-signed certificate: you need a truststore with that CA's certificate (or the broker's own certificate) in it. Your Kafka administrator can usually give you one; it's the same file other Java clients use as ssl.truststore.location.

Pick the file with the folder button, or type or paste its path. KafkIO checks that the file exists (The specified file does not exist, and Save stays disabled), but not what's in it: a wrong password or a file in the wrong format only shows when you test the connection. KafkIO stores the path, not a copy of the file, so keep the file where it is (or edit the cluster when you move it).

Then set Truststore type to the file's format and, unless it's a PEM file, its Truststore password. Many truststores have one even though they hold no secrets; leave it blank if yours doesn't.

Supported certificate and key formats

The Truststore type and Keystore type lists offer the three formats the Kafka client reads. KafkIO passes your choice on as ssl.truststore.type or ssl.keystore.type:

A PEM truststore can hold several certificates one after the other, such as a CA and its intermediate. A PEM keystore needs the private key and the certificate (chain) in the same file, with the key in PKCS#8 format (-----BEGIN PRIVATE KEY-----, or -----BEGIN ENCRYPTED PRIVATE KEY----- for a key with a password).

Hostname validation

With SSL validation ticked, KafkIO also checks that each broker's certificate was issued for the host name it connected to, which is what stops someone in the middle from presenting a different valid certificate. That's Kafka's standard behaviour (ssl.endpoint.identification.algorithm=https). Unticked, KafkIO sets ssl.endpoint.identification.algorithm to empty and the check is skipped: the certificate must still be trusted, but any host name will do.

Mutual TLS (mTLS)

In two-way TLS, KafkIO proves who you are too, with a client certificate, and the broker uses that certificate's name as your identity (for ACLs, for example). You need it when the listener requires client authentication (ssl.client.auth=required on the broker); your administrator gives you a keystore, or a certificate and key to put in one.

Switch on Two-way SSL/TLS? and fill in the keystore fields that appear:

Turning two-way TLS off hides the keystore fields and stops KafkIO sending them; what you entered is kept.

Need a JKS?

Usually you don't: KafkIO reads PKCS12 and PEM files directly, so choose the matching type instead of converting. If you do want a Java keystore (to share one file with other Java tools, say), the Want to create a JKS? link at the top of the section opens the JKS Converter (also in the Tools menu), which builds one from other certificate and key files.

SASL

With SASL plaintext or SASL SSL, the SASL / JAAS config section appears. SASL is how you log in: a user name and password, a Kerberos ticket, an OAuth token or cloud credentials, depending on the mechanism. Start from one of the Examples under the JAAS config box rather than from scratch: they fill in every field for a common setup, and you replace the placeholders.

1 2 3 4 5 6

1. SASL type (sasl.mechanism). See SASL mechanisms.
2. SASL client callback handler and
3. SASL login callback handler: leave on <default> unless your mechanism needs one. See Callback handlers.
4. SASL other config: extra properties, one per line. See Additional properties.
5. JAAS config (sasl.jaas.config): required. See JAAS configuration.
6. The examples. See Configuration examples.

SASL mechanisms

The SASL type has to be one the broker's listener has enabled. The list offers the common ones; it's also a text field, so you can type any other mechanism name your broker and client libraries support. Left blank, Kafka's default applies, which is GSSAPI.

A mechanism the listener doesn't support fails straight away, with a message that lists the ones it does, for example UnsupportedSaslMechanismException: Client SASL mechanism 'SCRAM-SHA-256' not enabled in the server, enabled mechanisms are [PLAIN].

JAAS configuration

JAAS config is Kafka's sasl.jaas.config, and it's required with a SASL security type (Please enter JAAS config.). It names a login module, the word required, then the module's options as name="value" pairs, and ends with a semicolon:

org.apache.kafka.common.security.scram.ScramLoginModule required
username="storefront-app"
password="your-password";

Line breaks are optional (it can all be on one line), but the closing ; isn't: without it the client rejects the configuration. Put values in double quotes, especially ones containing =, ; or spaces, such as Azure connection strings. KafkIO passes the text to Kafka as it is, with spaces trimmed from the ends.

Callback handlers

Callback handlers are classes that do part of the login work for a mechanism, typically fetching a token. Kafka has two settings, and KafkIO a list for each:

• SASL login callback handler (sasl.login.callback.handler.class): used when logging in, for example to fetch an OAuth token from your identity provider.
• SASL client callback handler (sasl.client.callback.handler.class): used during each SASL exchange with a broker, for example to sign an AWS IAM request.

<default> leaves the setting out, so Kafka uses the mechanism's own handler. That's right for PLAIN, SCRAM and GSSAPI. The examples set a handler where one is needed. The lists are searchable and contain the handlers that ship with KafkIO:

The lists also hold Kafka's own handlers: the defaults for GSSAPI (KerberosClientCallbackHandler), PLAIN and SCRAM (SaslClientCallbackHandler) and OAUTHBEARER (OAuthBearerSaslClientCallbackHandler), which <default> already gives you, and OAuthBearerUnsecuredLoginCallbackHandler, for development brokers that accept unsigned tokens.

Additional properties

SASL other config takes any further client properties your setup needs, one key=value per line, typically the sasl.* settings of an OAuth handler:

sasl.oauthbearer.token.endpoint.url=https://login.example.com/oauth2/token
sasl.login.retry.backoff.ms=500
# lines starting with # are ignored
sasl.login.connect.timeout.ms=20000

• Spaces around the key and value are trimmed, and double quotes around a value are removed. Lines without an = are ignored.
• They're applied last, so a property here overrides the same one set by a field on the tab.
• They're only used with a SASL security type, and they aren't limited to sasl.*: any Kafka client property works.
• Recent Kafka clients refuse to contact an OAuth token endpoint that isn't on an allowed list (org.apache.kafka.sasl.oauthbearer.allowed.urls). KafkIO adds the URL of any property here whose name contains both auth and endpoint.url, such as sasl.oauthbearer.token.endpoint.url, to that list for you.

Kerberos service name

When SASL type is GSSAPI, a Kerberos Service Name field (sasl.kerberos.service.name) appears under it. It's the first part of the brokers' Kerberos principal: for brokers running as kafka/broker1.example.com@EXAMPLE.COM, enter kafka, which is the usual value. A sasl.kerberos.service.name line in SASL other config does the same and, if both are set, wins. The GSSAPI example uses that line, so fill in one or the other.

Configuration examples

The links under JAAS config fill in the SASL section for a common setup: the SASL type, both callback handlers, SASL other config and the JAAS config. Clicking one replaces everything in the SASL section, so pick the example first and then edit it. The examples don't change the Security type or the TLS fields: set those yourself (all the cloud services below need SASL SSL).

The values in quotes are placeholders. Replace them with your own, then click Test Connection (see Testing and common errors). For the full settings per provider, including bootstrap servers and Schema Registry, see Provider Recipes.

PLAIN

org.apache.kafka.common.security.plain.PlainLoginModule required
username="kafkauser"
password="kafkapassword";

Replace the user name and password. For Confluent Cloud, the user name is the API key and the password the API secret. PLAIN sends the password itself, so use it with SASL SSL anywhere but a development cluster.

SCRAM

org.apache.kafka.common.security.scram.ScramLoginModule required
username="kafkauser"
password="kafkapassword";

Replace the user name and password, and change SASL type to SCRAM-SHA-512 if that's what your user was created with (the example picks SCRAM-SHA-256). Amazon MSK's SCRAM authentication uses SCRAM-SHA-512.

Kerberos (GSSAPI)

com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
keyTab="/etc/security/keytabs/kafka_client.keytab"
principal="kafkaclient1@example.com";

The example logs in with a keytab: replace keyTab with the path to your keytab file and principal with your Kerberos principal. It also puts sasl.kerberos.service.name="serviceName" in SASL other config: change serviceName to the brokers' service name (usually kafka), or delete the line and use the Kerberos Service Name field, as here:

1 2 3

1. SASL type GSSAPI adds the field below it.
2. Kerberos Service Name: the brokers' service name.
3. The JAAS config with your keytab and principal. On Windows, write paths with forward slashes (or double the backslashes).

Kerberos also needs to know your realm and its KDC (key distribution centre). Usually that comes from a krb5.conf file: KafkIO uses the file in KRB5_CONFIG if that environment variable is set, or Java's default location otherwise. To point KafkIO at another file, or to set the realm and KDC host directly, use File ▸ Preferences ▸ Kerberos (see Kerberos preferences); those settings are applied when KafkIO starts, so restart it after changing them.

OAuth (OAUTHBEARER)

For brokers that accept OAuth 2.0 tokens from an identity provider such as Okta, Keycloak or Microsoft Entra ID, using Kafka's own OAuth support and the client credentials flow. The example sets the login callback handler to org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginCallbackHandler, and these two boxes:

sasl.oauthbearer.token.endpoint.url="https://login.microsoftonline.com/12345/oauth2/v2.0/token"
sasl.login.connect.timeout.ms=20000

org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required
clientId="clientId"
clientSecret="secret";

Replace the token endpoint URL (in SASL other config) with your provider's, and the client ID and secret (in the JAAS config) with your application's. If your provider needs a scope, add scope="..." to the JAAS options. KafkIO allows the token endpoint URL for you (see Additional properties).

Amazon MSK IAM

software.amazon.msk.auth.iam.IAMLoginModule required
awsProfileName="your-aws-profile";

For MSK clusters with IAM access control, using the AWS MSK IAM library that ships with KafkIO. The example sets SASL type AWS_MSK_IAM and the client callback handler software.amazon.msk.auth.iam.IAMClientCallbackHandler. Replace your-aws-profile with the name of a profile in your AWS credentials and config files (~/.aws/credentials, ~/.aws/config), or remove the option to use the AWS default credentials (environment variables, the default profile and so on). Use SASL SSL and the cluster's IAM bootstrap servers.

Strimzi OAuth

org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required
oauth.token.endpoint.uri="https://oauth.provider.com/token"
oauth.client.id="kafka-client-id"
oauth.client.secret="kafka-client-secret"
oauth.audience="kafka-broker"
oauth.ssl.truststore.location="/path/to/truststore.jks"
oauth.ssl.truststore.password="truststore-password"
oauth.ssl.truststore.type=jks;

For Strimzi clusters (Kafka on Kubernetes) whose listeners use Strimzi's OAuth library. The example sets the login callback handler to io.strimzi.kafka.oauth.client.JaasClientOauthLoginCallbackHandler; the options are all in the JAAS config. Replace the token endpoint, client ID and secret, and the audience (or remove that line if your provider doesn't use one). The three oauth.ssl.truststore.* lines are for the TLS connection to the token endpoint, not to Kafka: remove them if the endpoint's certificate is signed by a public CA. Keep the ; at the end of the last remaining line.

Azure Event Hubs: connection string

org.apache.kafka.common.security.plain.PlainLoginModule required
username="$ConnectionString"
password="Endpoint=sb://mynamespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=XXXXXXXXXXXXXXXX";

Event Hubs' Kafka endpoint with a shared access signature (SAS) policy. The user name is literally $ConnectionString; leave it as it is. Replace the password with the connection string of your namespace's (or event hub's) shared access policy, from the Azure portal. Use SASL SSL and mynamespace.servicebus.windows.net:9093 as the bootstrap server.

Azure Event Hubs: Microsoft Entra ID

org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required;

Event Hubs with Microsoft Entra ID (formerly Azure Active Directory) instead of a connection string. The JAAS config stays as it is: the work is done by KafkIO's own client callback handler, com.certak.kafkio.logic.kafka.pt.auth.azure.AzureEntraClientCallbackHandler, which the example selects. It gets a token for https://your-namespace-host/.default using Azure's standard credential chain (DefaultAzureCredential), so it uses whatever Azure sign-in is available on your machine, for example a service principal in the AZURE_CLIENT_ID, AZURE_TENANT_ID and AZURE_CLIENT_SECRET environment variables, or the account you signed in with using the Azure CLI (az login). That identity needs an Event Hubs data role on the namespace.

• Use SASL SSL, and exactly one bootstrap server, mynamespace.servicebus.windows.net:9093: the handler takes the token's audience from it and refuses a list.
• If no token can be had, KafkIO's log (View Logs in the error dialog) shows Failed to acquire Azure token for Event Hub Authentication, with the reason from Azure.

Testing and common errors

Click Test Connection at the bottom of the dialog after any change on this tab. Most security mistakes show up there as one of a few errors:

Show Details shows the full exception and View Logs KafkIO's log, which has the client's own messages. For more, see Troubleshooting Connections.

How passwords are stored

Everything on this tab is saved in your clusters.json file in plain text, exactly as you typed it: the truststore and keystore passwords, and the JAAS config with any password, client secret or connection string inside it. KafkIO doesn't encrypt the file or keep these values in your operating system's keychain. (The proxy password in Preferences ▸ Network is the exception: if you choose to save it, it goes into the system keychain, encrypted.)

• Truststores, keystores, keytabs and AWS or Azure credentials aren't copied into KafkIO: it saves the path or uses them from where they are, protected as they already are.
• The cluster configuration shown on a connected cluster's Overview page masks the truststore and keystore passwords, but shows the JAAS config, password included. Mind it when sharing your screen.
• Duplicating a cluster copies its credentials too.

So treat clusters.json like a password file: keep it in your own profile, and don't attach it to a ticket or commit it anywhere. Where you can, prefer credentials that don't live in the file at all (Kerberos keytabs, AWS profiles, Azure sign-in), or short-lived, narrowly scoped ones.

Related

• Managing Cluster Configurations
• General Connection Settings
• Provider Recipes
• Troubleshooting Connections
• JKS Converter
• Kerberos preferences
• Security & Authentication Matrix