Network, Proxy & Trust

The Network page of File ▸ Preferences controls how KafkIO makes its HTTP connections: the check for new versions, and the REST APIs of Schema Registry, Kafka Connect and ksqlDB. Set it up when you're behind a corporate proxy, or when those endpoints use certificates from your organisation's own certificate authority. None of it affects the connections to the Kafka brokers themselves.

1 2 3 4 5 6 7 8 9 10

1. Check for latest version online.
2. Custom proxy switches the proxy on, as HTTP or SOCKS.
3. Auto-detect fills in the proxy your system uses.
4. Hostname and, below it, Port of the proxy.
5. Proxy requires authentication unlocks Username and Password.
6. Persist password between sessions keeps the password in your system's keychain.
7. Check connection tests the settings above before you save them.
8. Also set JDK proxy system properties, for OAuth sign-ins behind a proxy. The values it would set are listed under it.
9. http.nonProxyHosts, hosts to reach without the proxy (not applied in the current version: see the warning below).
10. Accept non-trusted certificates automatically, and below it (scrolled out of view here) the list of custom certificates.

As on the other preference pages, nothing changes until you click Save, and Cancel throws your edits away (see Opening Preferences).

Update checks

Check for latest version online 1 is ticked by default: KafkIO checks for a new release shortly after it starts and then once a day, and shows a New version available dialog when there is one. Untick it to turn the automatic check off; Help ▸ Check for Updates still works whenever you want to check by hand. Updating KafkIO describes the dialog and the manual check.

Proxy

KafkIO talks to Kafka brokers with Kafka's own protocol, which doesn't go through HTTP proxies: the proxy never applies to broker connections. If the brokers are only reachable from your organisation's network, you need to be on it, or on its VPN. The proxy is for KafkIO's HTTP traffic only:

So setting up a proxy is two steps: configure it here, once, then tick Use proxy (if configured) on each REST endpoint that must use it, in the cluster's Schema Registry, Kafka Connect and ksqlDB tabs. Endpoints inside your network usually must not use the proxy; public ones, such as a managed Schema Registry in the cloud, often must. The REST clients pick up proxy changes when they're created, so disconnect and connect the cluster again after you change the proxy.

Custom HTTP or SOCKS proxy

1. Switch the proxy on

   Tick Custom proxy 2. The rest of the section is greyed out until you do. Choose HTTP (the default, and what most corporate proxies are) or SOCKS; given the limits in the table above, SOCKS is only worth choosing for the update check and Kafka Connect.

2. Enter the host and port

   Type the proxy's Hostname 4 without http://, for example proxy.corp.example.com, and its Port (80 until you change it; 3128 and 8080 are common). Your browser's or operating system's proxy settings, or your IT department, will tell you both; or try Auto-detect.

3. Check and save

   Add a username and password if the proxy needs them, click Check connection 7, then Save.

With Custom proxy ticked, Save refuses an empty host name or a port that isn't a number from 1 to 65534, with an error such as "Network > Proxy configuration: Please specify a hostname". Unticking Custom proxy keeps the values you entered, ready for next time.

Auto-detect

Auto-detect 3 (enabled once Custom proxy is ticked) looks for the proxy your system is set up to use, from the operating system's and browsers' proxy settings, including automatic configuration scripts. A Proxy Auto-detect message says how many it found ("Detected proxies: 1"), and KafkIO fills in the type, host name and port of the first. If it found more than one, click Auto-detect again to cycle through them. With none found, the fields stay as they were: you're probably connected directly, or the proxy is set up in a way KafkIO can't read, so ask your IT department for the address.

Proxy authentication

If the proxy asks for a login, tick Proxy requires authentication 5 and enter the Username and Password. Press and hold the eye icon at the end of the password field to see what you typed. KafkIO sends them as basic proxy authentication; proxies that only accept Windows (NTLM or Kerberos) authentication can't be used. Remember that Schema Registry gets neither (see the table above).

Where the password is kept depends on Persist password between sessions 6:

• Ticked: KafkIO saves the password, encrypted, in your operating system's keychain (Windows Credential Manager, macOS Keychain, or the Secret Service on Linux), never in its preferences or in clusters.json. If no keychain is available the box is greyed out, with the tooltip "Credential store not available."
• Unticked: the password is kept in memory only. Each time KafkIO starts it asks for it in a Proxy configuration dialog ("Proxy requires password"); if you cancel, requests through the proxy go without a password and are likely to be refused.

Unticking the box later doesn't remove a password already saved in the keychain. To get rid of it, clear the Password field and save while the box is still ticked, then untick it.

Checking the proxy

Check connection 7 tries to open a public website (https://www.google.ie) with the settings on the page, before you save them, and answers in a Check Proxy Setting message: "Connection successful!" or "Connection failed!" (the reason is in the log). With Custom proxy unticked it tests a direct connection instead. A success proves that the proxy and login work for the internet; it doesn't prove that your Schema Registry or Connect cluster can be reached through it. Use Test Schema Registry, Test Kafka Connect or Test ksqlDB in the cluster dialog for that.

JDK proxy system properties

Some broker sign-ins fetch an OAuth token over HTTP with libraries that KafkIO can't hand a proxy to: the note on the page names Strimzi, AWS IAM, Azure Event Hubs and Kafka's own OAuth libraries (see Security & Authentication). These libraries do read Java's standard proxy settings. Tick Also set JDK proxy system properties 8 and KafkIO sets http.proxyHost, http.proxyPort and their https.* equivalents from the fields above, plus http.proxyUser and http.proxyPassword when the proxy needs a login. The list under the box shows the values (the password masked); it's greyed out while the box is unticked.

Leave it off unless such a sign-in fails behind a proxy: the properties apply to every part of KafkIO that uses Java's own HTTP client, not only to the endpoints you chose. They're set when you save, but libraries that have already started may have read the old values, so restart KafkIO if you've connected to clusters since it started (the page says so too). The properties are for HTTP proxies; with SOCKS selected they still describe an HTTP proxy at that address.

Trusted certificates

The SSL configuration section decides which server certificates KafkIO accepts on HTTPS connections: the update server, and Schema Registry and Kafka Connect endpoints on https:// URLs whose Truststore file is left blank in the cluster dialog. An endpoint with its own truststore file uses that file instead (see TLS for Schema Registry). ksqlDB endpoints only follow Accept non-trusted certificates automatically: the certificate list doesn't apply to them (see below).

1 2 3 4

1. Accept non-trusted certificates automatically, ticked by default.
2. A certificate you added, with its issuer, validity dates and signature algorithm (a Version column is further right when the window is wide enough).
3. The bin icon removes the certificate from the list.
4. Add certificate picks a certificate file.

Accepting untrusted certificates

With Accept non-trusted certificates automatically 1 ticked, which is the default, KafkIO accepts any certificate on these connections: self-signed, from an unknown authority or expired. That's why an https:// Schema Registry with an internal certificate usually works without any setup. The price is that KafkIO can't tell your real server from one impersonating it, which is why the hint on the box says it's not recommended for production environments.

Untick it to have certificates checked properly. KafkIO then trusts the certificate authorities that Java trusts (the well-known public ones), plus the certificates in the list below. An endpoint whose certificate is neither then fails with an error such as PKIX path building failed, until you add its certificate. For ksqlDB the box overrides the endpoint's own truststore too: while it's ticked, any ksqlDB certificate is accepted (ksqlDB Connections). With it unticked, a ksqlDB endpoint with a blank truststore trusts only the certificate authorities Java trusts, not the certificates in the list: for a ksqlDB server with an internal or self-signed certificate, choose a truststore on its tab.

Adding custom certificates

To trust the certificates of your organisation's own certificate authority (CA), or one self-signed server, without accepting everything:

1. Get the certificate

   Ask for the CA's certificate as a .crt or .cer file (PEM or DER). Adding the CA rather than each server's certificate covers every server it signs, and survives server certificate renewals. The file is read as one certificate: for a chain or bundle, add each certificate from its own file.

2. Add it

   Click Add certificate 4 and choose the file. It appears in the list with its serial number, issuer, validity and signature algorithm, so you can check you picked the right one and see when it expires. Adding the same certificate twice gives a Duplicate Certificates warning.

3. Turn off automatic acceptance, save and restart

   Untick Accept non-trusted certificates automatically 1 (while it's ticked, the list makes no difference), click Save, and restart KafkIO.

Any change in this section, whether the box or the list, ends with a Restart required message ("SSL configuration changed"): KafkIO sets up its certificate trust when it starts, so the change takes effect the next time you start it. To remove a certificate, click the bin icon 3 on its row, then save and restart.

Certificates added here apply to every cluster's Schema Registry and Kafka Connect endpoints at once (not ksqlDB), which saves choosing a truststore file on each endpoint. They're stored in ct_truststore.jks, in the ssl folder of KafkIO's data directory (File Locations), or next to the KafkIO program in Memory Stick Mode. Resetting all preferences removes them (Managing Preferences).

Related

• Troubleshooting: proxies and corporate networks
• Troubleshooting: self-signed certificates on REST endpoints
• Schema Registry Connections
• Security & Authentication
• Updating KafkIO
• Kafka & Timeouts
• Kerberos
• Privacy: what the update check sends