Dumping a Topic to a File

A dump writes a topic's messages to a text file, one message per line, with the fields you choose separated by a delimiter. Use it to hand data to someone else, load it into a spreadsheet or a script, or keep a copy of what was on a topic at a point in time. It reads the messages the search form on the Messages tab describes, so you can dump a whole topic or just the part you need.

Opening the dump dialog

Open the topic's Messages tab, set up the search form for the messages you want, then click the arrow beside Search and choose Dump Topic To File. You don't need to run the search first.

1 2

1. Search: the button runs a search, the arrow at its right opens the menu.
2. Dump Topic To File opens the Topic File Dump dialog.

The item is greyed out while the search type is Stream Live: a live stream has no end to write up to. The dialog takes a copy of the search form as it is when you open it, so changing the form afterwards doesn't change that dialog's dump; close it and open a new one instead.

How a dump uses your search

A dump reads exactly what a search with the same form would read, page after page, until there's nothing left in the range, and writes each message to the file instead of the table:

So to dump a whole topic, use Oldest, all partitions and no text filter. To dump part of it, narrow the form: Occurring after for the last hour's messages, one partition, or a text filter such as an order ID. Run the search first if you want to check what you'll get: the table shows the first page of the same messages.

The dump dialog

1 2 3 4 5 6 7

1. Dump file location: the file to write (The file location).
2. Included fields: the columns in the file (Choosing fields).
3. List column headers on first line (Column headers).
4. Remove newlines (if present) (Newlines in values).
5. Delimiter: what goes between fields (Delimiters).
6. Start Dump: dump now and wait (In the foreground).
7. Start In Background: dump while you carry on working (In the background).

Each time you open it, the dialog starts with the default fields and options; it doesn't remember the last dump's choices.

Choosing fields

Tick the fields you want. They're always written in the order below, whatever order you tick them in, and the column header is the name KafkIO writes on the first line.

At least one of Timestamp ISO string, Key and Value/message must stay ticked: untick the last of them and KafkIO ticks Value/message (or Key) again. A null key or value, or a missing field, is written as an empty field, so it looks the same as an empty string.

File format

The dump of orders set up above starts like this (tabs between fields):

TIMESTAMP_ISO	OFFSET	PARTITION	KEY	VALUE	HEADERS
2026-10-25T08:52:54.285Z	0	2	customer-9	{"orderId": 0, "customer": "customer-9", "amount": 215.75, "status": "NEW"}	[format=json,source=kafkio-lab]
2026-10-25T08:52:54.292Z	0	0	customer-5	{"orderId": 2, "customer": "customer-5", "amount": 448.63, "status": "NEW"}	[format=json,source=kafkio-lab]
2026-10-25T08:52:54.292Z	1	0	customer-10	{"orderId": 3, "customer": "customer-10", "amount": 63.04, "status": "PAID"}	[format=json,source=kafkio-lab]

One message per line, fields joined by the delimiter. Nothing is quoted or escaped, so the file is simplest to read back when the delimiter never appears in your data and the newlines are removed.

Column headers

List column headers on first line (on by default) writes the column names from the table above as the first line. Turn it off when you append dumps together or feed the file to a tool that doesn't expect a header row.

Delimiters

Delimiter is the text written between fields: a tab by default, shown as \t. Type any character or string, such as |, , or ;. These escapes are understood: \t (tab), \s (space), \n (newline) and \r (carriage return).

Pick a delimiter that can't occur in your keys, values or headers. JSON values are full of commas and colons, so , would split them into extra columns; a tab or | is usually safe. The header field also uses commas inside its brackets.

Newlines in values

Remove newlines (if present) (on by default) deletes line breaks from every field before writing it, so each message stays on one line. They're removed, not replaced with spaces: pretty-printed JSON is still valid JSON, but two lines of plain text run together. Untick it to keep values exactly as they are, for example to look at multi-line text; the file then no longer has one message per line.

The file location

KafkIO suggests a name made of the topic and the current date and time, such as orders_2026-10-25T09-10-42.dump, in the folder you last saved to (your home folder the first time). Type a different path, or use the folder button beside it to choose one. Any name and extension works: .tsv or .csv helps a spreadsheet open the file.

• Missing folders in the path are created.
• If the file exists, it's replaced. Typing the path of an existing file doesn't ask first (the folder button's save dialog does).
• After a dump the dialog stays open and suggests a fresh name in the same folder, ready for another dump.

Running a dump

In the foreground

Start Dump covers the dialog with a progress message while KafkIO reads and writes. The figures show the messages and data read for the current batch, not a running total.

1

1. Cancel stops the dump.

Cancelling stops reading and closes the file: what was written so far stays in it, without a completion message. A finished dump opens Topic Dump Complete (below).

In the background

Start In Background closes the dialog and runs the dump as a background task, so you can keep working, including in other topics and clusters. Use it for anything that takes more than a few seconds.

1 2

1. The background task indicator spins while a task runs; click it for the list.
2. Cancel stops that dump, keeping what's been written.

Each task is named after its file (Dump to clickstream_….dump), so you can run several dumps at once and tell them apart. If you quit KafkIO while one is running, it asks first (see Background tasks and cancelling).

When it finishes

1 2 3

1. Open Terminal: a terminal in the file's folder, to look at the file with your usual tools.
2. Open Folder: the folder in your file manager.
3. Done closes the message.

The File box can be selected and copied. Background dumps show the same message when they finish. If a dump fails (the cluster becomes unreachable, the file can't be written), an Error dumping topic dialog gives the reason; any messages written before the error stay in the file.

Large topics

Dumps are written a batch at a time and flushed to disk every few seconds, so KafkIO doesn't need memory for the whole topic. What grows is time, network traffic and the file:

• Use a large page size. The dump reads in batches of the Max page/poll size, and every batch has a cost. In the lab, a 1,000,000-message topic dumped in seconds with the default 5,000, and several times slower with 200. Set 50,000 or more before opening the dialog for big topics, especially with a text filter.
• Reading forwards is cheaper. Kafka is built to be read from oldest to newest. Newest, Offset end and Occurring before read backwards, which takes extra work per batch; prefer Oldest, Offset start or Occurring after when the order of the file doesn't matter.
• Every message crosses the network. A text filter doesn't help here: Kafka can't filter, so KafkIO downloads every message in the range to check it. On a cloud cluster that's data transfer you may pay for, and on a busy production cluster it's load. Narrow the range (partitions, a start offset or date) rather than relying on the filter alone.
• Plan for the file size. Values are written in full (no trim), as text, plus the other fields. A million 200-byte messages made a 228 MB file with the default fields. Avro and Protobuf values usually get bigger, because they're written as JSON.

The dialog's own text puts it plainly: a dump works on large topics, but it isn't built for dumping them quickly. For regular bulk exports, a Kafka consumer or a Kafka Connect sink is the better tool.

Related

• Searching Messages
• Deserialization & Formats
• Copying messages (for a handful of messages)
• Streaming Messages Live
• Background tasks and cancelling