Data export

Export pages, frontmatter, links, and embeddings as JSONL, CSV, or Parquet.

Data export streams every markdown page, or the pages under one folder, into a format that analytics tools, warehouses, and ML pipelines read directly. To render pages as PDF, HTML, slides, or a site instead, see Document export.

CLI

kiwifs export --format jsonl --include-content -o knowledge.jsonl

Formats

JSONL

One JSON object per page. Frontmatter is nested under frontmatter.

{"path":"concepts/auth.md","frontmatter":{"title":"Authentication","status":"published"},"word_count":412}

Optional fields appear when you ask for them:

Field	Added by	Content
content	include-content	The whole file, frontmatter included
links_out	include-links	Link targets found in the page
links_in	include-links	Paths of pages that link here
embeddings	include-embeddings	One object per chunk with chunk_idx, text, and vector

CSV

One row per page. The first column is always path.

path,name,status,grade
students/alice.md,Alice,active,A
students/bob.md,Bob,active,B+

Pass columns to choose frontmatter fields. Use dots for nested fields, such as author.name. Without columns, KiwiFS uses the frontmatter keys of the first page, skipping keys that start with _. Lists are joined with , .

Parquet

One row per page, Snappy-compressed, for warehouses and dataframes.

Column	Content
path	Page path
frontmatter	Frontmatter as a JSON string
tags, status	Copied out of frontmatter
word_count, heading_count	Page statistics
content	The whole file, with include-content
link_count, links_out, links_in, backlink_count	Link data, with include-links

Parquet doesn’t support embeddings. Use JSONL for vectors.

Options

CLI

Flag	Default	Description
--root, -r	./knowledge	Knowledge root
--format	jsonl	jsonl, csv, or parquet
--output, -o	stdout	Output file
--path		Only export pages whose path starts with this prefix, such as students/
--columns		Comma-separated frontmatter fields for CSV
--include-content	false	Include the full file
--include-links	false	Include outgoing and incoming links
--include-embeddings	false	Include vector embeddings
--limit	0 (no limit)	Maximum pages
--webhook		URL to notify after a successful export
--webhook-secret		Secret used to sign the webhook

REST

GET /api/kiwi/export streams the result with chunked transfer encoding.

Parameter	Default	Description
format	jsonl	jsonl, csv, or parquet
path		Path prefix
columns		Comma-separated frontmatter fields for CSV
include_content	false	Include the full file
include_links	false	Include link data
include_embeddings	false	Include embeddings. Not allowed with parquet.
limit	0 (no limit)	Maximum pages

The response Content-Type is application/x-ndjson, text/csv, or application/vnd.apache.parquet.

Terminal

curl "http://localhost:3333/api/kiwi/export?format=csv&path=students/&columns=name,status" \
  -o students.csv

MCP

The kiwi_export tool takes format, path, columns (array), include_content, include_embeddings, and limit, and returns the JSONL or CSV text in its result. It works in local mode (kiwifs mcp --root) only, and doesn’t return Parquet. Use the REST endpoint for binary output.

Embeddings

include-embeddings adds the stored vectors from vector search, so it returns nothing unless vector search is enabled and the index is built.

When you export embeddings from the CLI to a file, KiwiFS also writes a sidecar next to it, such as vectors.schema.json for -o vectors.jsonl. The sidecar describes the record fields and includes embedding_model, embedding_dimensions, exported_at, and record_count.

Notify a webhook

--webhook POSTs a JSON notification after the export finishes, which is useful for triggering a site rebuild or a warehouse load.

Terminal

kiwifs export --format jsonl -o data.jsonl --webhook https://example.com/hooks/kiwi

Payload

{
  "type": "export.completed",
  "format": "jsonl",
  "file_count": 248,
  "timestamp": "2026-10-19T18:46:15Z",
  "output": "data.jsonl"
}

With a secret, the request carries webhook-id, webhook-timestamp, webhook-signature, and X-Kiwi-Signature-256 headers. If you omit --webhook-secret, KiwiFS uses the secret of a configured webhook with the same URL. See Webhooks.

Recipes

Filter with jq

Terminal

# All drafts
kiwifs export | jq 'select(.frontmatter.status == "draft") | .path'

# Count pages by status
kiwifs export | jq -s 'group_by(.frontmatter.status) | map({status: .[0].frontmatter.status, count: length})'

Load into BigQuery

Terminal

kiwifs export --format csv --columns title,status,owner -o export.csv
bq load --source_format=CSV --skip_leading_rows=1 mydataset.knowledge export.csv

Read Parquet with pandas

Terminal

kiwifs export --format parquet --include-links -o kb.parquet
python -c "import pandas as pd; print(pd.read_parquet('kb.parquet').head())"

Build a retrieval dataset

Terminal

kiwifs export --format jsonl --include-content --include-embeddings -o vectors.jsonl

Related

Document export

Render PDF, HTML, slides, and static sites.

Export API

REST reference.