Import and export

Pull data from databases, files, and SaaS tools into pages, and stream your knowledge base out as JSONL, CSV, Parquet, or rendered documents.

Every import turns source rows or files into markdown pages with frontmatter and writes them through the normal write pipeline, so they are committed to git, indexed, and broadcast like any other write. Exports read the same files back out. All paths below are relative to http://localhost:3333/api/kiwi.

A typical import has three steps:

1. Browse

   For databases, list tables or collections with POST /import/browse.

2. Preview

   Render sample pages with POST /import/preview and get suggested field mappings from POST /import/infer-fields. Nothing is written.

3. Import

   Send the same body to POST /import. The server saves the connection (without secrets) so you can re-run it later.

Run an import

Import records from a source and write one markdown page per record.

POST /api/kiwi/import

from selects the source. The other fields depend on the source (see Sources). /import/preview and /import/infer-fields accept the same body.

from · string · required

Source type, for example postgres, csv, or obsidian.

path · string

Directory on the server to read from. Required for markdown and obsidian.

file · string

File on the server to read from. Required for csv, json, jsonl, and bibtex. Also accepted by croissant.

url · string

Croissant metadata URL, as an alternative to file.

dsn · string

Connection string for postgres and mysql. Also accepted as the URI for mongodb.

uri · string

Connection URI for mongodb.

db · string

SQLite database file path. Required for sqlite.

table · string

Table to import from postgres, mysql, or sqlite. SQL sources need either table or query.

query · string

SQL query to run instead of reading a whole table.

database · string

Database name. Required for mongodb.

collection · string

Collection name. Required for mongodb and firestore.

project · string

Google Cloud project ID. Required for firestore.

credentials · object

Service account JSON for firestore. Never stored.

database_id · string

Notion database ID. Required for notion.

base_id · string

Airtable base ID. Required for airtable.

table_id · string

Airtable table ID. Required for airtable.

api_key · string

API key for notion or airtable. Falls back to the NOTION_API_KEY or AIRTABLE_API_KEY environment variable. Never stored.

prefix · string

Directory to write pages into. Defaults to the source name.

id_column · string

Field used as the page filename. Defaults to the source’s primary key.

columns · string[]

Only keep these fields in frontmatter.

field_mappings · object[]

Rename, retype, or skip fields. Each entry is {"source", "target", "type", "skip"}, where type is string, number, boolean, or date. When you send any mapping, unmapped fields are dropped.

dry_run · boolean · default: "false"

Count what would be written without writing anything.

limit · integer · default: "0"

Stop after this many records. 0 means no limit.

via · string

Force a connector path: builtin, airbyte (local Docker), or airbyte-cloud. Leave empty to pick automatically.

airbyte_config · object

Raw Airbyte connector config. Sending it routes the import through Airbyte.

airbyte_image · string

Override the Airbyte Docker image.

streams · string[]

Airbyte streams to sync. Defaults to all streams.

X-Actor · string · default: "api-import"

Git commit author for the imported pages.

Postgres table

curl -X POST 'http://localhost:3333/api/kiwi/import' \
  -H 'Content-Type: application/json' \
  -H 'X-Actor: agent:importer' \
  -d '{
    "from": "postgres",
    "dsn": "postgres://user:pass@localhost:5432/mydb",
    "table": "articles",
    "prefix": "articles",
    "id_column": "slug",
    "columns": ["title", "status", "updated_at"]
  }'

imported · integer

Pages created or updated.

skipped · integer

Records whose page was already up to date.

archived · integer

Pages marked with _archived_at because their source row disappeared. Omitted when zero.

errors · string[]

Per-record error messages.

Response

{"imported": 42, "skipped": 3, "archived": 1, "errors": []}

Returns 400 when from or a required source field is missing or the source type is unknown, and 500 when the import fails.

path, file, and db are paths on the server’s filesystem. To import a file from your own machine, use POST /import/upload.

The request is synchronous. For syncable sources (postgres, mysql, mongodb, firestore, notion, airtable, firebase-rtdb), a run without dry_run or limit is a full sync: pages whose source rows no longer exist are archived. A successful import also saves its settings as a connection and turns on hourly auto-sync for syncable sources.

Sources

from	Required fields
markdown	path
obsidian	path (the vault folder)
csv	file
json, jsonl	file
bibtex	file
croissant	file or url
sqlite	db, plus table or query
postgres	dsn, plus table or query
mysql	dsn, plus table or query
mongodb	uri (or dsn), database, collection
firestore	project, collection, optional credentials
notion	database_id, api_key
airtable	base_id, table_id, api_key
firebase-rtdb and other Airbyte connectors	airbyte_config (needs Docker or an Airbyte Cloud key)

For Excel and YAML files, use the kiwifs import CLI. See Import overview for source-specific guides.

Upload a file and import it

Upload a local file and import, preview, or infer fields from it in one request.

POST /api/kiwi/import/upload (multipart/form-data)

file · file · required

The data file. Maximum 256 MB.

from · string · required

One of csv, json, jsonl, bibtex, or sqlite. Excel and YAML uploads are accepted by the form but not yet imported; use the kiwifs import CLI for those.

mode · string · default: "import"

import writes pages, preview returns the first 5 rendered records, and infer-fields returns suggested field mappings.

prefix · string

Target directory. Defaults to the uploaded filename without its extension.

id_column · string

Field used as the page filename.

field_mappings · string

Field mappings as a JSON-encoded array, in the same shape as POST /import.

table · string

Table name for sqlite. A sqlite upload needs either table or query.

query · string

SQL query for sqlite.

curl -X POST 'http://localhost:3333/api/kiwi/import/upload' \
  -H 'X-Actor: agent:importer' \
  -F 'file=@customers.csv' \
  -F 'from=csv' \
  -F 'id_column=id' \
  -F 'mode=preview'

The response depends on mode: the import response, the preview response, or the infer-fields response. Returns 400 for an unsupported from and 413 when the file is over 256 MB.

List sources

List the source types this server can import from, and whether Airbyte connectors are available.

GET /api/kiwi/import/sources

curl 'http://localhost:3333/api/kiwi/import/sources'

builtin · string[]

Sources handled natively by KiwiFS.

airbyte · string[] | null

Airbyte connector sources. null when neither Docker nor an Airbyte Cloud key is available.

docker_available · boolean

Whether a Docker daemon is reachable for local Airbyte connectors.

cloud_key_present · boolean

Whether an Airbyte Cloud API key is configured.

Response

{
  "builtin": ["markdown", "obsidian", "csv", "json", "jsonl", "excel", "yaml", "bibtex", "croissant", "sqlite", "postgres", "mysql", "mongodb", "firestore"],
  "airbyte": ["firebase-rtdb", "notion", "airtable"],
  "docker_available": true,
  "cloud_key_present": false
}

The order of the lists is not guaranteed.

Browse tables and collections

List the tables or collections in a database before you import.

POST /api/kiwi/import/browse

from · string · required

postgres, mysql, mongodb, or firestore.

dsn · string

Connection string for postgres and mysql.

uri · string

Connection URI for mongodb. dsn also works.

database · string

Database name. Required for mongodb.

project · string

Project ID. Required for firestore.

credentials · object

Service account JSON for firestore.

curl -X POST 'http://localhost:3333/api/kiwi/import/browse' \
  -H 'Content-Type: application/json' \
  -d '{"from": "postgres", "dsn": "postgres://user:pass@localhost:5432/mydb"}'

tables · object[]

Show properties

name · string

Table or collection name.

estimated_count · integer

Approximate row count, when the database reports one.

Response

{"tables": [{"name": "articles", "estimated_count": 1280}, {"name": "comments", "estimated_count": 9400}]}

Returns 400 for other source types or when the connection fails.

Preview an import

Render the first few records as pages without writing anything.

POST /api/kiwi/import/preview

Takes the same body as POST /import, including path for markdown and obsidian and query for SQL sources, plus:

limit · integer · default: "5"

Number of records to preview. Values outside 1–20 fall back to 5.

curl -X POST 'http://localhost:3333/api/kiwi/import/preview' \
  -H 'Content-Type: application/json' \
  -d '{"from": "obsidian", "path": "/data/vaults/research", "limit": 3}'

records · object[]

Show properties

path · string

Page path the record would be written to.

frontmatter · object

Frontmatter the page would get, including _source and _source_id.

body_preview · string

Page body after the frontmatter, cut at 800 characters.

Response

{
  "records": [
    {
      "path": "research/reading-list.md",
      "frontmatter": {"_source": "obsidian", "_source_id": "reading-list.md", "tags": ["papers"]},
      "body_preview": "# Reading list\n\n- [[Attention is all you need]]\n..."
    }
  ]
}

Infer field mappings

Sample up to 100 records and suggest a field mapping with detected types.

POST /api/kiwi/import/infer-fields

Takes the same body as POST /import/preview, including path and query. prefix, columns, field_mappings, and limit are ignored.

curl -X POST 'http://localhost:3333/api/kiwi/import/infer-fields' \
  -H 'Content-Type: application/json' \
  -d '{"from": "postgres", "dsn": "postgres://user:pass@localhost:5432/mydb", "query": "SELECT * FROM articles"}'

fields · object[]

Show properties

source · string

Field name in the source.

target · string

Suggested frontmatter key.

type · string

Detected type: string, number, date, or boolean.

Response

{
  "fields": [
    {"source": "published_at", "target": "published_at", "type": "date"},
    {"source": "title", "target": "title", "type": "string"},
    {"source": "views", "target": "views", "type": "number"}
  ]
}

Fields whose names start with _ are left out. Edit the result as needed and pass it as field_mappings to preview or import.

Saved connections

A connection is the non-secret part of an import: source, table, prefix, columns, and sync settings. KiwiFS saves one automatically after each successful POST /import (one per source and prefix) and keeps them in .kiwi/state/imports.json. Credentials and API keys are never stored, so you send them again when you re-run a connection that needs them.

Connection object

id · string

Connection ID (UUID).

from · string

Source type.

name · string

Display name, for example postgres:articles.

prefix · string

Target directory.

dsn, uri, table, collection, database, database_id, base_id, table_id, project, id_column, columns · various

Source settings, as sent on import.

last_run · string

RFC 3339 time of the last run.

last_stats · object

imported, skipped, archived, and errors from the last run.

created_at · string

RFC 3339 creation time.

sync_enabled · boolean

Whether auto-sync is on.

sync_interval · string

Go duration such as 5m, 1h, or 24h.

next_sync · string

RFC 3339 time of the next scheduled sync.

sync_status · string

idle, running, or error.

sync_error · string

Last sync error, if any.

via, airbyte_image, streams · various

Airbyte settings, when the import ran through Airbyte.

List connections

List saved import connections.

GET /api/kiwi/import/connections

curl 'http://localhost:3333/api/kiwi/import/connections'

Returns an array of connection objects, or [] when the connection store is unavailable.

Response

[
  {
    "id": "6f1c2a4e-8b0d-4c1e-9a77-2f3b5d9e1c40",
    "from": "postgres",
    "name": "postgres:articles",
    "dsn": "postgres://user:pass@localhost:5432/mydb",
    "table": "articles",
    "prefix": "articles",
    "last_run": "2026-10-19T14:02:11Z",
    "last_stats": {"imported": 42, "skipped": 3},
    "created_at": "2026-10-12T09:30:00Z",
    "sync_enabled": true,
    "sync_interval": "1h",
    "next_sync": "2026-10-19T15:02:11Z",
    "sync_status": "idle"
  }
]

dsn and uri are saved exactly as sent, including any password inside them.

Save a connection

Create or update a connection without running an import.

POST /api/kiwi/import/connections

The body is a connection object. from is required, and name defaults to from. Include id to update an existing connection or omit it to create one.

curl -X POST 'http://localhost:3333/api/kiwi/import/connections' \
  -H 'Content-Type: application/json' \
  -d '{"from": "notion", "name": "Notion roadmap", "database_id": "a1b2c3d4e5f6", "prefix": "roadmap"}'

Returns 200 with the saved connection, including its id. Returns 400 without from and 503 when the connection store is unavailable.

Delete a connection

Remove a saved connection. Pages it imported are not touched.

DELETE /api/kiwi/import/connections/:id

curl -X DELETE 'http://localhost:3333/api/kiwi/import/connections/6f1c2a4e-8b0d-4c1e-9a77-2f3b5d9e1c40'

Returns 204 on success and 404 for an unknown ID.

Run a connection

Re-run a saved connection’s import.

POST /api/kiwi/import/connections/:id/run

id · string · required

Connection ID.

credentials · object

Service account JSON for this run only (firestore).

api_key · string

API key for this run only (notion, airtable).

curl -X POST 'http://localhost:3333/api/kiwi/import/connections/6f1c2a4e-8b0d-4c1e-9a77-2f3b5d9e1c40/run' \
  -H 'Content-Type: application/json' \
  -d '{"api_key": "secret_abc123"}'

Returns the same body as POST /import and updates last_run and last_stats. Returns 404 for an unknown ID.

Configure auto-sync

Turn auto-sync on or off for a connection, or change its interval.

POST /api/kiwi/import/connections/:id/sync

id · string · required

Connection ID.

enabled · boolean

Turn auto-sync on or off. Omit to keep the current state.

interval · string

Go duration such as 15m, 1h, or 24h. Invalid values fall back to 1h.

curl -X POST 'http://localhost:3333/api/kiwi/import/connections/6f1c2a4e-8b0d-4c1e-9a77-2f3b5d9e1c40/sync' \
  -H 'Content-Type: application/json' \
  -d '{"enabled": true, "interval": "6h"}'

Returns the updated connection object with next_sync rescheduled. Returns 404 for an unknown ID.

Scheduled syncs run without credentials, so they only work for sources that don’t need any at run time, such as a postgres DSN that includes its password.

Sync status

List every connection with auto-sync enabled.

GET /api/kiwi/import/sync/status

curl 'http://localhost:3333/api/kiwi/import/sync/status'

Returns an array of connection objects where sync_enabled is true. Check sync_status and sync_error for failures.

Airbyte endpoints

Configure Airbyte connectors, locally through Docker or through Airbyte Cloud. Airbyte Cloud needs airbyte_api_key and airbyte_workspace_id under [import] in .kiwi/config.toml, or the AIRBYTE_API_KEY and AIRBYTE_WORKSPACE_ID environment variables.

Method	Path	Body	Returns
POST	/import/airbyte/spec	from or airbyte_image	Connector config schema. Without Docker but with a Cloud key: {"mode": "cloud", "message"}.
POST	/import/airbyte/check	from or airbyte_image, config	Connection check result. Needs Docker.
POST	/import/airbyte/discover	from or airbyte_image, config	Stream catalog. Needs Docker.
POST	/import/airbyte-cloud/check	from, config	Check result plus the temporary source_id.
POST	/import/airbyte-cloud/discover	source_id	{"streams": [...]}
GET	/import/airbyte-cloud/connections	none	{"connections": [...]} for the workspace.
POST	/import/airbyte-cloud/sync	connection_id	Triggered job details.

The local endpoints return 503 when Docker is not running. The Cloud endpoints return 503 without an API key and 502 when Airbyte Cloud rejects the request.

curl -X POST 'http://localhost:3333/api/kiwi/import/airbyte/check' \
  -H 'Content-Type: application/json' \
  -d '{"from": "notion", "config": {"credentials": {"auth_type": "token", "token": "secret_abc123"}}}'

See Airbyte import for the full walkthrough.

Ingest a document

Convert a PDF, Office file, image, or other document to markdown and write it as one or more pages.

POST /api/kiwi/ingest

file · string · required

Path to the file on the server. Supported extensions: pdf, docx, pptx, xlsx, html, htm, csv, json, xml, epub, jpg, jpeg, png, gif, bmp, tiff, mp3, wav, m4a, zip.

split_mode · string · default: "single"

single writes one page. sections writes one page per heading when the document has more than one section.

prefix · string

Directory for the output pages. Defaults to imports/<file-name-slug>/.

extract_keywords · boolean · default: "false"

Extract keywords for each section.

max_keywords · integer · default: "10"

Keywords to extract per section.

convert_crossrefs · boolean · default: "false"

Turn references like “see Section 2.1” into wiki links.

actor · string · default: "kiwi-ingest"

Git commit author.

curl -X POST 'http://localhost:3333/api/kiwi/ingest' \
  -H 'Content-Type: application/json' \
  -d '{"file": "/data/inbox/q3-report.pdf", "split_mode": "sections", "extract_keywords": true}'

source_file · string

The input file.

format · string

Detected input format.

total_sections · integer

Sections found in the converted document.

output_files · string[]

Pages written.

top_keywords · string[]

Extracted keywords, when requested.

Response

{
  "source_file": "/data/inbox/q3-report.pdf",
  "format": "pdf",
  "total_sections": 3,
  "output_files": ["imports/q3-report/summary.md", "imports/q3-report/revenue.md", "imports/q3-report/outlook.md"],
  "top_keywords": ["revenue", "churn", "pipeline"]
}

Returns 400 for a missing file or an unsupported extension.

Export pages

Stream every markdown page, or the pages under a path, as JSONL, CSV, or Parquet.

GET /api/kiwi/export

format · string · default: "jsonl"

jsonl, csv, or parquet.

path · string

Only export pages whose path starts with this prefix, for example concepts/.

columns · string

Comma-separated frontmatter fields to use as CSV columns. Without it, columns are detected from the first page.

include_content · boolean · default: "false"

Include the full markdown file in content.

include_links · boolean · default: "false"

Include outgoing (links_out) and incoming (links_in) wiki links.

include_embeddings · boolean · default: "false"

Include vector chunks. Not supported with parquet.

limit · integer · default: "0"

Maximum pages to export. 0 means no limit.

JSONL with content

curl 'http://localhost:3333/api/kiwi/export?format=jsonl&path=concepts/&include_content=true' > concepts.jsonl

The response streams with Content-Type set to application/x-ndjson, text/csv, or application/vnd.apache.parquet. Each JSONL line is one page:

path · string

Page path.

frontmatter · object

Parsed frontmatter.

word_count · integer

Words in the body.

content · string

Full file, with include_content=true.

links_out · string[]

Outgoing wiki links, with include_links=true.

links_in · string[]

Pages linking here, with include_links=true.

embeddings · object[]

chunk_idx, text, and vector for each chunk, with include_embeddings=true.

concepts.jsonl

{"path":"concepts/auth.md","frontmatter":{"title":"Authentication","status":"verified"},"word_count":412,"links_out":["users","billing"],"links_in":["concepts/users.md"]}
{"path":"concepts/billing.md","frontmatter":{"title":"Billing","status":"draft"},"word_count":230}

CSV output always starts with a path column:

pages.csv

path,title,status,updated
concepts/auth.md,Authentication,verified,2026-10-02
concepts/billing.md,Billing,draft,2026-09-20

Returns 400 for an unknown format, a negative limit, or include_embeddings with parquet.

Export a document

Render a page or folder to PDF, HTML, slides, or a static site and download the result.

POST /api/kiwi/export/document

format · string · required

pdf, html, slides, or site.

path · string · required

File or directory to render.

curl -X POST 'http://localhost:3333/api/kiwi/export/document' \
  -H 'Content-Type: application/json' \
  -d '{"format": "pdf", "path": "reports/q3.md"}' \
  -o q3.pdf

The response is the rendered file with a Content-Disposition: attachment header. Rendering times out after 5 minutes. See Document export for theme, bibliography, slide, and site options.

Related

Import overview

Source guides and the kiwifs import CLI.

Saved connections

Re-running imports and auto-sync.

Airbyte import

Docker and Airbyte Cloud connectors.

Export overview

Formats and when to use each.