Airbyte import

Run Airbyte source connectors in Docker and write their records into KiwiFS.

KiwiFS speaks the Airbyte protocol, so it can run an Airbyte source connector as a Docker container and turn every record it emits into a markdown page. Use it for sources that have an Airbyte connector but no built-in importer, or when you want a connector’s incremental streams.

For one-off imports of supported sources, the built-in importers (kiwifs import --from notion, and so on) need no Docker and are simpler.

Requirements

• Docker must be reachable from the KiwiFS process. KiwiFS checks this with docker info.
• Airbyte imports run over REST only, through POST /api/kiwi/import and the web UI import wizard. The CLI and MCP tool don’t run connectors.

Built-in connector images

from	Image
firebase-rtdb	airbyte/source-firebase-realtime-database:latest
notion	airbyte/source-notion:latest
airtable	airbyte/source-airtable:latest

To use any other connector, pass its image in airbyte_image, or pass the image name as from (any value with a / is treated as an image).

Run an import

1. Get the connector spec

   The spec is the JSON Schema for the connector’s config.

   Terminal

   curl -X POST http://localhost:3333/api/kiwi/import/airbyte/spec \
     -H "Content-Type: application/json" \
     -d '{"from": "firebase-rtdb"}'
   
2. Check the config

   Terminal

   curl -X POST http://localhost:3333/api/kiwi/import/airbyte/check \
     -H "Content-Type: application/json" \
     -d '{"from": "firebase-rtdb", "config": {"database_name": "my-app", "google_application_credentials": "..."}}'
   
3. Discover streams

   Terminal

   curl -X POST http://localhost:3333/api/kiwi/import/airbyte/discover \
     -H "Content-Type: application/json" \
     -d '{"from": "firebase-rtdb", "config": {"database_name": "my-app", "google_application_credentials": "..."}}'
   

   The response is the connector’s catalog of streams.

4. Import

   Terminal

   curl -X POST http://localhost:3333/api/kiwi/import \
     -H "Content-Type: application/json" \
     -d '{
       "from": "firebase-rtdb",
       "via": "airbyte",
       "airbyte_config": {"database_name": "my-app", "google_application_credentials": "..."},
       "streams": ["users"],
       "prefix": "firebase"
     }'
   

   Omit streams to sync every stream.

The spec, check, and discover endpoints all take from, an optional airbyte_image, and, except for spec, config. Each returns 503 when Docker isn’t available.

Import request fields

Field	Description
via	airbyte to force a connector. builtin to force the built-in importer. Empty picks automatically.
airbyte_config	Connector config that matches its spec. Sending it also selects Airbyte.
airbyte_image	Override the connector image
streams	Streams to sync. Empty means all.

For notion and airtable over REST, KiwiFS prefers the Airbyte connector when Docker is available and you send an api_key. It translates the key into connector config for you.

Firebase Realtime Database records

Firebase RTDB emits one record per top-level key. When a key holds a collection, such as users mapping user IDs to objects, KiwiFS writes one page per child instead of one page for the whole tree. This keeps large databases searchable and linkable.

Auto-sync

firebase-rtdb is a syncable source. A successful REST import saves the connection, including its image and streams, and re-runs it every hour by default.

Airbyte Cloud Experimental

These endpoints manage sources and connections in an Airbyte Cloud workspace. They don’t import records into KiwiFS. Use Docker mode to get data into your knowledge base.

Method	Path	Body	Purpose
POST	/api/kiwi/import/airbyte-cloud/check	from, config	Create a source in the workspace and test it. Returns status and source_id.
POST	/api/kiwi/import/airbyte-cloud/discover	source_id	List the source’s streams
GET	/api/kiwi/import/airbyte-cloud/connections		List the workspace’s connections
POST	/api/kiwi/import/airbyte-cloud/sync	connection_id	Start a sync job for an existing Airbyte Cloud connection

Configure the workspace in .kiwi/config.toml. Values support ${VAR} expansion.

.kiwi/config.toml

[import]
airbyte_api_key = "${AIRBYTE_API_KEY}"
airbyte_workspace_id = "${AIRBYTE_WORKSPACE_ID}"

Without these keys, KiwiFS falls back to the AIRBYTE_API_KEY and AIRBYTE_WORKSPACE_ID environment variables. Cloud check currently supports firebase-rtdb, notion, and airtable.

If Docker is missing and AIRBYTE_API_KEY is set in the server’s environment, REST imports of notion and airtable that include an api_key try Airbyte Cloud and fail. Add "via": "builtin" to those requests.

List available sources

Terminal

curl http://localhost:3333/api/kiwi/import/sources

Response

{
  "builtin": ["markdown", "obsidian", "csv", "postgres", "..."],
  "airbyte": ["firebase-rtdb", "notion", "airtable"],
  "docker_available": true,
  "cloud_key_present": false
}

airbyte appears only when Docker or an Airbyte Cloud key is available.

Related

Browse, preview, and sync

Saved connections and auto-sync.

Import API

REST reference.