Search

Grep, SQLite FTS5, hybrid RRF fusion, vector embeddings, and token-budgeted context packs.

KiwiFS provides three tiers of search, from zero-dependency grep to semantic vector similarity, plus a hybrid mode that fuses lexical and vector ranks. You can start simple and scale up without changing your data or workflow.

Tier 1: grep

Zero dependencies. Exact string matching. Works everywhere.

kiwifs serve --search grep

Best for small knowledge bases or environments where SQLite isn’t available.

Tier 2: SQLite FTS5 (default)

BM25-ranked full-text search powered by SQLite FTS5. Supports boolean operators, phrase matching, and path prefix filtering.

kiwifs serve --search sqlite

This is the default. KiwiFS builds and maintains a FTS5 index automatically. Searches are fast even with thousands of pages.

Query syntax

Syntax	Example	Behavior
Keywords	payment timeout	Match pages containing both terms
Boolean AND	payment AND timeout	Explicit AND
Boolean OR	payment OR billing	Match pages containing either term
Boolean NOT	payment NOT refund	Exclude pages containing a term
Phrase	"payment timeout"	Match the exact phrase

Trust-ranked search

KiwiFS also offers trust-ranked search at /api/kiwi/search/verified. Pages with status: verified or source-of-truth: true in their frontmatter are boosted in results.

Tier 3: Vector search

Semantic similarity search using pluggable embedder providers and vector stores. Enable it in .kiwi/config.toml:

[search.vector]
enabled = true

[search.vector.embedder]
provider = "openai"
model = "text-embedding-3-small"
api_key = "${OPENAI_API_KEY}"

[search.vector.store]
provider = "sqlite-vec"

Supported embedders

Provider	Notes
OpenAI	text-embedding-3-small, text-embedding-3-large
Ollama	Local models, no API key needed
Cohere	embed-english-v3.0, multilingual variants
Vertex AI	Google Cloud embeddings
Bedrock	AWS embeddings
Custom HTTP	Any endpoint returning a vector array
ONNX	Offline local model via kiwifs model download

Supported vector stores

Store	Notes
sqlite-vec	Embedded, zero config (default)
Qdrant	Self-hosted or cloud
pgvector	PostgreSQL extension
Pinecone	Cloud-hosted
Weaviate	Self-hosted or cloud
Milvus	Self-hosted or cloud

Hybrid search

When both FTS and a vector index exist, mode=hybrid fuses the two ranked lists with Reciprocal Rank Fusion. If embeddings are not configured, the same request degrades to lexical-only.

curl 'http://localhost:3333/api/kiwi/search?mode=hybrid&q=payment+timeout&limit=10'

Results include engines (fts, semantic), per-engine ranks, and the fused RRF score. Tune fusion with rrf_k (default 60) and boost.

Context packs

POST /api/kiwi/brief retrieves pages with hybrid search, extracts sections, and packs them into a token budget. It never summarizes — overflow is listed in a dropped manifest so you can see what was cut.

curl -X POST 'http://localhost:3333/api/kiwi/brief' \
  -H "Content-Type: application/json" \
  -d '{"query": "how does auth work?", "budget_tokens": 4000, "max_pages": 20}'

Use this as the first retrieval step for an agent instead of dumping whole files.

Retrieval evaluation

Score FTS, semantic, and hybrid engines against a golden set in .kiwi/eval/:

kiwifs eval --set leave-one-out --exclude-prefix drafts/ --min-hit-rate 0.6

Or POST /api/kiwi/eval with { "set": "leave-one-out" } or an inline queries array. Metrics include hit rate and nDCG.

API endpoints

Endpoint	Method	Description
/api/kiwi/search	GET	Full-text search (mode=hybrid for RRF)
/api/kiwi/search/verified	GET	Trust-ranked search
/api/kiwi/search/semantic	GET, POST	Vector similarity search
/api/kiwi/brief	POST	Token-budgeted context pack
/api/kiwi/eval	POST	Retrieval quality benchmark
/api/kiwi/similar	GET	Frontmatter similarity profiles
/api/kiwi/meta	GET	Frontmatter metadata query

MCP tools

Tool	Description
kiwi_search	Full-text search with pagination
kiwi_search_hybrid	RRF fusion of FTS and semantic
kiwi_search_semantic	Vector similarity search
kiwi_brief	Token-budgeted context pack
kiwi_similar	Frontmatter similarity profiles
kiwi_eval	Search quality benchmark
kiwi_query_meta	Frontmatter metadata query
kiwi_query	DQL queries over metadata

See the Search API reference for detailed endpoint documentation.