Ollama vector search on a small VPS

Run semantic search with a local Ollama embedder and the built-in SQLite vector store, tuned for CPU-only machines.

This setup gives you semantic search with no external embedding API and no extra database. Ollama computes embeddings on your own machine, and KiwiFS stores the vectors in SQLite next to its other indexes. On a small CPU-only VPS, Ollama is slower than a hosted API, so you lower the indexing concurrency and raise the request timeout.

Prerequisites

• An Ollama server that the KiwiFS process can reach. KiwiFS calls its /api/embed endpoint.

• An embedding model pulled into Ollama:

  Terminal

  ollama pull nomic-embed-text
  

Configure KiwiFS

1. Edit the config

   .kiwi/config.toml

   [search]
   engine = "sqlite"
   
   [search.vector]
   enabled = true
   worker_count = 1
   
   [search.vector.embedder]
   provider = "ollama"
   base_url = "http://127.0.0.1:11434"
   model = "nomic-embed-text"
   timeout = "120s"
   
   [search.vector.store]
   provider = "sqlite"
   
2. Start the server

   Terminal

   kiwifs serve --root ./knowledge
   

   When the vector store is empty, the server embeds every existing page in the background on startup. After that, new and edited pages are embedded as they’re written.

3. Rebuild on demand (optional)

   To embed everything ahead of time, or to start over, stop the server and run:

   Terminal

   kiwifs reindex --root ./knowledge --vector
   

   --vector makes the command fail fast if vector search isn’t configured.

Settings

Key	Default	Description
[search.vector] worker_count	5	Parallel embed and upsert workers
[search.vector.embedder] base_url	http://localhost:11434	Ollama URL
[search.vector.embedder] model	nomic-embed-text	Embedding model
[search.vector.embedder] timeout	30s	HTTP timeout per Ollama request, as a Go duration such as 120s or 3m
[search.vector.embedder] dimensions	Detected	Vector size. KiwiFS detects it from the first response.
[search.vector.store] provider	sqlite	sqlite (alias sqlite-vec) stores vectors in .kiwi/state/vectors.db

The SQLite store needs no API key and no separate service. Like every index in .kiwi/state/, it’s derivative data that kiwifs reindex rebuilds from your markdown files.

Choose the Ollama URL

base_url depends on where KiwiFS and Ollama run.

Same host

.kiwi/config.toml

[search.vector.embedder]
provider = "ollama"
base_url = "http://127.0.0.1:11434"

KiwiFS in Docker, Ollama on the host

On Linux, map host.docker.internal to the host gateway.

docker-compose.yml

services:
  kiwifs:
    image: ameliaanhlam/kiwifs:latest
    extra_hosts:
      - "host.docker.internal:host-gateway"

.kiwi/config.toml

[search.vector.embedder]
provider = "ollama"
base_url = "http://host.docker.internal:11434"

Ollama listens on 127.0.0.1 by default. Start it with OLLAMA_HOST=0.0.0.0 so the container can reach it.

Both in one Compose project

Use the Ollama service name as the host name.

docker-compose.yml

services:
  ollama:
    image: ollama/ollama
    volumes:
      - ollama:/root/.ollama
  kiwifs:
    image: ameliaanhlam/kiwifs:latest
    volumes:
      - ./knowledge:/data
    depends_on:
      - ollama
volumes:
  ollama:

.kiwi/config.toml

[search.vector.embedder]
provider = "ollama"
base_url = "http://ollama:11434"

Pull the model inside the Ollama container: docker compose exec ollama ollama pull nomic-embed-text.

Tune for your machine

Symptom	Change
Ollama requests time out during reindexing	Raise timeout, for example from 120s to 3m
CPU stays saturated or the VPS becomes unresponsive	Set worker_count = 1
Reindexing is stable but slow	Try worker_count = 2

Keep worker_count = 1 for the first reindex of a large knowledge base, and raise it later only if CPU, memory, and Ollama latency stay stable.

Test semantic search

Terminal

curl "http://localhost:3333/api/kiwi/search/semantic?q=how+do+we+rotate+keys"

See Search for semantic and hybrid query options.

Troubleshooting

KiwiFS can't reach Ollama

Check the URL from the machine or container that runs KiwiFS. The KiwiFS image has BusyBox wget but not curl.

Terminal

curl http://127.0.0.1:11434/api/tags
docker exec kiwifs wget -qO- http://host.docker.internal:11434/api/tags

The model is missing

Terminal

ollama list
ollama pull nomic-embed-text

The first reindex failed partway

Keep the same config, raise timeout, and run kiwifs reindex --root ./knowledge again. Vectors are rebuilt from the markdown files, so nothing is lost.

A full vector rebuild, from kiwifs reindex or the server’s startup backfill, stops after 10 minutes. If a large knowledge base can’t finish in that time, raise worker_count as far as your machine allows or use a smaller embedding model.

Results look wrong after changing the model

Vectors from different models aren’t comparable. After changing model, run kiwifs reindex, which clears the vector store before embedding every page again.

Related

Search

Full-text, semantic, and hybrid search.

Docker

Run KiwiFS in a container.