My homelab has grown a knowledge base over time: sessions, lessons learned, backlog items, facts about how each piece of infrastructure is wired together. Hundreds of entries, all searchable, except “searchable” was doing a lot of quiet lying. The search was pure keyword matching: type the exact word that was in the original text, get a result; paraphrase it even slightly, get nothing.

That’s a real problem when the whole point of the knowledge base is to stop me, or an AI assistant working alongside me, from repeating a mistake I’d already solved once. If the fix is recorded under one phrasing and I go looking with a different one, the knowledge base might as well not exist.

Why not just throw it at a bigger model

The obvious lazy fix is to hand the whole knowledge base to a large language model and ask it to find the relevant bit. That works, right up until the knowledge base is too big to fit in a single prompt, or you’re doing it often enough that the cost and latency start to matter. What I actually needed was something narrower and much cheaper: a way to turn every chunk of text into a vector that captures its meaning, store those vectors, and then find the nearest neighbours to a new query’s vector. Classic semantic search, not a job for a frontier model at all.

Keeping it deliberately small

I built the vector layer using ChromaDB for storage and a compact sentence-embedding model: all-MiniLM-L6-v2, run through ONNX rather than PyTorch. That distinction mattered more than I expected. PyTorch on a homelab VM with no GPU is a heavy, fiddly dependency for something that just needs to produce a 384-dimension vector from a sentence. ONNX runtime does the same job with a fraction of the footprint, no GPU requirement, and none of PyTorch’s version-matching headaches.

Getting there wasn’t entirely friction-free. The VM this runs on had its CPU type set to a generic baseline profile, which turned out to silently disable a modern instruction set that the numpy build underneath the embedding model expected. The symptom wasn’t an obvious crash. It was numpy quietly failing in a way that pointed everywhere except the actual cause. The permanent fix was switching the VM’s CPU type to pass through the host’s real instruction set, at the cost of one reboot, rather than patching around a mismatch that would just resurface with the next dependency update.

What it actually indexes

Every category of entry in the knowledge base gets embedded: session write-ups, hard-won lessons, backlog items, standalone facts, and a reference table of infrastructure details. A little over 370 chunks in total once everything was indexed, each one now findable by meaning rather than exact wording. Ask it about “the outage where a config file wasn’t the one being read” and it doesn’t need me to remember whether I originally wrote “stale,” “outdated,” or “wrong file.” The vectors cluster on meaning, not spelling.

The gap I didn’t catch until later

The first version had one real flaw, and it wasn’t in the search itself. It was in the timing. New entries only got embedded when a work session formally closed. That’s fine when every session ends cleanly. It’s not fine when a connection drops mid-session, which happened a few days later: the new knowledge had been written to the database correctly, but it stayed invisible to semantic search because nothing had triggered the embedding step.

The fix was to stop relying on a single trigger point. Now, meaningful writes get embedded immediately rather than waiting for a clean close, and there’s an independent scheduled job re-checking for anything that slipped through every twenty minutes regardless of how a session ended. One trigger is a single point of failure no matter how reliable it seems; two independent ones, checking on different schedules, aren’t.

The actual payoff

None of this is exotic machine learning. It’s a genuinely small, self-hosted, CPU-only system doing one job well: making a knowledge base searchable by what you meant, not just what you typed. For a homelab that’s meant to catch its own recurring mistakes, that difference is the whole point.