Knowledge Capture

Provenance-weighted knowledge cache attributing answers to the humans who gave them, with a 71% cosine similarity quality standard and quarterly promotion gates.

What is it?

Knowledge Capture (COS-004) is Warburton's system for preserving and serving community knowledge. Every useful thing said in your community is captured, weighted by provenance, and retained. When someone asks a question that was answered six months ago, the answer is still there — with the name of the person who gave it.

The system uses a provenance-weighted cache, a defined quality standard, quarterly promotion gates, and AI answer seeding for unanswered questions.

Why does it exist?

Communities forget. The same questions get asked every month. The expert who answered them last time has moved on, or is tired of repeating themselves. The knowledge was there — it was said, in the chat, by a real person — but nobody captured it, nobody attributed it, and nobody can find it now.

Knowledge Capture exists because the collective knowledge of a community is one of its most valuable assets, and in most communities it disappears into a chat history that nobody reads.

Why is it different?

Most knowledge base systems are either manual (someone has to write documentation) or AI-generated (a model summarises conversations without attribution). Knowledge Capture is neither.

It captures knowledge as it's created — in real conversations, by real people — and attributes it to the human who gave the answer. The provenance weighting means answers from consistently helpful members carry more weight. The 71% cosine similarity quality standard ensures that only high-quality knowledge is served. And when AI does generate an answer, it's clearly marked as AI-sourced until a human confirms it.

How does it work?

Five Knowledge Domains

Knowledge Capture manages five distinct retrieval domains, each with its own storage, indexing, and provenance rules:

Learned community answers: Human-provided answers captured from community conversations, attributed to the contributor, and cached with provenance weights. Entries carry a 90-day TTL and a 500-entry FIFO cap — stale knowledge expires automatically.
Substack diary retrieval: Content from the community's Substack publication, fetched, summarised, and stored with source attribution.
Specialist lore retrieval: Domain-specific knowledge (e.g. Warhammer lore in gaming communities), summarised via local models and stored with source provenance.
Wikipedia topic search: On-demand Wikipedia retrieval for factual context, with SSRF protection and allowlist-only fetch.
Reddit topic search: Community-relevant Reddit content, fetched through the same safe-fetch pipeline.

Three Retrieval Tiers

Retrieval follows a tiered model designed to minimise cost while maximising quality: the system first checks the local cache, then queries a local model (Ollama), and only escalates to Claude when the local model cannot produce a satisfactory answer. Each tier's decision is recorded in the decision provenance trail.

Provenance and Quality

Provenance-weighted cache: Answers are attributed to the humans who gave them. Each answer carries a provenance weight based on the contributor's history of accurate, helpful responses.
71% cosine similarity standard: Only answers meeting the 71% threshold are served. Below that, the knowledge is flagged for review rather than served, preventing tangentially related answers from reaching members.
Quarterly promotion gates: Knowledge progresses from cache to canon through quarterly promotion gates. Cached knowledge must demonstrate sustained accuracy — confirmed by community use, not just by initial capture.
Opus answer seeding: When no human answer exists, Claude generates a candidate answer clearly marked as AI-sourced. This answer remains flagged until a human confirms or corrects it. The system never presents AI-generated answers as human knowledge.

Retrieval Security

Every retrieval path is guarded against prompt injection. Write-time guards scan content before storage; read-time guards re-scan before injection into the LLM context (defence-in-depth). A canonical injection detection function covers eight patterns — from "ignore all previous instructions" to financial advice claims — applied consistently across all five knowledge domains. Poisoned content is rejected at write time and marked ineligible for retrieval.

Example

Scenario: A member asks how to configure a specific game setting — the same question asked three times in the past six months. Knowledge Capture retrieves the original answer from the provenance-weighted cache, attributed to the community expert who first provided it, with a cosine similarity score above the 71% threshold. The expert does not need to repeat themselves. The new member gets an accurate, attributed answer. The expert's contribution is recognised in the Contributor Recognition system.

Why should anyone care?

The people who answer questions in your community are doing work. Knowledge Capture ensures that work persists — attributed to them, quality-checked, and available to everyone who asks the same question in the future. It respects the humans who contribute knowledge and ensures that the community's collective intelligence outlasts any individual member's participation.