A self documenting wiki where the difference is the gold

By Mark Brownlie ·

Andrej Karpathy published a write-up on what he calls the LLM Wiki: an LLM agent builds and maintains a persistent wiki over everything you collect, so knowledge is compiled once and kept current instead of re-derived every time you ask. If you run something like it, this takes the premise and moves it along. If you haven’t read it yet, it is ten minutes well spent, but that one line is enough to follow along so don’t go anywhere just yet.

Follow the gist and you’ll get a well put together summary of the articles you hold on any given subject. The destination is encyclopedic by nature (as intended): tidy topic pages, one calm voice, facts kept current. A disagreement between your sources has nowhere to live in a page like that, so it gets flattened into a neat paragraph and it reads beautifully.

My version treats those disagreements as tensions: held and worked in a companion wiki page beside the main one.

The gold is the difference

Between sources that refuse to agree, between what a page claims and what its evidence can carry, and between the record of what everyone says and your own ruling on what to do.

So take the LLM Wiki as written - the persistent wiki, the model doing the maintenance, you directing. Here are the four steps I use to extend it.

Step 1: Give the tension its own page

Karpathy’s write-up spends two lines on contradiction. The agent should note “where new data contradicts old claims”, and a periodic health check should look for “contradictions between pages”. For facts, that is the right treatment: a price moved, a tool died, one value is wrong. Fix it and move on.

Methods break that logic. When two respected practitioners disagree about how to do something, neither value is wrong. That contradiction is a tension: both positions can be valid in different contexts, and the deciding context is often the most valuable thing your collection holds. An agent running the fix-it instruction will repair the tension away, and with it the nuance you built the wiki to keep.

So the first step is a routing rule, written into the agent’s standing instructions - the schema file Karpathy himself credits with making the agent “a disciplined wiki maintainer”. Mine says: a fact contradiction gets fixed; a method contradiction gets space of its own.

That space is a second wiki page. Any subject that holds real disagreement gets a pair: the main write-up states what the sources collectively say, and a companion wiki page beside it holds every tension, cross-referenced, with the contradictions intact and the new evidence each one is forced to provide. The write-up stays readable, and the disagreement stays honest, in full, next door.

One subject, two wiki pages, flowing top to bottom. Everything held on the subject is read in one pass and hits the routing rule in the agent's standing instructions: a fact contradiction goes to the main write-up as a fix, a method contradiction goes to the companion wiki page as a held tension with both positions verbatim. Each tension must land in one of three states - settled, reconciled by a named condition, or left open with the reason recorded - and where it stays open, your dated ruling sits on top with the record intact underneath. A held tension sooner or later yields a sentence none of your sources wrote: that sentence is the gold.

The companion has a discipline of its own. Every entry holds both positions verbatim, with citations, and then must land in exactly one of three states:

  • settled,
  • reconciled by a named condition, or
  • left open with the reason recorded

“It depends” is banned as a resting place, and sooner or later that forcing produces a sentence none of your sources wrote: the condition under which both sides are right.

My conversion companion holds the cleanest example. One respected school says a landing page keeps the full site navigation, because stripping it damages trust. Another treats removing navigation as a standard structural test and quotes lifts for it. Avinash Kaushik has argued the keep side; Sam Tomlinson works the strip side. Both quote their evidence correctly. Read together in one pass, the three-state rule forced the entry to land somewhere, and it landed on a sentence neither of them wrote:

The implied deciding variable is page type and traffic source … written for homepages and full landing-pages that double as credibility scaffolding … while the strip-it guidance is written for single-purpose paid-traffic landing-pages where every nav link is an exit. No source states this condition explicitly, so a reviewer scoring the same page against the two rules reaches opposite verdicts.

Reading everything on a subject in one pass keeps finding smaller versions of this. Loss aversion turned up in my pages wearing four costumes: a guarantee rule, a risk-reversal move, urgency copy, and a friction principle and the companion now holds the principle named once, with all four vocabularies pointing at it. That is knowledge the collection contained but could not surface - it existed in no single source and answered to no single search term until a page wrote it down.

Step 2: The page keeps listening

None of this surfaces if pages are assembled from search results at question time. The pair comes from a full read: everything the collection holds on a subject, read in one pass, before a word is written. Buried positions and cross-source clashes only show up when the sources actually sit side by side.

The companion has to keep listening. Every new source that lands gets checked against the held tensions: a new voice can settle an open one, or reopen one that looked settled, and the entry’s state moves with it. Twenty-one subjects in my collection currently carry the pair.

The navigation tension is why the listening matters. This year a third position arrived - Kaushik again - arguing that the disagreement is about the wrong object. Navigation and site search are both find-it furniture: they assume a visitor who already knows what they want. His question - “why still persist with the most prominent spot given to an inferior technology?” - is aimed at the search box holding a site’s prime spot, when the visitor who arrives with a real question is better served by a surface that answers it. The companion now holds it as a third side, dated, with the first two intact. Karpathy’s wiki would have caught the arrival too - filed as one more fact, folded into the calm paragraph. What a settled page cannot do is mark a new voice as a new side of an old disagreement, so the moment the tension moved would sit in plain text, waiting for a reader who already knew to look for it.

Step 3: A page that knows where it is thin

The second thing the pair buys is a page that grades the evidence it used to build itself. My conversion companion carries a section on where its own knowledge base is weak: “Of the fourteen issues now in scope, three carry any load” - the rest of that prolific source’s output is acquisition content, PR, or pitch. Two outside voices carry nearly all the external weight; remove six documents and the outside layer adds little my own curated notes don’t already say. That is thin coverage that knows it is thin, and says so on the page.

Grading also means refusing. One real disagreement in that collection never made it into the companion as a named tension, because one side rests on a vendor’s account inside a sponsored issue and the other on a single practitioner’s assertion. Declining to promote weak evidence is grading too: a near-entry filed as nothing, with the reason recorded.

And the grading runs before a page exists at all.

When a subject earns its pair

A subject earns its pair only when the collection disagrees with itself enough to hold a difference, at a scope one page can carry.

When I graded my own candidates, I demoted one after a two-minute check showed about 80 percent of everything I held on it came from a single author - one voice is no difference at all, however much of it you have. Another is parked as too broad, because it spans several real subjects one page would flatten. The routing has four exits: build now, confirm density first, demote to a narrower page, or wait. “Wait” is a legitimate answer.

Step 4: Your ruling sits on top

A companion that leaves a tension open leaves you a problem: you still have to act. The last step is a ruling layer. Where the record stays open, a house ruling sits on top of the entry, dated and named as mine, with everything either source said left intact underneath. On the navigation question my ruling is that every page keeps its navigation, single-purpose paid landing-pages included. The two schools stay on the page in full, as the record.

The page records what the sources say; the ruling decides what I do. Neither overwrites the other. When the evidence moves, I re-read the tension as it actually stood and re-rule, with nothing overwritten in the meantime.

Karpathy’s write-up has a line most readers skim: good answers “can be filed back into the wiki as new pages”, so your explorations compound. A ruling is exactly that - an answer you file back. Starting costs three edits, all of them inside step one:

  • a routing rule in your schema file that sends method contradictions to a companion wiki page instead of a fix
  • one companion pair for the subject where your sources disagree most
  • the three-state rule, so every tension you hold must end settled, reconciled by a named condition, or open for a reason you wrote down

The gold

Run a full review on a subject and watch for the sentence that appears in nobody’s source. That sentence is the gold.