A self documenting wiki where the difference is the gold

If you follow newsletters, blogs and YouTube channels to get better at your work, you know the moment: two people you trust give opposite advice on the same job, and you are left to work out which one is right. An AI can now read everything you follow and keep a tidy page on each topic for you. This article is about what that tidy page drops - the disagreements - and why they are where most of the useful knowledge sits.

The tidy page has a familiar model. On Wikipedia, each topic gets one page, many people write it and its editors keep it updated. When the sources disagree, Wikipedia’s own rule is to report the dispute and stay out of it: it “aims to describe disputes, but not engage in them.” That suits an encyclopedia written for millions of readers. Your wiki has one reader, you, and you have to decide what to do.

Andrej Karpathy published a write-up on what he calls the LLM Wiki: the same idea, applied to your own collection of knowledge such as newsletter subscriptions or YouTube channels that you follow. An AI agent builds and maintains a persistent wiki over everything you collect, so knowledge is compiled once and kept current instead of re-derived every time you ask. If you run something like it, this takes the premise and moves it along. If you haven’t read it yet, it is ten minutes well spent, but that one line is enough to follow along so don’t go anywhere just yet.

Follow his write-up and you’ll get a well put together summary of the articles you hold on any given topic. The destination is encyclopedic by nature, like Wikipedia (as intended): tidy topic pages, one calm voice, facts kept current. A disagreement between your sources gets, at best, a neutral paragraph that reports both sides, and at worst a smooth one that blends them. Either way it reads beautifully, and nothing on the page tells you which way to go.

My version treats those disagreements as tensions, held on a second page. Here is how the pieces fit, and the names I use for them from here on:

• Sources: the files you collect - articles, newsletters, transcripts, notes. The AI reads them; nothing edits them.
• Topic: what a group of sources is about - conversion, paid search, AI agents.
• Main page: one per topic, written by the AI from every source on that topic. It says what the sources collectively say. This is the page Karpathy’s wiki already gives you.
• Companion page: written by the AI from the same sources and kept beside the main page. It holds what the sources only show when they are read side by side: where they disagree, where they say the same thing in different words, how much weight their evidence can carry, and my rulings.

A main page and its companion page together make a pair.

The gold is the difference

Between sources that refuse to agree, between what a page claims and what its evidence can carry, and between the record of what everyone says and your own ruling on what to do.

So take the LLM Wiki as written - the persistent wiki, the model doing the maintenance, you directing. Here are the four steps I use to extend it.

Step 1: Give the tension its own page

Karpathy’s write-up spends two lines on contradiction. The agent should note “where new data contradicts old claims”, and a periodic health check should look for “contradictions between pages”. For facts, that is the right treatment: a price moved, a tool died, one value is wrong. Fix it and move on.

Methods break that logic. When two respected practitioners disagree about how to do something, neither value is wrong. That contradiction is a tension: both positions can be valid in different contexts, and the deciding context is often the most valuable thing your collection holds. An agent running the fix-it instruction will repair the tension away, and with it the nuance you built the wiki to keep.

So the first step is a routing rule, written into the agent’s standing instructions - the schema file Karpathy himself credits with making the agent “a disciplined wiki maintainer”. Mine says: a fact contradiction gets fixed; a method contradiction gets space of its own.

That space is the companion page. A topic whose sources really disagree gets one (Step 3 covers how a topic earns it), and every tension on it is cross-referenced, with both sides intact. The main page stays readable, and the disagreement stays honest, in full, next door.

The companion page has a discipline of its own. Every entry holds both positions verbatim, with citations, and then must land in exactly one of three states:

• settled,
• reconciled by a named condition, or
• left open with the reason recorded

“It depends” is banned as a resting place, and sooner or later that forcing produces a sentence none of your sources wrote: the condition under which both sides are right.

My companion page for conversion (turning visitors into enquiries and sales) holds the cleanest example. One respected school says a landing page keeps the full site navigation, because stripping it damages trust. Another treats removing navigation as a standard structural test and quotes conversion gains for it. Avinash Kaushik has argued the keep side; Sam Tomlinson works the strip side. Both quote their evidence correctly. Read together in one pass, the three-state rule forced the entry to land somewhere, and it landed on a sentence neither of them wrote:

The implied deciding variable is page type and traffic source … written for homepages and full landing-pages that double as credibility scaffolding … while the strip-it guidance is written for single-purpose paid-traffic landing-pages where every nav link is an exit. No source states this condition explicitly, so a reviewer scoring the same page against the two rules reaches opposite verdicts.

Reading everything on a topic in one pass keeps finding smaller versions of this. Loss aversion turned up in my pages wearing four costumes: a ‘guarantee’ rule, a ‘risk-reversal’ offer, ‘urgency’ copy, and a ‘friction’ principle - and the companion page now holds the principle named once, with all four vocabularies pointing at it. That is knowledge the collection contained but could not surface - it existed in no single source and answered to no single search term until a page wrote it down.

Step 2: The page keeps listening

The companion page has to keep listening. The AI checks every new source that lands against the held tensions: a new voice can settle an open one, or reopen one that looked settled, and the entry’s state moves with it. Twenty-two topics in my collection currently carry the pair.

The navigation tension is why the listening matters. This year a third position arrived - Kaushik again - arguing that the disagreement is about the wrong object. Navigation and site search are both find-it furniture: they assume a visitor who already knows what they want. His question - “why still persist with the most prominent spot given to an inferior technology?” - is aimed at the search box holding a site’s prime spot, when the visitor who arrives with a real question is better served by a surface that answers it. The companion page now holds it as a third side, dated, with the first two intact. Karpathy’s wiki would have caught the arrival too - filed as one more fact, folded into the calm paragraph. What a settled page cannot do is mark a new voice as a new side of an old disagreement, so the moment the tension moved would sit in plain text, waiting for a reader who already knew to look for it.

Step 3: A page that knows where it is thin

Beyond holding tensions, the companion page grades itself on how much I should trust its insights. My pages on the topic of ‘conversion’ carry a section on where their knowledge sources are weak:

1. One prolific newsletter is thin against its volume: “Five issues carry load”, and the rest of its output is written to win subscribers, PR, or a sales pitch. (I stopped using the newsletter as a source for this topic.)
2. Two outside voices carry nearly all the evidence, which indicates a lack of diversity; remove their pieces and the outside sources add little my own curated notes don’t already say.

That is thin coverage that knows it is thin, and says so on the page.

Grading also means refusing. One real disagreement in that collection never made it onto the companion page as a named tension, because one side rests on a vendor’s account inside a sponsored issue and the other on a single practitioner’s assertion. Declining to promote weak evidence is grading too: a near-entry left off the list of tensions, with the reason recorded in the page’s notes on its own gaps.

And the grading runs before a page exists at all.

When a topic earns its pair

A topic earns its pair only when the collection disagrees with itself enough to hold a difference, at a scope one page can carry.

When I graded my own candidates, I demoted one after a two-minute check showed about 80 percent of everything I held on it came from a single author - one voice is no difference at all, however much of it you have. Another is parked as too broad, because it spans several real topics one page would flatten. The routing has four exits: build now, first check that enough independent sources disagree, demote to a narrower page, or wait. “Wait” is a legitimate answer.

Step 4: Your ruling sits on top

A companion page that leaves a tension open leaves you a problem: you still have to act. The last step is a ruling layer. Where the record stays open, or where I disagree with where it landed, a house ruling sits on top of the entry, dated and named as mine, with everything either source said left intact underneath. The navigation question is the second kind. The condition was a fair reading of the sources, but it doesn’t match how I build. Navigation is how a visitor checks they are dealing with a real company, and I don’t think a paid landing page is the exception. So my ruling is that every page keeps its navigation, single-purpose paid landing pages included. The two schools stay underneath it in full, as the record of what the sources say.

The page records what the sources say; the ruling decides what I do. Neither overwrites the other. When the evidence moves, I re-read the tension as it actually stood and re-rule, with nothing overwritten in the meantime.

Karpathy’s write-up has a line that is easy to skim past: good answers “can be filed back into the wiki as new pages”, so your explorations compound. A ruling is exactly that - an answer you file back. Starting costs three edits, all of them inside step one:

• a routing rule in your schema file that sends method contradictions to a companion page instead of a fix
• one pair for the topic where your sources disagree most
• the three-state rule, so every tension you hold must end settled, reconciled by a named condition, or open for a reason you wrote down

The gold

When you read the main page and companion page for a topic, watch for the sentence that appears in nobody’s source. That sentence is the gold.