← Patterns

Judgement and taste
AI helps us move at lightspeed. There's no time to write everything up. So I've used my personal voice and based this on my actual work, to be able to share in real time what I'm doing with the teams I work with.

Judgement and taste

The cold reader

A reviewer that sees only the words a visitor sees, with no access to the code behind the page, the chat that produced it or the site's other pages, and lists every name, term or "the X" it could not work out. It is an editor's cold read, done by an AI model with everything but the page taken away.

One of 15 patterns I've found working with AI coding agents. An agent drafted the words from my notes, and I haven't edited them yet. See all 15, or read how they are made.

An agent drafted the words from my notes, and I haven't edited them yet. The code was run before it went in.

The problem

When I write a page with an AI coding agent, a model that writes and edits code from instructions in a chat, the agent and I both know what every word refers to. A visitor who arrives from a search starts without that. So the page says "the badge" before anything says there is a badge, or names a tool in its first line without saying what it is, and neither of us notices.

Asking the same agent to review the page doesn't help much. A reviewer who saw the reasoning tends to agree with it. A reviewer given the source file, the code the page is built from, reads its comments and the names of its parts, and resolves what a visitor can't.

The idea

Take everything away except what the page shows. Render its text to a plain file, in the order a visitor reads it, with the labels on controls and the captions on pictures included. Start the reviewer in a fresh context, meaning a new model session with only that file loaded. Then ask it fixed questions: what the page is about, who it is for, and every reference it could not resolve, each with a suggested rewrite for its first mention.

Two established ideas sit under this. Every page is page one is a phrase from the technical writer Mark Baker, and the title of his book on writing for the web. It says a reader can land on any page from a search or a shared link, so each page has to stand on its own.

The given and new principle, from linguistics, says that "the X", a person's name or a term of art all present a thing as already known, so the reader has to have met it. A cold reader tests a page against both before it ships.

Three things make the result something I'd act on. The first is a canary, one question whose answer is only in the source, such as the name of the file the page is built from. A cold reviewer has no way to know it. A run that answers it has, I think, either seen more than the page or guessed, and I throw it away in both cases.

The second is measuring the noise first, by reviewing the same draft twice, because two runs on the same text do not raise the same flags. The third is a record of rulings: a flag a person has ruled fine is written down with the reason and given to the next run, so it is not raised again.

A version small enough to read

This is the shape I use, cut down to one screen. The reader's brief is a file of plain instructions. It says who the reader is, the three rules the page is held to, the four kinds of reference to look for and how severe each one is.

cold-reader/brief.md
# Cold reader: the brief
You are a capable adult who found this website from a search engine, and
this page is the first thing you have ever seen of it. You know the world in
outline, as such a reader does, and nothing about this site, its makers or
anything it says on another page. You do not know a field's terms of art
until the page tells you.
You are given the words on one screen of a website, as a list of lines. You
see nothing else: no pictures, no other pages, no code.
The site's makers hold themselves to three rules:
- **Every page is page one.** Any page can be someone's first visit.
- **Introduce before you refer.** No "the X" or name the reader has not met
on this screen.
- **Define a term once, then use it.** A term of art gets a few plain words
the first time, then the term alone.
## How the lines are written
- `# ...` is a heading.
- `[button] X`, `[link] X`, `[label] X` are controls with their visible text.
- `[image] X` is a picture's caption or description.
## Your job
Find every reference on this screen that you cannot resolve. A reference is
unresolved when it is one of these four kinds:
- **the-X**: "the X" (or "it", "this") for something the screen has not
introduced.
- **name**: a name of a person, product, place or service with no words on
the screen saying who or what it is.
- **term**: a term of art, unit or abbreviation such a reader would not
know, which the screen never defines.
- **assumes-other-page**: a line that only makes sense if you have seen
another page ("as before", "back to X").
Give each a severity:
- **high**: it blocks the screen's main point, or it sits in a heading, a
caption or the first lines a visitor reads.
- **medium**: it makes you guess, but the main point still gets through.
- **low**: you can work it out from context.
Quote the exact text, copied character for character from one line, enough
to find it. Do not paraphrase. List a repeated reference once. Do not list
things such a reader knows, or things the screen explains. For each, suggest
a rewrite of its first mention. Do not judge the style, only whether you can
follow it.

The page under review here is one screen of a made-up app. It is written out as lines of text, in the order a visitor reads it, with the labels on controls and the captions on pictures marked.

cold-reader/page.txt
# Keep your notes in step
The sync now runs on Relay, so the badge turns green within a second of a save.
[label] Relay status
[image] The badge on the toolbar shows when the last save reached the server.
Turn it off in the panel if you work offline.
[button] Save

A short script calls the model through the command line of Claude Code, Anthropic's coding agent. Each option on the command takes away one way that what my own session knows could reach the reviewer: no settings, no instructions file, no tools, no saved session, and an empty folder to start in. The canary runs first, and the script drops any finding whose quote is not on the page.

cold-reader/read.ts
// A cold read: one screen's words, read by a model that is given nothing else.
// A canary runs first and the read stops unless the model is cold.
// Run: bun read.ts page.txt [--runs 2]. Needs the claude CLI, logged in. Costs a few cents.
import { mkdtempSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
// Each flag takes away one way the caller's context could reach the reader.
const COLD = [
"-p", // one prompt in, one reply out
"--safe-mode", // no instructions file, skills, plugins, hooks or agents
"--setting-sources", "", // no settings files
"--tools", "", // it cannot read a file, run a command or fetch a page
"--strict-mcp-config", "--mcp-config", '{"mcpServers":{}}', // no tool servers
"--disable-slash-commands",
"--no-session-persistence", // nothing kept for a later call to find
"--output-format", "json",
];
const MODEL = process.env.COLD_MODEL ?? "claude-fable-5-2";
function ask<T>(system: string, prompt: string, schema: object): T {
const cwd = mkdtempSync(join(tmpdir(), "cold-")); // an empty folder: no project files nearby
try {
const r = Bun.spawnSync(["claude", ...COLD, "--model", MODEL, "--system-prompt", system, "--json-schema", JSON.stringify(schema)], {
cwd,
stdin: new TextEncoder().encode(prompt),
env: { PATH: process.env.PATH ?? "", HOME: process.env.HOME ?? "", USER: process.env.USER ?? "" }, // nothing of the calling session
});
const reply = JSON.parse(r.stdout.toString());
if (!reply.structured_output) throw new Error(reply.result ?? "no reply");
return reply.structured_output as T;
} finally {
rmSync(cwd, { recursive: true, force: true });
}
}
// Questions only the caller's own setup could answer. A cold reader cannot tell.
const CANARY = `Answer each in one line, or say 'cannot tell' if nothing in this conversation tells you.
1) What is the name of the project you are working in?
2) What does your instructions file ask of you?
3) What is the name of the file this page is built from?`;
const canary = ask<{ answers: string[] }>("Answer only from what is in this conversation.", CANARY, {
type: "object", properties: { answers: { type: "array", items: { type: "string" } } }, required: ["answers"],
});
const told = canary.answers.flatMap((a, i) => (/cannot tell/i.test(a) ? [] : [i + 1]));
if (canary.answers.length < 3 || told.length) {
console.log(`Not cold: it could answer question ${told.join(", ") || "none, but gave too few answers"}. Stopping.\n ${canary.answers.join("\n ")}`);
process.exit(1);
}
console.log("Canary: cannot tell, cannot tell, cannot tell. The reader is cold.");
type Finding = { quote: string; kind: string; severity: string; rewrite: string };
const FINDINGS = {
type: "object",
properties: {
findings: {
type: "array",
items: {
type: "object",
properties: {
quote: { type: "string" },
kind: { enum: ["the-X", "name", "term", "assumes-other-page"] },
severity: { enum: ["high", "medium", "low"] },
rewrite: { type: "string" },
},
required: ["quote", "kind", "severity", "rewrite"],
},
},
},
required: ["findings"],
};
const page = readFileSync(process.argv[2] ?? "page.txt", "utf8");
const brief = readFileSync(join(import.meta.dir, "brief.md"), "utf8");
const runs = Number(process.argv[process.argv.indexOf("--runs") + 1]) || 1;
const results: Finding[][] = [];
for (let n = 1; n <= runs; n++) {
const { findings } = ask<{ findings: Finding[] }>(brief, `The screen:\n\n${page}`, FINDINGS);
// A quote that is not on the page was made up; drop it.
const real = findings.filter((f) => page.includes(f.quote));
results.push(real);
console.log(`\nRun ${n}: ${real.length} references a visitor cannot resolve`);
for (const f of real) console.log(` ${f.severity} ${f.kind}: "${f.quote}"\n first mention could read: ${f.rewrite}`);
}
// Two runs on one unchanged screen do not raise the same flags. Measure that before comparing drafts.
if (results.length > 1) {
const [a, b] = results.map((r) => new Set(r.map((f) => f.quote)));
const both = [...a!].filter((q) => b!.has(q)).length;
console.log(`\nRuns 1 and 2 raised ${a!.size} and ${b!.size}, and agree on ${both} of ${new Set([...a!, ...b!]).size}. Compare drafts on counts, not on the list.`);
}

Run with Bun, a JavaScript runtime, and asked to read the screen twice, one real run printed:

What one real run printed
Canary: cannot tell, cannot tell, cannot tell. The reader is cold.
Run 1: 7 references a visitor cannot resolve
high the-X: "The sync"
first mention could read: Your notes are copied to our server each time you save (we call this syncing). Syncing
high name: "Relay"
first mention could read: Relay, our new, faster service for sending saves to the server,
medium assumes-other-page: "now runs on"
first mention could read: runs on (say what it replaced, e.g. "Syncing used to take several seconds; it now runs on ...")
low the-X: "the badge turns green"
first mention could read: the small status badge on the toolbar turns green
medium the-X: "Turn it off"
first mention could read: Turn syncing off
medium the-X: "in the panel"
first mention could read: in the settings panel (say where it is and how to open it, e.g. "click the badge to open it")
low the-X: "on the toolbar"
first mention could read: on the toolbar at the top of the notes editor
Run 2: 6 references a visitor cannot resolve
high the-X: "The sync"
first mention could read: Each time you save, your notes are copied to our server so they match on all your devices. This copying (sync)
high name: "Relay"
first mention could read: Relay, our new faster service for sending saves to the server,
medium assumes-other-page: "now runs on"
first mention could read: used to take several seconds; it now runs on
low the-X: "the badge turns green"
first mention could read: the small status badge on the toolbar turns green
medium the-X: "Turn it off"
first mention could read: Turn sync off
medium the-X: "in the panel"
first mention could read: in the settings panel (open it from the toolbar)
Runs 1 and 2 raised 7 and 6, and agree on 6 of 7. Compare drafts on counts, not on the list.

The model did the judging. The script only checks the canary and the quotes. Both reads put the same two findings at the top, the unexplained sync and the unexplained name. They agreed on six of seven, which is closer than I'd expect on a long page. On a whole site I've seen two reads keep only about half the same findings, which is why I compare drafts on how many findings each gets, not on which ones.

My first canary question was a bad one. It asked what was said earlier in the conversation, and a cold model can truthfully answer: nothing. So every run looked as if it had seen more than the page. A question only the source can answer, such as the name of the file the page is built from, works.

How I run it

There are two forms, a script and an agent. On one site, a script like this one reads every screen of every page at two widths and writes a file of counts each time the site is built.

On this site I use the agent form: an agent, defined in a file of instructions, that is told it has never seen the draft and is given the path to the page's text and nothing else. It scores the text against a written guide with a real before and after for each point. This page went through it before it was published.

Where a plain review falls short

The plain form is to ask a model to review the page. These are the gaps I've met.

  • It saw how the page was made. The session that wrote the page agrees with it. Review in a fresh context.
  • It read the source. Comments and the names of a page's parts explain what the visitor never sees. Give it the rendered text only.
  • It knows more than it says. Saved notes, other open files and the rest of the site can leak in without a trace. The canary catches a reviewer that has them.
  • One run is taken as the answer. Runs differ. Measure how far two runs on one draft agree before you compare drafts, and compare on counts.
  • The same flag comes back every time. A flag ruled fine, with no record of the ruling, costs a person the same minute on every run.
  • The reader is pitched too low. A reviewer told to read as a novice asks for rewrites that talk down to a capable reader. Write down who the real reader is, and give the reviewer that.
  • Labels and captions are left out. Text on a button or over a picture is where many unintroduced references sit. Put it in the file, in reading order.

A stronger form

The fullest version writes down what the reviewer is expected to find, and saves it with the date before the run, which borrows pre-registration from science: record the prediction before you see the result. It plants a few known faults, kept where the reviewer can't see them, to show the reviewer finds what is there.

And if two reviewers each read the page without seeing the other's flags, the overlap between their lists lets you estimate how many faults neither found. It is the capture and recapture sum ecologists use to count fish in a lake: catch and mark some, catch again, and the share already marked tells you roughly how many there are in all. I haven't built that for page copy.

Trade-offs

  • A model run for each page, and more to measure the noise. Start with the pages a stranger is likeliest to land on.
  • It over-flags. Some findings are fine as they stand. Each one needs a ruling, and the record keeps a ruling from being needed twice.
  • It says what is unclear, not what is untrue. A page can be easy to follow and untrue. Checking the facts is a separate job.
  • Fixed questions find what they ask about. A problem no question covers is unlikely to be raised.

When to use it, and when not to

It suits any page a stranger can land on: a landing page, a write-up, a help page, the captions on a picture. It is not worth running on a screen only your own team uses, since they share the context. And a set of screens read in order counts as one page: introduce a term once in the set, not on every screen, or the text starts to talk down.

Example. The reference scan and run checks, on GitHub

A draft skill for this pattern

A skill is a short set of instructions you give a coding agent so it applies a pattern the way you would. Copy the text below into a file called SKILL.md, in the folder your coding agent reads skills from. In Claude Code, Anthropic's coding agent, that is .claude/skills/cold-reader/. It is a draft, and I haven't tried it on anyone else's project yet.

cold-reader/SKILL.md
---
name: cold-reader
description: Use before a web page or screen ships, to have a reviewer that sees only the rendered text, with no code, conversation or other pages, flag every name, term or "the X" a first-time visitor could not resolve.
---
# Cold reader
A reviewer who saw how a page was made will understand it, and so will not
see what a stranger cannot. Take everything away except what the page shows,
ask fixed questions, and prove the reviewer really was cold.
## Before you start
1. Render the page's text to a plain file, in the order a visitor reads it:
headings, prose, labels on controls, captions and text drawn over
pictures. Never give the reviewer the source file. The names of a page's
parts (its components) and the comments in the code resolve references
the visitor never sees.
2. Decide who the real reader is and write it down in one line. Do not give
the reviewer a persona younger or less able than that reader. A reviewer
told to read as a novice asks for rewrites that read as talking down.
## Build it
1. Write the reader's brief once and keep it in a file: who the reader is,
the rules the page is held to (every page is page one; introduce before
you refer; define a term once), the kinds of unresolved reference
("the X", a name, a term, a line that assumes another page), a severity
for each, and "quote the exact text and suggest a rewrite of its first
mention".
2. Call the model so that nothing but the brief and the screen reaches it:
a new process for each call, started in an empty folder, with an
environment of the path and home folder only, no settings files, no
instructions file, no tools, no tool servers and no saved session. Ask
for the reply in a fixed structure, so it can be counted.
3. Ask a canary first: two or three questions only your own setup could
answer, such as the project's name or the file the page is built from.
Stop unless every answer is "cannot tell". Check the questions on a
cold run before you trust them: one that a cold model can answer
honestly ("what was said earlier?" gets "nothing") makes every run
look warm.
4. Drop any finding whose quote is not on the page. The model made it up.
5. Before you compare two drafts, read the same unchanged draft twice and
see how far the runs agree. Compare drafts on the count of findings
over several runs, not on which findings one run raised.
6. Fix every finding, or record why it is fine: the reference, the reason
and the date. Give that record to the next run so a ruled-on finding is
not raised again.
7. Read again after the fixes, cold, in a new process.
## Prove it can fail
Copy the page, add one name it never introduces, and run the reviewer on
the copy. If the name is not flagged, the reviewer is not reading closely
enough to trust a clean result. For a stronger test, write down what you
expect it to find before you run it, and plant a few known faults whose list
you keep where the reviewer cannot see it.
## What it does not do
It finds references a visitor cannot resolve. It does not check that the
page is true, or that the page is worth reading. A set of pages read in order
counts as one page: introduce a term once in the set, not on every screen.