In this article
A scorecard forced the decision
By 13 October, account research looked finished. It ran, scored companies and filled screens.
On 12 October, we had written an answer key by hand for one real company. Against it, an old run scored 28% recall and 52% precision. The table says what each score means.
| Score | Measures | The old run |
|---|---|---|
| Recall | How much of the hand-checked truth the brief found | 28% |
| Precision | How much of the brief is true | 52% |
Half of what it said was right, and most of what mattered was missing. Nobody tunes their way out of that, so we rebuilt.
The bar is a brief you can read
We wrote the brief a great researcher would write for one real company. By hand, slowly, every claim with a source.
That gold brief became the bar. After every research change, the system's brief for the same company is compared with it, section by section.
A change that makes claims more accurate by dropping hard facts fails that comparison, loudly.
The two ways a brief fails
A recall miss looks like this. The key holds a dated fact that matters, say a price change on the company's own page. The brief never found it.
A precision miss is the other side. The brief says something like 'growing fast' with nothing behind it. Readers cannot tell that line from the solid ones, and that is the harm.
The rebuild fixed both at once. Wider reading fixed the recall misses. For the precision misses, a new rule: a claim with no source does not ship.
The verdict comes first, and can say no
Each brief now opens with a verdict: go after this company or not, and why now.
Facts land on the company record with their sources. Open questions stay open, and the brief says what it could not learn.
The measured result of the rebuilt research, failure rate included, is its own post.
80% of the work is people
The rebuild also moved the effort. About a fifth of the work studies the company. The rest studies the people who decide.
Company news matters when it touches a person who would buy. Fit says whether the company is worth any effort. The people decide what to write.
Copy the method before you buy anything
Write one gold brief by hand for a company you know well. It takes an afternoon and becomes your measuring stick.
Score any research tool against it for recall and precision. Ask that the brief end in a next step, not a pile of facts.
The September lessons that taught us to test against real records are told once, in the build post.
Common questions
What goes in the gold brief?
The verdict, fit, why now, unmet needs, the next step and the people. Check each by hand, with a source. Keep it to one page so the comparison stays honest.
Where the numbers come from
Dates and scores come from our code history and journal. The gold-brief company and every customer stay unnamed.
