In this article
Find people asking in public, right now
The first commit landed on 15 October. The idea: find people asking in public, right now, for what the seller sells.
Not lists of lookalike companies. A dated request, a hiring burst, new funding, a new leader in the seat. What that finds is in one real run's breakdown.
Seventeen versions, each backed by a blind check
The judging rules went through seventeen versions in five days, across 142 commits. The last eleven came in the final three days.
Each version cites a blind check. A reader labels real results without seeing the judge's answer. Where they disagree, the cases go on a list.
Early versions chased keeps and kept too much, so checking time ate the savings. Later versions chased drops with named reasons.
| Version | Frozen rows | Real leads kept | Wrong keeps |
|---|---|---|---|
| v16 | 633 | 24 of 24 | 8 to 9 |
| v17 | 633 | 26 of 26 | 3 to 4 |
| v17, newest rows | 187 | 16 of 18 | 3 to 4 |
Named drops beat clever keeps
A keep is a bet. A drop with a named reason is a fact you can check.
On 18 October, a run marked 234 of 276 posts as freelancer requests, and most were news or opinions. The drops were right, but the reason was wrong. A wrong reason spoils every check after it.
Three new tests came out of that day, one per wrong reason. The news test failed before the fix.
The hiring check that found nothing real
The first hiring check read 1,235 job ads and called 7 of them bursts. All 7 came from job boards, none from a buyer.
After the rebuild, a replay found 30 of 31 real bursts and let no middleman through.
The difference was a tighter question, with a test of known bursts behind it.
Each source ships with its own lock
Requests for proposals shipped with a reader test over 126 real results. Funding rounds shipped after a live check found 4 of 4 rounds with a founder, for 50 cents.
New technology leaders: 173 posts read, 26 kept, 27 cents. Company moves: 62 posts read, 0 kept, 2 cents. That true zero shipped too.
A source that cannot pass its lock does not ship.
19 October: lock it and stop touching it
On 19 October, version 17 became the baseline. Before any change ships, a live lock asks the live judge again about 820 frozen results, labeled blind.
It fails if a real lead would be dropped. Two known misses are allowed, each written down with its reason. Offline copies drift, so the lock asks the real thing.
Copy the checks, not the rules
Number every version of your judging rules. Stamp each result with the version that graded it.
Check real results blind before every change. Freeze a labeled set, and write the allowed misses down.
Fund sources by what survives, and keep the plan bigger than the budget so every dollar has competition.
Common questions
Why number the judging rules at all?
Every graded lead records the version that graded it. Old grades stay explainable after the rules change, and two runs can be compared honestly.
Where the numbers come from
Commits, versions and check figures come from our code history and journal. Customers and vendors stay unnamed.
