Lead search tests: the bad matches I need to see

The question

Before I trust a lead-search instruction, I need to see its bad matches.

A prompt can sound sensible and still send the search toward the wrong work. A list can contain real companies while giving me little reason to approach them. My bar is a set of examples I can inspect: what passed, what failed, why the decisions were made and what the work cost.

I want a person to see that record before a trial becomes the team’s method. Testing an idea should leave room to reject it, change it or keep only part of it.

The criteria are part of the question

In an r/sales discussion about finding leads[1], u/No_Way_1569 described spending time tweaking filters and job titles while “hoping we’re not missing out on good prospects.” They asked how to prospect when the right criteria are not known in advance.

That is one team’s account. It does not establish that a particular experiment will improve results. It describes the uncertainty I would want a test to make visible.

In the same discussion, u/MidFunnel[2] described starting from a customer pain, translating it into company filters and setting up multiple searches when one filter could not express the idea. The commenter also described their own product, so the account has a commercial context.

I would keep the reasoning behind each of those searches. It gives a reviewer something more specific to challenge than whether the final list looks good. Neither commenter measured an improvement from the method proposed here.

Name the decision before running the test

Consider a fictional software-services team. Its current instructions search for companies hiring data engineers. It wants to test whether an announced data-platform migration produces companies that are easier to qualify.

For that test, I need both instruction versions and a written description of the change. The offer, countries and time window should stay stable unless one is the variable being tested.

Then I would define a useful result. In this example, it might need a real company in the chosen market, a dated source about a relevant project, a connection to the service and a person connected to the work.

A result whose source cannot be opened needs its own label. So does a result where the event is relevant but the person’s role is uncertain. I would leave those uncertainties visible for review.

Give both versions the same review

The review rule has to exist before I see which version found more companies. Otherwise it is too easy to change the standard around a result I like.

I would review a mixed sample from both versions and, where practical, hide which version found each company. If the review rule changes, I would record the change and revisit earlier examples.

The NIST guide to experimental design[3] describes setting objectives and planning how variables will change before observing the response. A live web comparison has limits that remain even with that discipline. Pages, dates and provider coverage can change between runs.

My read is that a small test earns another test when its examples hold up. It gives me a clearer next decision; it does not forecast pipeline by itself.

Show me the rejections too

Suppose the existing instructions return 10 companies and the draft returns 6. Those numbers are invented for this example. I would open the evidence before choosing either version.

I need to know whether both searches found the same companies through different events, whether the contact is connected to the work, and whether the reason to approach the company survives a source check.

I would also read rejected examples. A migration announcement might be correctly set aside because the work is finished, while the explanation calls it a request for a freelancer. This is a fictional example of a wrong reason. Changing the freelancer rule would address the wrong problem.

The result I want from the review is a correction I can name. It could be a search term, a source choice, a reading error or a rule the team disagrees with. Keeping the rejected record beside its explanation makes that discussion possible.

A test boundary and a budget limit

Mach’s inspected test path accepts saved instructions or a draft that has not been made live. It keeps test results apart from the normal lead list. A person can inspect the prompt, searches, results and costs, and the comparison groups company domains found by both tests and those unique to either one.

Selected results from a finished test can go through a separate judging run. I want that choice to remain visible: a candidate from a trial still needs a decision before the team treats it as an accepted lead.

The budget wording needs care. The form’s source text calls the field “Most to spend, US dollars.” In this server path, the selected amount is a data budget and a model allowance of up to $1.50 is added. Adding selected results starts a separate judging operation. The source review did not run a paid test or observe an actual bill.

That is a limit in the current wording I would keep beside any cost comparison. My standard for a useful test record is the full cost of the work: data, model, later judging and the time people spend checking results.

Keep the decision record

Before adopting a change, I would start by opening 3 results from each version with another seller. I want to know whether they can understand why each company belongs and what they would ask next. That is a starting diagnostic, not a sample-size guarantee.

If both versions produce weak matches, I want that finding kept too. It tells the team what remains unresolved before it spends again.

For a test that finds nothing, use the empty-search review. For an evaluation that continues into outreach, the published AI SDR pilot scorecard separates research quality, messages, replies, meetings and cost.

How this post was made: AI assisted the research and drafting. Quotes were checked against retrieved public records, and implementation details against a source snapshot dated 5 November 2026. The source review covered code and public writing; live searches and customer outcomes were outside its scope.

Which rejected example would change your mind about a lead-search instruction?

Related reading

This post is part of our guide to finding buyers who are asking for what you sell.

Related reading: What one real lead search cost, line by line and Lead search shipped in five days and seventeen rule versions.