Quality
How well the AI does, with the method
ReadyBuyerFinder is free; you pay third-party costs at cost. We publish our AI test results so you can judge them yourself. These are small, fixed test sets that we wrote, not customer data. We make no reply-rate or meeting-rate claims, and won't until there is real customer data to measure.
Model
Surge's own server, qwen3.8-27b (open-weight Qwen)
Date
24 October 2026, two runs
Script
scripts/eval-ai.ts in our codebase (not published)
| Task | What is checked | Sample size | Run 1 | Run 2 |
|---|---|---|---|---|
| Signal screening | Keep the real buying moments and drop the noise (for example an investor's fund announcement or an unrelated celebrity launch). | 6 fixed signals | 6/6 | 6/6 |
| Company named | Name the company the signal is about, not the investor or the publisher. | 3 signals with a company | 3/3 | 3/3 |
| First-message drafts | Drafts that pass the code fact-check (numbers, rounds, titles, first name, no tracking language). | 4 drafts | 4/4 | 4/4 |
| Reply labels | Label a reply: interested, meeting, referral, not now, not interested, asked to stop, out of office, wrong person. | 5 fixed replies | 5/5 | 4/5 |
| "Ask your pipeline" parsing | Turn a plain-English question into the right lead filters. | 3 fixed questions | 3/3 | 3/3 |
The one miss: in the second run, a reply that pointed us to a colleague (a referral) was labelled "wrong person". We then tightened the prompt and the rules for those two labels; results after that fix will be published here from the next run.
Method
- Each test case has a fixed expected answer written before the run. The script sends them to the AI one call at a time and compares each answer with the expected one.
- Drafts are scored by the same code fact-check that guards real drafts, not by another AI.
- The test companies are fictional sample companies, and no real person is contacted.
- Runs never happen between 02:00 and 06:00 IST, when the shared AI server is reserved.
What these numbers don't tell you
- With 3 to 6 cases per task, a perfect score shows the basics work; it is not a measured accuracy rate.
- They say nothing about reply rates, meetings or revenue.
- Your own results depend on your ICP, your signals and your market. In the app, every signal decision and every fact-check result is shown with its reason, so you can check the AI on your own leads.
We will add results here as we run larger tests, with the date and sample size each time. See how drafts are fact-checked and how signals are screened and scored.