Benchmarked our AI-first RAG-web-search API against popular alternatives

When building AI powered applications connecting your large language model to the internet becomes necessary to keep it relevant and aware of present day facts.

A few weeks ago, I set out to test a simple hypothesis: For RAG systems, does the way you scrape a webpage actually matter?

I took one public article, sent it through five different extraction methods, and measured what came out the other side. Not to declare a “winner”, many of these tools are built for different use cases, but to see what happens when you optimize purely for RAG data quality.

The Test Setup

Source: A Yale School of Public Health news article about pandemic forecasting – dense with names, numbers, and research references.

Prompt (for our pipeline): “What is the next pandemic going to look like and what are the chances of it ever occurring?”
(Note: The other APIs don’t accept a prompt for scraping, they simply extract the page content.)

APIs compared:

• Antinode-AI (our deterministic pipeline)

• Tavily (default mode)

• Tavily (with prompt in request)

• Firecrawl

• Jina Reader

What I Measured

• Fact retention: Imanually identified 40 concrete facts in the original article (names, numbers, dates, quotes) and checked how many survived in each output.

• Noise artifacts: Navigation menus, cookie banners, social share buttons, repeated image captions, footers, and any other non‑article content.

• Payload size & RAG‑readiness: Could you feed the output directly into an embedding pipeline, or would you need to write post‑processing scripts first?

The Results

• Antinode-AI (Our API)- Time taken- 6.7 seconds , fact retention- 100%(40/40), Noise     elements-0,     Output payload size - 10kb clean markdown, Ready for RAG- yes! feed directly into embeddings/LLM, privacy-All on device, cost- $0 self hosted

• Tavily (default) -   Time taken 2-3 seconds , fact retention- 97.5%(39/40), Noise     elements-8, Output payload size - 12-15kb clean markdown, Ready for RAG- needs cleanup, privacy-Data sent to tavily servers, cost- Freemium with per query cost

• Tavily (with prompt) - Time taken 2 seconds , fact retention- 12.5%(5/40), Noise  elements-3, Output payload size - 5kb clean markdown, Ready for RAG- no for us massive data loss, privacy-Data sent to tavily servers, cost- Freemium with per query cost

• Firecrawl -  Time taken- 3 seconds , fact retention- 100%(40/40), Noise     elements-25+(full menu, footers,etc), Output payload size ~ 50kb full page content, Ready for RAG- Massive NO! Needs heavy cleanings, privacy-Running on firecrawl servers, cost- Freemium then paid API.

• Jina reader - Time taken- 3.5seconds , fact retention- 100%(40/40), Noise     elements-40+, Output payload size - ~50kb full page content, Ready for RAG-Absolutely not, needs heavy cleaning, privacy-Jina servers, cost-Freemium then read token cost

Time to return text ranged from 2.5s (Jina) to 6.7s (our pipeline). We’re a bit slower, but deliver a string that needs zero cleanup.

Key Takeaways

• LLM‑based “smart” extraction is unpredictable. Tavily’s short markdown quietly deleted 87.5% of the article’s facts. The same service, under a different mode, kept almost everything but added noise.

• Raw “scrape everything” approaches waste tokens. Firecrawl and Jina dumped the entire page , mega‑menus, cookie banners, browser warnings , directly into the output. That’s extra work and cost downstream.

• Deterministic rules (our approach) offer consistency. Every run returns the same fact‑complete, noise‑free text. No random data loss, no hallucinated cleaning.

Final Thoughts

I'm not saying these tools are bad. They’re built for different jobs , some focus on AI‑powered search, others on end‑user answers. But if you’re building a RAG system where data quality and privacy are non‑negotiable, the extraction layer deserves as much attention as the retrieval or generation steps.

I'm be happy to share the raw outputs, the 40‑fact checklist, or our full methodology. Always open to corrections, suggestions, or alternative tests.

You can see the detailed report at -https://docs.google.com/document/d/1gCArVyDAAPsDnMQ7ry7qfG-PTBFuy5wq4Uh7hwBpqi0/edit?usp=sharing