How I used Promise.any() to make free AI models reliable enough for production

Free AI models on OpenRouter are genuinely good. But they come with a catch — 429s, slow responses, models that go offline mid-generation, and no way to know which one is actually available at any given moment.

I hit this problem building siteblaze, a CLI that scaffolds React sites from a plain English prompt. The generation step needed to be reliable. Paying for a premium model wasn't the goal — the whole point was free by default.

Here's how I solved it.

If this sounds useful, support siteblaze on Peerlist 👉 https://peerlist.io/deepcoomer45/project/siteblaze — then read on for how it works.

The problem with free models

When you call a single free model on OpenRouter you're gambling on:

• Rate limits — 429 if too many requests hit the same model

• Availability — models go offline, get slow, or return empty responses

• Context limits — smaller free models sometimes truncate mid-response

The naive fix is a retry loop — call one model, if it fails try the next. But that's sequential. If model 1 takes 8 seconds to fail, you've already wasted 8 seconds before even trying model 2.

The fix — race them all

const result = await Promise.any(
 models.map(model => callModel(model, prompt))
);

Promise.any() fires all requests simultaneously and resolves with the first successful response. The rest are aborted via AbortController.

Here's the actual implementation from siteblaze:

async function generateWithRacing(
 prompt: string,
 models: string[],
 apiKey: string
): Promise<string> {
 const controllers = models.map(() => new AbortController());

 const promises = models.map((model, i) =>
 callOpenRouter(model, prompt, apiKey, controllers[i].signal)
 .then(result => {
 // abort all other in-flight requests
 controllers.forEach((c, j) => { if (j !== i) c.abort(); });
 return result;
 })
 );

 return Promise.any(promises);
}

If all models fail, Promise.any() throws an AggregateError — you catch it and surface a clean error to the user instead of a cryptic timeout.

Why this works for free tier specifically

Free models fail independently — a 429 on one model doesn't mean the others are rate limited. By racing 9+ models simultaneously:

• Speed — you get the fastest responding model, not the first one you tried

• Reliability — one 429 or dead model doesn't block you

• No cost — you're still on the free tier, just using it smarter

The key insight: free model flakiness is a coordination problem, not a quality problem. Promise.any() solves the coordination.

The constraint that makes it work

Racing models only works when your prompt is small and well-defined. If you're asking models to generate 500 lines of code, racing doesn't help — you need a capable model, not just a fast one.

siteblaze sidesteps this by never asking models to generate code. Instead the AI fills a structured JSON schema:

{
 "sections": ["HERO", "FEATURES", "PRICING"],
 "metadata": {
 "siteName": "Syncly",
 "primaryColor": "#6366f1",
 "themeMode": "dark"
 }
}

A fixed, tested component library turns that config into real React code. The AI's job is small and well-defined enough that free models handle it reliably — and fast enough that racing actually saves meaningful time.

Results

Before racing: generation would fail or timeout on busy models frequently.

After racing 9+ models: near-zero failures in practice. First valid response typically arrives in 8-15 seconds.

Try it

siteblaze is open source and free to use with a free OpenRouter key:

npx siteblaze@latest generate "your idea here"

The model racing logic lives in apps/cli/src/index.ts if you want to see the full implementation.

• GitHub → https://github.com/DeepCoomer/siteblaze

• npm → https://www.npmjs.com/package/siteblaze

• Support on Peerlist → https://peerlist.io/deepcoomer45/project/siteblaze