Backend / Applied AI Engineer — LLM agents, RAG, evals, AI control. Building AI systems that can prove what they claim.

hashnode

github

Ebin Babu Thomas  • Peerlist

THE PROBLEMPurchase pre-approvals in OPWDD Self-Direction (a Medicaid-audited NY disability program) require checking every request against category rules, fee caps, and evidence from vendor websites. Reviews are slow, manual, and audit-sensitive — and an AI assistant that hallucinates evidence would be worse than no assistant at all.WHAT IT DOESPre-Approvals Reviewer automates the full review as a five-stage pipeline:<ol><li>Vision-based PDF extraction (Claude) turns request documents into structured data</li><li>Category-routed YAML checklists — editable by non-engineers — decide what must be verified</li><li>Deterministic code enforces fee caps and eligibility rules</li><li>A browser research agent (Anthropic SDK tool-runner + Playwright) verifies claims on real vendor websites</li><li>HTML/JSON reports assemble the verdict with a date-stamped screenshot evidence folder</li></ol>WHY IT'S DIFFERENT: EVIDENCE IT CAN'T FAKEIntegrity gates make fabricated evidence structurally impossible, not just unlikely:<ul><li>A "Found" verdict cannot be recorded without a real on-disk screenshot</li><li>Quotes are rejected unless they appear verbatim on a visited page</li><li>Timestamps, URLs, and SHA-256 hashes are written by code the model never touches</li><li>Anything the web can't prove stays explicitly "Internal — not answered"</li></ul>VALIDATION18 deterministic tests, a rerunnable hash-manifest audit over every evidence package, human ground-truthing of negative findings, and planted-trap cases (an excluded laptop, an upheld denial) the agent caught. The find-fix-verify loop also surfaced and fixed a real prompt-labeling bug.Open source, MIT licensed. Built solo, July 2026.<a target="_blank" rel="noopener noreferrer nofollow ugc" class="break-all link-break text-gray-500 hover:underline" href="https://github.com/ebt55/preapproval-verification-challenge">https://github.com/ebt55/preapproval-verification-challenge</a>