Chokepoint: LLM Call Governance & Observability Gateway

Chokepoint is a powerful, drop-in reverse proxy designed to sit between your applications and Large Language Model (LLM) providers like OpenAI and Anthropic. It acts as a single interception point for all LLM API calls, enabling robust governance and observability.

Key Features:

• Drop-in Proxy: Seamlessly integrate by pointing your existing SDKs (OpenAI, Anthropic) to Chokepoint without code rewrites.
• PII and Secret Detection: Automatically scans prompts for sensitive data such as email addresses, phone numbers, Social Security Numbers, credit card details, IP addresses, AWS keys, API keys, and JWTs, redacting them before they leave your environment.
• Policy Engine: Enforce custom policies with an ordered rule set that can block, redact, or reroute calls based on detected PII, the model being used, the team making the request, or prompt size. Rules can be updated dynamically via the API.
• Cost and Usage Tracking: Monitor LLM spending with per-call cost calculations based on token usage and a configurable price list, aggregated by model, team, and feature.
• Immutable Audit Log: Maintain a secure, queryable history of all LLM interactions. By default, logs contain redacted excerpts, with an explicit opt-in for storing raw content.
• Configurable Retention: Manage log data efficiently with a background pruner that removes records beyond a specified retention window.
• Latency and Error Monitoring: Track provider and model-specific latency, timeouts, and rate limits. Metrics are exposed in both JSON and Prometheus text formats.
• Dashboard: Access a zero-dependency HTML dashboard for an at-a-glance view of your LLM activity.

Chokepoint provides the essential API-gateway pattern, adapted for LLM calls, to give teams visibility and control over their AI feature usage, costs, and data security.