About This Project

This project, fannypack-agents, provides fast, reversible tool calls for LLM agents. It focuses on optimizing the performance of Large Language Model (LLM) agents by shaving off latency and enabling selective undo functionality over an action ledger.

Key features include:

• Latency Shaving: Achieves 1.2–1.8x speed improvements in tool-calling turns by optimizing runtime processes, especially beneficial for slower or repeated calls.

• Selective Undo: Allows for the reversal of specific actions without affecting independent, non-dependent actions. The system clearly indicates whether an action is reversible or irreversible.

• Efficient Tool Integration: Tools are registered using decorators, and their capabilities are defined by their effect classes. The system supports various effect types such as READ_ONLY and IDEMPOTENT_WRITE.

• Reversible Operations: Implements compensator functions to enable the reversal of operations like transfers, ensuring data integrity.

• Snapshotting for Writes: Captures the state before a write operation to facilitate accurate reversals for operations like file edits.

• Prefill and Policy Learning: Optimizes performance by pre-firing read-only calls and learning policies to decline operations that would not yield performance gains.

• Deterministic Routing and Classification: Utilizes deterministic mechanisms for tool selection and classification, reducing reliance on the LLM for these tasks.

The project is built with Python and is compatible with Python 3.10+. It integrates with various LLM providers, including Anthropic, OpenAI, and Fireworks, and supports Ollama.

Installation can be done via pip or from source. The project also includes extensive documentation and experimental results to demonstrate its performance benefits.