Datool is an open-source alternative to Langfuse for developers building AI applications, with a focus on evaluations.

Follow every trace, find what went wrong, and turn real examples into eval cases. Keep prompts versioned, write custom JavaScript or Python scorers, run evaluations, and compare changes against a baseline. Inspect the cases behind the scores so you can see what improved and what still fails.

Datool connects traces, datasets, prompt versions, scorers, and evaluation runs in one workspace. It also provides CLI and MCP workflows for development and agent tooling.

Self-host under Apache 2.0, or use Datool Cloud.

Website: https://www.trydatool.com/

Source: https://github.com/vinipace/datool

Feedback welcome: what would make it easier to turn your AI failures into regression tests?