Integrations
Use Pi with other popular frameworks
Braintrust
Braintrust lets you evaluate an LLM. This Colab shows you how to use Pi as a custom scorer within a Braintrust Eval.
Langfuse
Langfuse provides a set of tools for building LLM applications.
Pi can serve as a Scoring function within Langfuse.
Promptfoo
Promptfoo focuses on evaluating guardrails for LLM applications. Pi can be a custom metric within the platform.
Langsmith
Langsmith by Langchain helps you build orchestrated applications. This notebook demonstrates how to use Pi as a Langsmith evaluator.
SFT
This notebook demonstrates how to evaluate a model checkpoint using Pi with the Unsloth toolkit.
GRPO
GRPO enables tuning a large language model using a robust reward function—without the need for labeled training data. This notebook demonstrates how to use Pi as the reward function with the Unsloth toolkit.
Helicone
See webhook integration examples
Helicone is an open-source observability platform for LLM applications. Pi can be a custom metric within the platform, helping you annotate quality in addition to performance.