Integrations

​

Braintrust

Open in Colab

Braintrust lets you evaluate an LLM. This Colab shows you how to use Pi as a custom scorer within a Braintrust Eval.

​

Langfuse

Open in Colab

Langfuse provides a set of tools for building LLM applications.
Pi can serve as a Scoring function within Langfuse.

​

Promptfoo

Open in Colab

Promptfoo focuses on evaluating guardrails for LLM applications. Pi can be a custom metric within the platform.

​

Langsmith

Open in Colab

Langsmith by Langchain helps you build orchestrated applications. This notebook demonstrates how to use Pi as a Langsmith evaluator.

​

SFT

Open in Colab

This notebook demonstrates how to evaluate a model checkpoint using Pi with the Unsloth toolkit.

​

GRPO

Open in Colab

GRPO enables tuning a large language model using a robust reward function—without the need for labeled training data. This notebook demonstrates how to use Pi as the reward function with the Unsloth toolkit.

​

Helicone

See webhook integration examples

Helicone is an open-source observability platform for LLM applications. Pi can be a custom metric within the platform, helping you annotate quality in addition to performance.