Pi lets you evaluate and control LLM applications without specialized ML expertise, whether they are conversational, generative, or agentic. Our core technology is a deterministic, fast foundation model that assigns a score to any question, like “Is this response relevant to the user’s request?”, or “Is all the information needed to answer the question present?” Our copilot helps you build, test, and calibrate these questions with examples. You can then use these questions, in real-time, to control applications and training workflows.

​
Quickstart

Try out Pi in the browser by following along with a sample notebook.
1

Grab an API key

You can get one from the account page.
2

Install the SDK

Choose your language and follow along.
pip install withpi
3

Start scoring!

Pass questions to the Score API to see how it responds
from withpi import PiClient

pi = PiClient()
scores = pi.scoring_system.score(
  llm_input="Pi Labs",
  llm_output="Score anything with Pi Labs today!",
  scoring_spec=[{"question": "Is there a strong call to action?"}]
)
print(scores.total_score)
4

What's next?

  • Scale up and score a bunch of your data!
  • Look at other notebooks for inspiration.
  • Dive into our API Reference on the left to see other more specialized components.
  • Check out the Copilot
  • Reach out to us on Discord and share how it looks!