Skip to main content
Pi Scorer is a deterministic, fast foundation model that scores text against a rubric of natural-language questions. In this guide, you will evaluate a simple text generation with one question.
See an Google Colab.
1

Grab an API key

You can get one from your account page.
2

Install the SDK

Choose your language and follow along.
pip install withpi
3

Start scoring!

Pass questions to the Score API to see how it responds
import os
from withpi import PiClient

pi = PiClient(api_key=os.environ['WITHPI_API_KEY'])
scores = pi.scoring_system.score(
  llm_input="Pi Labs",
  llm_output="Score anything with Pi Labs today!",
  scoring_spec=[{"question": "Is there a strong call to action?"}]
)
print(scores.total_score)

​
Next steps

Congratulations! You’ve issued your first call to Pi Scorer. Here are some suggestions for what to do next:

Writing Rubrics

Learn how to build an evaluation with Pi Scorer.

Example Notebooks

Jump into Examples in Google Colab.
https://mintcdn.com/pilabs/FuJrErWLPBFvXYVV/assets/pi-studio.svg?fit=max&auto=format&n=FuJrErWLPBFvXYVV&q=85&s=2b3018a826b9aa9cca0f630d1ee1d38b

Pi Studio

Generate a rubric tuned to your data.