Pi Scorer is a deterministic, fast foundation model that scores text against a rubric of natural-language questions. In this guide, you will evaluate a simple text generation with one question.
Try this quickstart out in Google Colab.
1

Grab an API key

You can get one from your account page.
2

Install the SDK

Choose your language and follow along.
pip install withpi
3

Start scoring!

Pass questions to the Score API to see how it responds
import os
from withpi import PiClient

pi = PiClient(api_key=os.environ['WITHPI_API_KEY'])
scores = pi.scoring_system.score(
llm_input="Pi Labs",
llm_output="Score anything with Pi Labs today!",
scoring_spec=[{"question": "Is there a strong call to action?"}]
)
print(scores.total_score)

​
Next steps

Congratulations! You’ve issued your first call to Pi Scorer. Here are some suggestions for what to do next:

Pi Scorer Overview

Learn how to build an evaluation with Pi Scorer.

Example Notebooks

Jump into Examples in Google Colab

Pi Studio

TODO: Build a rubric in Pi Studio