Example Notebooks
Play with code notebooks to get started quickly in the browser
Check out the Copilot to explore these techniques in a conversational interface.
Scoring
Bring some questions and a response to score, and see how Pi handles them, end-to-end.
Model Comparison
With a good scoring system, you can now evaluate different models to see which one hits the price/performance/latency tradeoff that you’re looking for, without relying on vibes alone as a benchmark.
Dataset Filtering
You can take a larger amount of data (from Hugging Face) or any other source, filter out irrelevant data, and arrive at a better set for training or evaluation.
Generate Sythetic Training Data
This notebook walks you through generating a synthetic training set and filtering it against your Pi Scoring System so that you can train on the “good” examples.
Calibration
Some questions are more important than others. Calibration lets you more heavily weight “important” questions when computing your final score. This weighting is learned from your own user feedback and ratings.