Parea AI LogoParea

Test and Evaluate your AI systems

From experiment tracking, to observability, to human annotation,Parea helps teams confidently ship LLM apps to production.

TRUSTED BY TEAMS AT

https://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395af8f432a560d37f8d_maestrolabs_logo.jpeghttps://framerusercontent.com/images/aY6Q4SWoy8enOp9JEFs0CtbDJCs.jpghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66424a5a501a76e230de443b_rowsie_ai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66298f22eba31dd243a9dd7e_6MB_fqkE_400x400.jpghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66145f31ee867a1c057e04b4_sweepai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66145f32fa8c5f9483066fdb_xeol_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395a893908db90a19a6e_ventaai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395acf9d83a421827345_sweetspot_logo.jpeg/usefulhq_logo.jpeg/gainsystems_logo.jpeghttps://media.licdn.com/dms/image/v2/D4E0BAQFE2gBdV6zaMw/company-logo_200_200/company-logo_200_200/0/1683807837260/sixfold_ai_logo?e=2147483647&v=beta&t=qsPd7i-XbGH5DXXPxT8GT3eRfsoz5SPm95AWdb1dILQhttps://codestory.ai/_next/image?url=%2Fcs-logomark.png&w=48&q=75
https://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395af8f432a560d37f8d_maestrolabs_logo.jpeghttps://framerusercontent.com/images/aY6Q4SWoy8enOp9JEFs0CtbDJCs.jpghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66424a5a501a76e230de443b_rowsie_ai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66298f22eba31dd243a9dd7e_6MB_fqkE_400x400.jpghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66145f31ee867a1c057e04b4_sweepai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66145f32fa8c5f9483066fdb_xeol_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395a893908db90a19a6e_ventaai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395acf9d83a421827345_sweetspot_logo.jpeg/usefulhq_logo.jpeg/gainsystems_logo.jpeghttps://media.licdn.com/dms/image/v2/D4E0BAQFE2gBdV6zaMw/company-logo_200_200/company-logo_200_200/0/1683807837260/sixfold_ai_logo?e=2147483647&v=beta&t=qsPd7i-XbGH5DXXPxT8GT3eRfsoz5SPm95AWdb1dILQhttps://codestory.ai/_next/image?url=%2Fcs-logomark.png&w=48&q=75
https://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395af8f432a560d37f8d_maestrolabs_logo.jpeghttps://framerusercontent.com/images/aY6Q4SWoy8enOp9JEFs0CtbDJCs.jpghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66424a5a501a76e230de443b_rowsie_ai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66298f22eba31dd243a9dd7e_6MB_fqkE_400x400.jpghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66145f31ee867a1c057e04b4_sweepai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66145f32fa8c5f9483066fdb_xeol_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395a893908db90a19a6e_ventaai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395acf9d83a421827345_sweetspot_logo.jpeg/usefulhq_logo.jpeg/gainsystems_logo.jpeghttps://media.licdn.com/dms/image/v2/D4E0BAQFE2gBdV6zaMw/company-logo_200_200/company-logo_200_200/0/1683807837260/sixfold_ai_logo?e=2147483647&v=beta&t=qsPd7i-XbGH5DXXPxT8GT3eRfsoz5SPm95AWdb1dILQhttps://codestory.ai/_next/image?url=%2Fcs-logomark.png&w=48&q=75
https://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395af8f432a560d37f8d_maestrolabs_logo.jpeghttps://framerusercontent.com/images/aY6Q4SWoy8enOp9JEFs0CtbDJCs.jpghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66424a5a501a76e230de443b_rowsie_ai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66298f22eba31dd243a9dd7e_6MB_fqkE_400x400.jpghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66145f31ee867a1c057e04b4_sweepai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/66145f32fa8c5f9483066fdb_xeol_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395a893908db90a19a6e_ventaai_logo.jpeghttps://assets-global.website-files.com/64754d86591b8cffd50b9e95/6602395acf9d83a421827345_sweetspot_logo.jpeg/usefulhq_logo.jpeg/gainsystems_logo.jpeghttps://media.licdn.com/dms/image/v2/D4E0BAQFE2gBdV6zaMw/company-logo_200_200/company-logo_200_200/0/1683807837260/sixfold_ai_logo?e=2147483647&v=beta&t=qsPd7i-XbGH5DXXPxT8GT3eRfsoz5SPm95AWdb1dILQhttps://codestory.ai/_next/image?url=%2Fcs-logomark.png&w=48&q=75
Eval Trends OverviewCompare score distributionsCompare experiments

Evaluation

Test, and track performance over time. Debug failures. Answer questions like “which samples regressed when I made a change?”, and “does upgrading to this new model improve performance?”

Manual Review View

Human Review

Collect human feedback from end users, subject matter experts, and product teams. Comment on, annotate and label logs for Q&A and fine-tuning.

Prompt Playground & Deployment

Tinker with multiple prompts on samples, test them on large datasets, and deploy the good ones into production.

Trace

Observability

Log production and staging data. Debug issues, run online evals, and capture user feedback. Track cost, latency, and quality in one place.

Datasets

Incorporate logs from staging & production into test datasets and use them to fine-tune models.

Simple Python & JavaScript SDKs

Python
from openai import OpenAI
from parea import Parea, trace
 
client = OpenAI()
p = Parea(api_key="PAREA_API_KEY")
p.wrap_openai_client(client)  # auto-trace LLM calls
 
# trace and evaluate any steps
@trace(eval_funcs=[...])
def func(...):
    ...
 
# run tests on datasets
p.experiment(...)
TypeScript
import OpenAI from "openai";
import { Parea, patchOpenAI, trace } from "parea-ai";
 
const openai = new OpenAI();
const p = new Parea(process.env.PAREA_API_KEY);
patchOpenAI(openai);  // auto-trace LLM calls
 
const func = trace(  // trace step
  'funcName', origFunc,
  { evalFuncs: [...], },  // evaluate step
);
 
// run tests on datasets
p.experiment(...)

Native integrations to major LLM providers & frameworks

Pricing for teams of all sizes

Get started on the Builder plan for free. No credit card required.

AnnualSave 20%

Free

$0 / month

  • All platform features
  • Max. 2 team members
  • 3k logs / month (1 mon retention)
  • 10 deployed prompts
  • Discord community

Team

$150 / month

  • 3 members ($50 / month per add'l. member up to 20)
  • 100k logs / month incl. ($0.001 / extra log)
  • 3 month data retention, (6/12 mon upgrade)
  • Unlimited projects
  • 100 deployed prompts
  • Private Slack channel

Enterprise

Custom

  • On-prem/self-hosting
  • Support SLAs
  • Unlimited logs
  • Unlimited deployed prompts
  • SSO enforcement and custom roles
  • Additional security and compliance features

AI Consulting

Custom

  • Rapid Prototyping & Research
  • Building domain-specific evals
  • Optimizing RAG pipelines
  • Upskilling your team on LLMs