What is free, and what Pro adds
Free shows you where your AI spend is going, and checks CI against the limits you set. Pro, coming soon at $99 per month billed yearly, compares every pull request with your last good run, task by task, to catch regressions a threshold lets through. Join the interest list to request a 30-day Pro trial key of the regression gate.
Free
Try ParetoOps on your own eval results, locally or in any CI.
- Pareto frontier of cost per successful task, with the sweet spot and recommendations (
analyze) - Fail a CI job on accuracy and cost-per-success thresholds (
gate), with configurable gating modes (Hard, Informational, Off) - CI gate with job summary and PR comments (
pareto-ops ci, free key at paretoops.dev/free) - Reads Promptfoo, LangSmith, Braintrust, DeepEval, CSV and JSON results
- Built-in model pricing catalog with prompt-caching, reasoning-token rates, and effective-dating (
--as-of) - Python SDK, and the same CLI via
pip install pareto-opsandnpm install pareto-ops - Runs fully offline: your eval data never leaves your machine or CI runner
Pro
$99 per month, billed yearly · per GitHub organization
For teams that ship prompt and model changes through pull requests.
- Regression gate against the last known good run, paired task by task, with confidence intervals
- Baselines in the CI gate (
pareto-ops ci --baseline): the pull request comment shows what changed since your last good run - Multi-model workflow and agentic trace step-level cost attribution (
analyze --by-step) - Python SDK Pro API: in-process paired regression gate (
compare_paired) and anytime-valid early stopping monitor (EarlyStopMonitor) - Baselines stored on a git branch: save, fetch, pin, diff and prune
- Early stopping: end an eval run once it is clearly failing, before it spends the full budget
- 3D frontier with P95 latency, and confidence-bound ranking
- Everything in Free
Join the interest list to request a 30-day Pro trial key. Early customers keep their launch price.
Enterprise
For organizations with negotiated rates, self-hosted models or air-gapped networks.
- Everything in Pro
- Signed live pricing catalog sync
- Signed catalogs with your negotiated rates, delivered as a feed or an offline file
- Self-hosted GPU cost amortization
- Air-gapped deployment support, with keys that are not tied to GitHub
Pro is licensed per GitHub organization. License keys are checked offline; nothing calls home.
What you get with Pro that a threshold can’t do
A threshold checks one number against a fixed bar. In our sample, a change that still clears its 50% accuracy bar fell 15.0 points against the last good run (95% CI -22.5 to -7.5) and cost ×1.42 more per success. The threshold gate passes it. The paired gate, which compares the same 200 tasks, fails it.
See the full sample report → Real engine output on the sample data shipped with ParetoOps.
Free and Pro side by side
| Feature | Free | Pro |
|---|---|---|
| In Free and Pro | ||
| Pareto frontier, sweet spot and recommendations | ✓ | ✓ |
| CI gate on accuracy and cost-per-success thresholds | ✓ | ✓ |
| CI gate with pull request comments (GitHub Actions step, free key) | ✓ | ✓ |
| Promptfoo, LangSmith, Braintrust, DeepEval, CSV, JSON | ✓ | ✓ |
| Pricing catalog with caching and reasoning-token rates | ✓ | ✓ |
| Python SDK (3D frontier and confidence ranking need Pro) | ✓ | ✓ |
| Pro: what a threshold can’t do | ||
| Paired regression gate vs your last known good run, with confidence intervals | – | ✓ |
| Baselines stored on a git branch (with anchor pinning to stop drift) | – | ✓ |
Early stopping for failing eval runs (watch & EarlyStopMonitor) | – | ✓ |
| Pro: more | ||
Step-level cost attribution for multi-model & agentic workflows (analyze --by-step) | – | ✓ |
| 3D frontier with P95 latency, confidence-bound ranking | – | ✓ |
Python SDK Pro API (compare_paired, EarlyStopMonitor, analyze_steps) | – | ✓ |
Questions about plans
What does ParetoOps Pro add over Free?
Pro is for engineering teams shipping LLM changes through pull requests: a paired, task-by-task regression gate against your last good run (with confidence intervals), baselines stored on a git branch, early stopping for failing eval runs, step-level cost attribution for agentic runs, and a 3D frontier with P95 latency. Pull request comments are part of Free with a free key.
When is ParetoOps Pro available and how do I get early access?
Pro and Enterprise are rolling out soon. Pro will cost $99 per month, billed yearly, per GitHub organization. Join the interest list to hear when it launches and to request a 30-day trial license key before general availability. Early customers keep their launch price.
How is ParetoOps Pro licensed?
Pro is licensed per GitHub organization or team with an offline cryptographically signed license key stored in CI secrets. It never phones home.
Do I need an account to use the Free version?
Local analysis (`analyze`) and threshold gating (`gate`) need no sign-up, credit card or key. The CI gate with PR comments (`ci`) needs a free license key, which you generate at paretoops.dev/free with your email.