Regression Test Prompts Agents And Rag Outputs Before Shipping Changes

---
name: "Regression-test prompts, agents, and RAG outputs before shipping changes"
slug: "regression-test-prompts-agents-and-rag-outputs-before-shipping-changes"
description: "Use promptfoo when an agent needs to evaluate prompt, agent, or RAG behavior against saved assertions before a change goes live. The value here is the repeatable evaluation workflow, not a generic AI tooling catalog entry."
github_stars: 20007
verification: "security_reviewed"
source: "https://github.com/promptfoo/promptfoo"
author: "promptfoo"
category: "Code Quality & Review"
framework: "Multi-Framework"
tool_ecosystem:
  github_repo: "promptfoo/promptfoo"
  github_stars: 20007
  npm_package: "promptfoo"
  npm_weekly_downloads: 248499
---

# Regression-test prompts, agents, and RAG outputs before shipping changes

Use promptfoo when an agent needs to evaluate prompt, agent, or RAG behavior against saved assertions before a change goes live. The value here is the repeatable evaluation workflow, not a generic AI tooling catalog entry.

## Prerequisites

Node.js, CI pipeline, model provider credentials

## Installation

Use the upstream install or setup path that matches your environment:
- npm install -g promptfoo
- Also available via brew install promptfoo and pip install promptfoo. You can also use npx promptfoo@latest to run any command without installing.

Requirements and caveats from upstream:
- Requires [Node.js](https://nodejs.org/en/download) ^20.20.0 or >=22.22.0 for npm and npx usage.
- Most LLM providers require an API key. Set yours as an environment variable:
- [Node.js Package](https://www.promptfoo.dev/docs/usage/node-package/)

Basic usage or getting-started notes:
- <a href="https://www.promptfoo.dev/docs/getting-started/">Getting Started</a> ·
- sh
- promptfoo init --example getting-started

- Source: https://github.com/promptfoo/promptfoo
- Extracted from upstream docs: https://raw.githubusercontent.com/promptfoo/promptfoo/HEAD/README.md

## Documentation

- https://www.promptfoo.dev/docs/intro/

## Source

- [Agent Skill Exchange](https://agentskillexchange.com/skills/regression-test-prompts-agents-and-rag-outputs-before-shipping-changes/)