Mental Health Chatbot Eval

---
name: mental-health-chatbot-eval
description: Evaluates the safety, clinical adherence, and crisis response quality of LLM-powered mental health chatbots against expert-defined guidelines. It probes the model's ability to provide evidence-based advice, identify health risks, maintain consistent crisis intervention, provide appropriate resources, and empower users. Use when the user wants to benchmark on Institute for Future Health Mental Health Query Set, or asks about evaluating this task. Reports TotalScore.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.16444
  bibtex_key: lyu2025domain
  confidence: high
---

# mental-health-chatbot-eval

> Domain-Specific Constitutional AI: Enhancing Safety in LLM-Powered Mental Health Chatbots — Lyu et al. (2025) (arXiv:2509.16444, 2025)

## What this evaluates

Evaluates the safety, clinical adherence, and crisis response quality of LLM-powered mental health chatbots against expert-defined guidelines. It probes the model's ability to provide evidence-based advice, identify health risks, maintain consistent crisis intervention, provide appropriate resources, and empower users.

## Datasets

- **Institute for Future Health Mental Health Query Set** — total 100; splits: test (100)

## Metrics

- `TotalScore` **(primary)** — range: [0, 50]
  - Sum of scores across five clinical guidelines. Each guideline is scored on a 1-10 scale by trained evaluators using detailed clinical rubrics. The maximum possible score is 50 per response.

## Input / output format

**Input**: Mental health-related queries covering scenarios such as depression, anxiety, crises, and general mental wellness.

**Output**: Natural language chatbot responses to the queries.

## Scoring recipe

```python
def compute_total_score(predictions, guidelines):
    total = 0
    for pred, guideline in zip(predictions, guidelines):
        score = evaluator_rubric_score(pred, guideline)  # 1-10 scale
        total += score
    return total  # Max 50
```

## Common pitfalls

- Evaluators must be trained and strictly follow the provided clinical rubrics; untrained scoring or automated LLM-as-judge methods often fail to capture clinical nuance and deviate from the 1-10 scale.
- The aggregate TotalScore can mask severe deficiencies in critical guidelines (e.g., Guideline 4 on crisis resource provision), so per-guideline breakdowns must be examined alongside the total.

## Evidence (verbatim from paper)

> We use the evaluation metric developed by the Institute for Future Health*[[13]]*, which includes five guideline questions with ground truth for mental health chatbot evaluation: Each model was evaluated based on 100 mental health-related queries about common scenarios like depression, anxiety, crises, and general mental wellness. Health experts provided ideal responses as ground truth. Trained evaluators scored responses on a 1-10 scale per guideline using detailed rubrics aligned with clinical best practices. Total scores represented the sum across all five guidelines, with a maximum possible score of 50 per response.

## Citation

```bibtex
@misc{lyu2025domain,
  title={Domain-Specific Constitutional AI: Enhancing Safety in LLM-Powered Mental Health Chatbots},
  author={Lyu et al. (2025)},
  year={2025},
  note={arXiv:2509.16444}
}
```

- arXiv: 2509.16444