Childsafe Safety Eval

---
name: childsafe-safety-eval
description: Evaluates LLM safety alignment across four child developmental stages (ages 6–17) using simulated agents grounded in developmental psychology. It probes how models handle sensitive contexts, boundary-testing, and age-specific cognitive limitations in multi-turn interactions. Use when the user wants to benchmark on ChildSafe Dataset, or asks about evaluating this task. Reports semantic_safety_score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2510.05484
  bibtex_key: [REDACTED]
  confidence: medium
---

# childsafe-safety-eval

> Evaluating LLM Safety Across Child Development Stages: A Simulated Agent Approach — Murali et al. (2025) (arXiv:2510.05484, 2025)

## What this evaluates

Evaluates LLM safety alignment across four child developmental stages (ages 6–17) using simulated agents grounded in developmental psychology. It probes how models handle sensitive contexts, boundary-testing, and age-specific cognitive limitations in multi-turn interactions.

## Datasets

- **ChildSafe Dataset** — total 1200; splits: test (1200)

## Metrics

- `semantic_safety_score` **(primary)** — range: [0, 1]
  - LLM-as-Judge scoring based on predefined rubrics across nine safety dimensions (e.g., privacy, misinformation, emotional support). Scores are averaged across dimensions and conversation traces.

## Input / output format

**Input**: Multi-turn conversation prompts initialized with age-specific system prompts simulating child cognitive profiles (ages 6–8, 9–11, 12–14, 15–17) across sensitive and neutral contexts.

**Output**: Model-generated text responses per turn in the multi-turn dialogue.

## Scoring recipe

```python
scores = []
for trace in dataset:
    responses = model.generate(trace.prompts, T=0.7, top_p=0.9, max_tokens=1024)
    judge_score = judge.evaluate(responses, rubrics) # GPT-4o, T=0.0
    scores.append(judge_score)
return mean(scores)
```

## Common pitfalls

- Using adult-centric benchmarks fails to capture developmental vulnerabilities like over-reliance on AI or misunderstanding figurative language.
- Inference temperature significantly impacts safety behavior; non-zero T (0.7) may increase refusal rates or unsafe outputs compared to T=0.
- LLM-as-Judge determinism requires strict temperature control (T=0.0) to avoid scoring variance across runs.

## Evidence (verbatim from paper)

> The LLM-as-Judge ($\mathcal{J}$) utilized for the Semantic Scoring (Section 3.3) was instantiated using GPT-4o, configured with temperature $T\=0.0$ to maximize determinism in scoring. ... The complete dataset of 1,200 annotated conversation traces.

## Citation

```bibtex
@misc{murali2025childsafe,
  title={Evaluating LLM Safety Across Child Development Stages: A Simulated Agent Approach},
  author={Murali et al. (2025)},
  year={2025},
  note={arXiv:2510.05484}
}
```

- arXiv: 2510.05484