Criticality Detection Accuracy

---
name: criticality-detection-accuracy
description: Evaluates the ability to detect the onset of systemic instability (criticality) in simulated AI systems by monitoring performance variance across multiple benchmarks. It probes whether a derivative-based threshold can reliably flag phase transitions before functional collapse. Use when the user has predictions and gold and needs to compute percentage of correct classifications.
metadata:
  skill_kind: metric
  source_arxiv: 2407.03652
  bibtex_key: teosusnjak2024over
  confidence: high
---

# criticality-detection-accuracy

> Over the Edge of Chaos? Excess Complexity as a Roadblock to Artificial General Intelligence — Teo Susnjak et al. (2024) (arXiv:2407.03652, 2024)

## What this evaluates

Evaluates the ability to detect the onset of systemic instability (criticality) in simulated AI systems by monitoring performance variance across multiple benchmarks. It probes whether a derivative-based threshold can reliably flag phase transitions before functional collapse.

## Datasets

- **Simulated Criticality Benchmarks** — total ?; splits: train (-1), test (-1); repo https://github.com/teosusnjak/AGI-and-criticality

## Metrics

- `percentage of correct classifications` **(primary)** — range: percent
  - Calculated as the number of simulations where criticality is detected within 10 time steps post-actual criticality, divided by the total number of simulations, expressed as a percentage.

## Input / output format

**Input**: Time-series performance trajectories of a simulated AI system across N benchmarks (N ∈ {2, 5, 10, 20}), along with the ground-truth criticality time step.

**Output**: Binary classification: 'detected' if the derivative threshold is crossed within the 10-step post-criticality window, otherwise 'not detected'.

## Scoring recipe

```python
def score(predictions, gold):
    correct = 0
    for pred_time, crit_time in zip(predictions, gold):
        if crit_time <= pred_time <= crit_time + 10:
            correct += 1
    return (correct / len(predictions)) * 100
```

## Common pitfalls

- Detection reliability heavily depends on the number of benchmarks used; fewer benchmarks yield high variance and poor detection.
- The 10-step detection window is an arbitrary experimental choice and may not generalize to real-world continuous monitoring.
- The metric relies on simulated performance data rather than live model weights or gradients, limiting direct applicability to production LLMs.

## Evidence (verbatim from paper)

> For the purposes of these experiments, we selected any detection that falls immediately within 10-time steps post-criticality to be a correct positive detection. Based on this criterion, Table [2] shows the percentage of correct classifications that were achieved on both the training and test datasets across all benchmark sizes.

## Citation

```bibtex
@misc{teosusnjak2024over,
  title={Over the Edge of Chaos? Excess Complexity as a Roadblock to Artificial General Intelligence},
  author={Teo Susnjak et al. (2024)},
  year={2024},
  note={arXiv:2407.03652}
}
```

- arXiv: 2407.03652