Feynman Sr Eval

---
name: feynman-sr-eval
description: Evaluates the ability of genetic programming systems to discover exact symbolic mathematical expressions from numerical data points. It probes search efficiency, robustness to domain constraints (e.g., NaNs), and the impact of different fitness functions on expression discovery. Use when the user wants to benchmark on Feynman dataset, or asks about evaluating this task. Reports typically_solved.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.12292
  bibtex_key: haut2026beagle
  confidence: high
---

# feynman-sr-eval

> GPU-Accelerated Genetic Programming for Symbolic Regression with Beagle Framework — Haut et al. (2026) (arXiv:2603.12292, 2026)

## What this evaluates

Evaluates the ability of genetic programming systems to discover exact symbolic mathematical expressions from numerical data points. It probes search efficiency, robustness to domain constraints (e.g., NaNs), and the impact of different fitness functions on expression discovery.

## Datasets

- **Feynman dataset** — total 100; splits: test (100); repo https://github.com/Noblis/beagle-v1.x

## Metrics

- `typically_solved` **(primary)** — range: count [0, 100]
  - Count of problems (out of 100) where 50% or more of the 10 independent runs found a validated solution.
- `best_solve_count` — range: count [0, 100]
  - Count of problems (out of 100) where at least one of the 10 independent runs found a validated solution.

## Input / output format

**Input**: Numerical data points (x, y) for symbolic regression problems from the Feynman dataset.

**Output**: Symbolic mathematical expression representing the underlying function.

## Scoring recipe

```python
typical_solved = 0
best_solved = 0
for problem in problems:
    validated_runs = 0
    for run in range(10):
        expr = run_gp(problem.data, time_limit)
        if validate(expr, problem.gold):
            validated_runs += 1
    if validated_runs >= 5:
        typical_solved += 1
    if validated_runs >= 1:
        best_solved += 1
return typical_solved, best_solved
```

## Common pitfalls

- Runtime constraints (10 vs 30 min) significantly impact solve rates, as systems have not stalled within the shorter limit.
- Fitness function choice drastically affects performance; correlation-based fitness improves typical performance over point-to-point error.
- NaN handling in the fitness function is critical for problems with restricted vs. wide input domains, as standard systems fail when imaginary numbers arise.

## Evidence (verbatim from paper)

> Across the benchmark problems, each GP setup was compared using the total number of problems typically solved (defined as problems where 50% or more of the independent runs found solutions that were validated) and the total number of problems solved at least once (best performance across 10 runs).

## Citation

```bibtex
@misc{haut2026beagle,
  title={GPU-Accelerated Genetic Programming for Symbolic Regression with Beagle Framework},
  author={Haut et al. (2026)},
  year={2026},
  note={arXiv:2603.12292}
}
```

- arXiv: 2603.12292