Ncrs Body Brain Coevolution Eval

---
name: ncrs-body-brain-coevolution-eval
description: This evaluation probes the ability of a unified neural cellular automaton substrate to simultaneously evolve robot morphology and control policies for navigation and manipulation tasks. It measures how well decentralized, local interactions enable robots to chase light, navigate obstacles, and carry objects in custom simulation environments. Use when the user wants to benchmark on NCRS Custom Simulation Environments (LC, LCO, CBT), or asks about evaluating this task. Reports fitness score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2203.12066
  bibtex_key: pontesfilho2022unified
  confidence: high
---

# ncrs-body-brain-coevolution-eval

> A Unified Substrate for Body-Brain Co-evolution — Pontes-Filho et al. (2022) (arXiv:2203.12066, 2022)

## What this evaluates

This evaluation probes the ability of a unified neural cellular automaton substrate to simultaneously evolve robot morphology and control policies for navigation and manipulation tasks. It measures how well decentralized, local interactions enable robots to chase light, navigate obstacles, and carry objects in custom simulation environments.

## Datasets

- **NCRS Custom Simulation Environments (LC, LCO, CBT)** — total ?; splits: test (-1); repo https://github.com/sidneyp/neural-cellular-robot-substrate

## Metrics

- `fitness score` **(primary)** — range: [0, 1]
  - Custom reward function from the simulation environment that aggregates task-specific objectives (e.g., distance to target, ball proximity, movement efficiency). Higher values indicate better performance.
- `QD-score` — range: [0, 1]
  - Quality-Diversity metric that measures the volume of the behavior space covered by the evolved population, weighted by the fitness of each behavior configuration.
- `success rate` — range: percent
  - Percentage of successful task completions averaged over 100 independent simulation runs per model.

## Input / output format

**Input**: Initial seed state for the neural cellular automaton (NCA), sensor inputs (e.g., light position), and environment configuration (grid size, obstacles, target/ball positions).

**Output**: Per time-step: NCA cell state updates during the developmental phase, followed by actuator/control signals during the operational phase.

## Scoring recipe

```python
def compute_fitness(env, agent):
    return env.get_reward() # Custom reward function

def compute_qd_score(elites, feature_extractor, fitness_func):
    behaviors = [feature_extractor(e) for e in elites]
    return quality_diversity_metric(behaviors, [fitness_func(e) for e in elites])

def compute_success_rate(agent, env, num_runs=100):
    successes = sum(1 for _ in range(num_runs) if env.run_and_check_success(agent))
    return (successes / num_runs) * 100
```

## Common pitfalls

- Higher fitness scores do not necessarily correlate with higher success rates in the simulation (e.g., CMA-ME achieved higher fitness but lower success rate in LC).
- Success rate is highly sensitive to the specific definition of task completion (e.g., passing the center of the body through an obstacle passage vs. just reaching the target).
- QD-score values are only comparable if the same feature space, emitter configuration, and elite selection criteria are used.

## Evidence (verbatim from paper)

> The results indicate that CMA-ES and CMA-ME can reach almost the same fitness scores after training. However, CMA-ME has fewer generations for the 15 emitters (4,000 generations per emitter).

## Citation

```bibtex
@misc{pontesfilho2022unified,
  title={A Unified Substrate for Body-Brain Co-evolution},
  author={Pontes-Filho et al. (2022)},
  year={2022},
  note={arXiv:2203.12066}
}
```

- arXiv: 2203.12066