Ludii Rbg Eval

---
name: ludii-rbg-eval
description: Evaluates the computational efficiency and human-readability of two General Game Playing systems (Ludii and RBG) by measuring their playout throughput and the token count required to define game rules. Use when the user wants to benchmark on Unspecified game rule suite, or asks about evaluating this task. Reports playouts.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1907.00244
  bibtex_key: piette2019ludii
  confidence: high
---

# ludii-rbg-eval

> An Empirical Evaluation of Two General Game Systems: Ludii and RBG — Piette et al. (2019) (arXiv:1907.00244, 2019)

## What this evaluates

Evaluates the computational efficiency and human-readability of two General Game Playing systems (Ludii and RBG) by measuring their playout throughput and the token count required to define game rules.

## Datasets

- **Unspecified game rule suite** — total ?; splits: (unstated)

## Metrics

- `playouts` **(primary)** — range: other
  - Number of flat Monte Carlo random playouts executed per second by the system's reasoning engine.
- `token_count` — range: other
  - Total number of tokens required to define a game's rules in the system's language.

## Input / output format

**Input**: Game rule definitions formatted in either Ludii's ludeme-based syntax or RBG's two-tiered language.

**Output**: System-reported token count for the definition, and the total number of random playouts completed within a fixed 10-minute runtime.

## Scoring recipe

```python
def evaluate_system(game_rules, system, time_limit_sec=600):
    token_count = count_tokens(system.parse(game_rules))
    start_time = time.time()
    playout_count = 0
    while time.time() - start_time < time_limit_sec:
        playout_count += system.run_flat_mcts_playout()
    playouts_per_sec = playout_count / (time.time() - start_time)
    return playouts_per_sec, token_count
```

## Common pitfalls

- RBG has both an interpreter and a compiler; the paper compares Ludii to both, so results must be clearly attributed to the correct RBG mode.
- Token counting methodology is not formally defined in the excerpt, making cross-system readability comparisons sensitive to tokenizer choices.
- The 10-minute fixed runtime may not allow MCTS to converge for complex games, so playout counts reflect raw engine speed rather than search quality.

## Evidence (verbatim from paper)

> Hence, we use flat Monte Carlo playouts as the metric for comparing the efficiency between Ludii and RBG. In order to make a comparison between the human-readability of the two systems we use two main criteria: clarity and simplicity. Clarity refers to the degree to which game descriptions would be self-explanatory to non-specialist readers, and simplicity refers to the ease with which game descriptions can be created and modified, and can be estimated by the number of tokens required to define games.

## Citation

```bibtex
@misc{piette2019ludii,
  title={An Empirical Evaluation of Two General Game Systems: Ludii and RBG},
  author={Piette et al. (2019)},
  year={2019},
  note={arXiv:1907.00244}
}
```

- arXiv: 1907.00244