Sheriff Eval

---
name: sheriff-eval
description: Evaluates EFCE solvers on a parametric sequential bargaining game modeling smuggling and inspection. It probes the solver's ability to handle multi-round negotiations, bribery, and deterrence to maximize social welfare. Use when the user wants to benchmark on Sheriff, or asks about evaluating this task. Reports Social Welfare (SW).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1905.12564
  bibtex_key: farina2019correlation
  confidence: high
---

# sheriff-eval

> Correlation in Extensive-Form Games: Saddle-Point Formulation and Benchmarks — Farina et al. (2019) (NeurIPS 2019, 2019)

## What this evaluates

Evaluates EFCE solvers on a parametric sequential bargaining game modeling smuggling and inspection. It probes the solver's ability to handle multi-round negotiations, bribery, and deterrence to maximize social welfare.

## Datasets

- **Sheriff** — total ?; splits: test (-1); repo https://github.com/Sandholm-Lab/efce-subgradient

## Metrics

- `Social Welfare (SW)` **(primary)** — range: other
  - Sum of expected utilities to the Smuggler and Sheriff under the computed correlation plan.
- `Expected Payoff` — range: other
  - Individual expected utility for the Smuggler (value of smuggled items minus bribes/fines) and Sheriff (bribes received minus compensation costs).

## Input / output format

**Input**: Game parameters: maximum illegal items n_max, maximum bribe b_max, bargaining rounds r, item value v, penalty p, and compensation s.

**Output**: Correlation plan specifying probability distributions over cargo loading, bribe proposals, inspection decisions, and punishment recommendations upon deviation.

## Scoring recipe

```python
def compute_sw(correlation_plan, game_params):
    smuggler_util, sheriff_util = simulate_game(correlation_plan, game_params)
    return smuggler_util + sheriff_util  # Social Welfare
# Verify incentive compatibility for both players across all bargaining rounds.
```

## Common pitfalls

- Assuming increasing cargo capacity (n_max) always increases social welfare; it can decrease SW due to strategic inspection responses.
- Ignoring the role of non-consequential early bargaining rounds, which serve as passcode verification to enforce compliance.

## Evidence (verbatim from paper)

> The game models the interaction of two players: the Smuggler—who is trying to smuggle illegal items in their cargo—and the Sheriff—who is trying to stop the Smuggler. ... The most striking observation is that increasing the capacity of the cargo n_max may decrease social welfare.

## Citation

```bibtex
@misc{farina2019correlation,
  title={Correlation in Extensive-Form Games: Saddle-Point Formulation and Benchmarks},
  author={Farina et al. (2019)},
  year={2019},
  note={NeurIPS 2019}
}
```

- arXiv: 1905.12564