Wds Pareto Front Eval

---
name: wds-pareto-front-eval
description: Evaluates the quality and diversity of multi-objective optimization algorithms for water distribution system design by comparing generated Pareto fronts against established benchmark fronts. It measures coverage of known solutions, discovery of novel non-dominated designs, and computational efficiency. Use when the user wants to benchmark on HAN, NYT, BLA, and GOY networks, or asks about evaluating this task. Reports N_A^u, N_B^u, N_A^a, N_B^a, N_c, N_{FE}.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1903.06127
  bibtex_key: patil2019water
  confidence: high
---

# wds-pareto-front-eval

> Water Distribution System Design Using Multi-Objective Particle Swarm Optimisation — Patil et al. (2019) (arXiv:1903.06127, 2019)

## What this evaluates

Evaluates the quality and diversity of multi-objective optimization algorithms for water distribution system design by comparing generated Pareto fronts against established benchmark fronts. It measures coverage of known solutions, discovery of novel non-dominated designs, and computational efficiency.

## Datasets

- **HAN, NYT, BLA, and GOY networks** — total ?; splits: test (-1)

## Metrics

- `N_A^u, N_B^u, N_A^a, N_B^a, N_c, N_{FE}` **(primary)** — range: count
  - Counts comparing two Pareto fronts (PF-A and PF-B): N_A^u (solutions in PF-A not covered by PF-B), N_B^u (new solutions in PF-B not in PF-A), N_A^a/N_B^a (covered solutions), N_c (common solutions), and N_{FE} (total function evaluations). Solutions are compared for dominance and exact coverage in objective space.

## Input / output format

**Input**: Water distribution network topology, pipe/nodes data, demand requirements, and cost/resilience objective functions.

**Output**: A set of non-dominated design configurations (e.g., pipe diameters, pump settings) representing the Pareto front.

## Scoring recipe

```python
def score_pf(pf_a, pf_b, n_fe):
    covered_a, uncovered_a, dominated_a = 0, 0, 0
    covered_b, uncovered_b, dominated_b = 0, 0, 0
    common = 0
    for sol_a in pf_a:
        if any(sol_b == sol_a for sol_b in pf_b):
            covered_a += 1; common += 1
        elif any(sol_b dominates sol_a for sol_b in pf_b):
            dominated_a += 1
        else:
            uncovered_a += 1
    # Repeat for pf_b against pf_a
    return {'N_A^u': uncovered_a, 'N_B^u': uncovered_b, 'N_A^a': covered_a, 'N_B^a': covered_b, 'N_c': common, 'N_{FE}': n_fe}
```

## Common pitfalls

- Evaluating only single-objective metrics like average or minimum cost, which ignores trade-offs and diversity in the Pareto front.
- Focusing solely on objective-space proximity while ignoring decision-space diversity, as solutions can be nearly identical in cost/resilience but vastly different in physical design.
- Comparing front quality without normalizing for computational budget (function evaluations), leading to unfair advantages for more expensive runs.

## Evidence (verbatim from paper)

> Table 3 shows the results (i.e., $N_A^t$, $N_A^u$, etc. as defined in Sec. 2) for the HAN network... In each case, a substantial number of new ND solutions ($N_B^u$) have been found by MOPSO+. Furthermore, almost all ND solutions in the UExeter PFs are covered by MOPSO+ (see the $N_A^u$ column in the table). The metrics proposed in Sec. 2 therefore seem to be more attractive from a practical perspective than metrics such as average cost, lowest cost, standard deviation, or spread.

## Citation

```bibtex
@misc{patil2019water,
  title={Water Distribution System Design Using Multi-Objective Particle Swarm Optimisation},
  author={Patil et al. (2019)},
  year={2019},
  note={arXiv:1903.06127}
}
```

- arXiv: 1903.06127