Pseldnets Seld Eval

---
name: pseldnets-seld-eval
description: Evaluates sound event localization and detection (SELD) performance on synthetic and real-world audio, measuring classification accuracy, localization precision, and overall detection quality across different network architectures and fine-tuning strategies. Use when the user wants to benchmark on synthetic-test-set, synthetic-training-set, Indoor Recordings, or asks about evaluating this task. Reports SELD.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2411.06399
  bibtex_key: hu2024pseldnets
  confidence: medium
---

# pseldnets-seld-eval

> PSELDNets: Pre-trained Neural Networks on a Large-scale Synthetic Dataset for Sound Event Localization and Detection — Jinbo Hu et al. (arXiv:2411.06399, 2024)

## What this evaluates

Evaluates sound event localization and detection (SELD) performance on synthetic and real-world audio, measuring classification accuracy, localization precision, and overall detection quality across different network architectures and fine-tuning strategies.

## Datasets

- **synthetic-test-set** — total ?; splits: test (-1)
- **synthetic-training-set** — total ?; splits: train (-1)
- **Indoor Recordings** — total ?; splits: test (-1)

## Metrics

- `ER20°` — range: degree
  - Classification Error Rate at a 20° angular threshold. Lower values indicate better performance. Computed by comparing predicted and ground-truth DOA angles against a 20° tolerance.
- `F20°` — range: percent
  - F-score at a 20° angular threshold. Higher values indicate better performance. Balances precision and recall of correctly localized sound events within the 20° tolerance.
- `LECD` — range: degree
  - Localization Error Classification Distance. Lower values indicate better localization accuracy. Measures the average angular distance for correctly classified events.
- `LRCD` — range: [0, 1]
  - Localization Recall Classification Distance. Higher values indicate better recall of localized events. Computed as 1 minus the normalized localization error for detected events.
- `SELD` **(primary)** — range: [0, 1]
  - Composite SELD score. Lower values indicate better overall detection and localization performance. Aggregates classification and localization errors into a single metric following standard SELD evaluation protocols.

## Input / output format

**Input**: Multi-channel or monophonic audio recordings (synthetic or real-world indoor recordings)

**Output**: SELD representations: ACCDOA, mACCCDOA, or EINV2 formats encoding sound class, onset/offset times, and azimuth/elevation coordinates

## Scoring recipe

```python
def compute_seld_metrics(preds, gold):
    # preds, gold: list of dicts with 'class', 'azimuth', 'elevation', 'onset', 'offset'
    # 1. Match predictions to ground truth within 20° angular threshold
    matches = []
    for p in preds:
        best_dist = 999
        for g in gold:
            dist = angular_distance(p['azimuth'], p['elevation'], g['azimuth'], g['elevation'])
            if dist < best_dist:
                best_dist = dist
        matches.append(best_dist)
    # 2. Compute ER20° and F20°
    tp = sum(1 for d in matches if d <= 20)
    er20 = 1 - (tp / max(len(gold), 1))
    f20 = tp / max(len(preds) + len(gold) - tp, 1)
    # 3. Compute LECD, LRCD, SELD (simplified aggregation)
    lecd = sum(d for d in matches if d <= 20) / max(tp, 1)
    lrcd = 1 - (lecd / 90)
    seld = (er20 + (1 - f20) + (1 - lrcd)) / 3
    return {'ER20°': er20, 'F20°': f20, 'LECD': lecd, 'LRCD': lrcd, 'SELD': seld}
```

## Common pitfalls

- Threshold dependency: ER20° and F20° strictly use a 20° angular threshold, which may not reflect performance at other tolerances or in highly overlapping scenarios.
- Representation format mismatch: Comparing results across ACCDOA, mACCCDOA, and EINV2 requires careful handling of their distinct coordinate and classification encodings.
- Low-resource adaptation: AdapterBit and fine-tuning strategies may suffer from catastrophic interference if not properly constrained, affecting generalization to unseen rooms.

## Evidence (verbatim from paper)

> Firstly, the performance of PSELDNets is evaluated on synthetic-test-set, investigating various networks and SELD output formats. ... TABLE II: Results of various networks with the mACCDOA representations. ... ER20°↓ F20°↑ LECD↓ LRCD↑ SELD↓

## Citation

```bibtex
@misc{hu2024pseldnets,
  title={PSELDNets: Pre-trained Neural Networks on a Large-scale Synthetic Dataset for Sound Event Localization and Detection},
  author={Jinbo Hu et al.},
  year={2024},
  note={arXiv:2411.06399}
}
```

- arXiv: 2411.06399