Mmii Medical Sonification Eval

---
name: mmii-medical-sonification-eval
description: Evaluates whether physically informed audiovisual feedback improves spatial perception and task performance in medical imaging. Specifically, it measures how well users learn auditory-visual anatomical mappings and their accuracy in localizing brain tumors within a VR environment compared to unimodal baselines. Use when the user wants to benchmark on Medical imaging volumes (unspecified), or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2407.07015
  bibtex_key: schutz2024mmii
  confidence: medium
---

# mmii-medical-sonification-eval

> A Framework for Multimodal Medical Image Interaction — Schütz et al. (2024) (arXiv:2407.07015, 2024)

## What this evaluates

Evaluates whether physically informed audiovisual feedback improves spatial perception and task performance in medical imaging. Specifically, it measures how well users learn auditory-visual anatomical mappings and their accuracy in localizing brain tumors within a VR environment compared to unimodal baselines.

## Datasets

- **Medical imaging volumes (unspecified)** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Proportion of correctly localized brain tumors in the VR environment, measured against ground-truth surgical annotations or radiologist labels.
- `usability` — range: other
  - Aggregate score from post-task questionnaires assessing perceived ease of use, cognitive load, and system acceptability during multimodal interaction.

## Input / output format

**Input**: 3D medical image volumes rendered in VR, paired with real-time physically modeled auditory feedback that dynamically correlates with tissue properties and spatial position.

**Output**: User selections via VR controllers indicating tumor location or anatomical correspondence, followed by Likert-scale questionnaire responses.

## Scoring recipe

```python
def calc_accuracy(predictions, ground_truth):
    correct = 0
    for pred, gt in zip(predictions, ground_truth):
        if distance(pred.center, gt.center) < spatial_threshold:
            correct += 1
    return (correct / len(predictions)) * 100

def calc_usability(questionnaire_responses):
    return sum(questionnaire_responses) / len(questionnaire_responses)
```

## Common pitfalls

- Audiovisual correspondence is learned rather than innate, requiring careful baseline comparison with unimodal controls to isolate framework benefits.
- VR hardware variability and individual motion sensitivity can confound usability metrics if not standardized across participants.
- Small clinical sample sizes typical in medical VR studies may limit statistical power despite reported significance levels.

## Evidence (verbatim from paper)

> Study 2 focused on the usability and accuracy of the framework for a medical localization task, more precisely, brain tumor localization. The two studies sought to answer the following research questions: - Can users learn the audiovisual correspondence of the auditory and visual representation of anatomy? (Study 1) - Is physical modeling synthesis a suitable sonification approach to create distinguishable auditory representations of anatomy? (Study 1) - Can audiovisual interaction improve the usability and accuracy of a medical localization task? (Study 2)

## Citation

```bibtex
@misc{schutz2024mmii,
  title={A Framework for Multimodal Medical Image Interaction},
  author={Schütz et al. (2024)},
  year={2024},
  note={arXiv:2407.07015}
}
```

- arXiv: 2407.07015