Omnifall Eval

---
name: omnifall-eval
description: Evaluates human fall detection and action recognition capabilities across controlled (staged) and uncontrolled (wild) video domains. It probes a model's ability to classify a 10-class activity taxonomy, detect binary fall/fallen states, and segment action timelines, while measuring generalization gaps between in-distribution and out-of-distribution settings. Use when the user wants to benchmark on CMDFall, UP-Fall, Le2i, GMDCSA24, EDF, OCCU, CaucaFall, MCFD, OOPS-Fall, or asks about evaluating this task. Reports Balanced Accuracy, Macro F1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.19889
  bibtex_key: [REDACTED]
  confidence: high
---

# omnifall-eval

> OmniFall: A Unified Staged-to-Wild Benchmark for Human Fall Detection — Schneider et al. (2025) (arXiv:2505.19889, 2025)

## What this evaluates

Evaluates human fall detection and action recognition capabilities across controlled (staged) and uncontrolled (wild) video domains. It probes a model's ability to classify a 10-class activity taxonomy, detect binary fall/fallen states, and segment action timelines, while measuring generalization gaps between in-distribution and out-of-distribution settings.

## Datasets

- **CMDFall** — total ?; splits: cross-subject (-1), cross-view (-1)
- **UP-Fall** — total ?; splits: cross-subject (-1), cross-view (-1)
- **Le2i** — total ?; splits: cross-subject (-1), cross-view (-1)
- **GMDCSA24** — total ?; splits: cross-subject (-1), cross-view (-1)
- **EDF** — total ?; splits: cross-subject (-1), cross-view (-1)
- **OCCU** — total ?; splits: cross-subject (-1), cross-view (-1)
- **CaucaFall** — total ?; splits: cross-subject (-1), cross-view (-1)
- **MCFD** — total ?; splits: cross-subject (-1), cross-view (-1)
- **OOPS-Fall** — total ?; splits: cross-subject (-1), cross-view (-1)

## Metrics

- `Balanced Accuracy` **(primary)** — range: [0, 1]
  - The average of sensitivity (recall) and specificity across classes. Computed as (TP/(TP+FN) + TN/(TN+FP)) / 2.
- `Macro F1` **(primary)** — range: [0, 1]
  - The unweighted mean of F1-scores calculated independently for each of the 10 activity classes.
- `Sensitivity` — range: [0, 1]
  - True positive rate for the positive class (e.g., fall or fallen). Calculated as TP / (TP + FN).
- `Specificity` — range: [0, 1]
  - True negative rate for the negative class. Calculated as TN / (TN + FP).
- `F1-score` — range: [0, 1]
  - Harmonic mean of precision and recall for binary fall/fallen detection. Calculated as 2 * TP / (2 * TP + FP + FN).
- `Segmental F1@25` — range: [0, 1]
  - Segmental F1 score computed at an Intersection-over-Union (IoU) threshold of 25% between predicted and ground-truth action segments.
- `Edit Distance` — range: [0, 1]
  - Normalized edit distance measuring the minimum number of insertions, deletions, and substitutions required to transform the predicted label sequence into the ground truth.

## Input / output format

**Input**: Video sequences (raw frames or pre-extracted I3D/VideoMAE features, downsampled to 10 fps for segmentation). Features are processed as temporal sequences (e.g., 18 tokens) with sinusoidal position encodings.

**Output**: Class labels (10-class taxonomy or binary fall/fallen/combined) and temporal segment boundaries for timeline segmentation.

## Scoring recipe

```python
def compute_classification_metrics(y_true, y_pred, classes):
    tp = sum((y_true == c) & (y_pred == c) for c in classes)
    fp = sum((y_true != c) & (y_pred == c) for c in classes)
    fn = sum((y_true == c) & (y_pred != c) for c in classes)
    tn = sum((y_true != c) & (y_pred != c) for c in classes)
    
    sensitivity = tp / (tp + fn) if (tp + fn) > 0 else 0
    specificity = tn / (tn + fp) if (tn + fp) > 0 else 0
    f1 = 2 * tp / (2 * tp + fp + fn) if (2 * tp + fp + fn) > 0 else 0
    balanced_acc = (sensitivity + specificity) / 2
    
    return balanced_acc, f1, sensitivity, specificity

def compute_segmental_f1(pred_segments, gt_segments, iou_thresh=0.25):
    # Match predicted and ground-truth segments by IoU >= threshold
    # Compute precision, recall, and F1 over matched segments
    pass
```

## Common pitfalls

- Confusing cross-subject (CS) and cross-view (CV) evaluation splits, which differ significantly in training corpus size and generalization behavior.
- Overlooking the staged-to-wild domain shift: models often maintain high specificity on OOPS-Fall but suffer drastic sensitivity drops due to overfitting to staged appearance cues.
- Ignoring the two-level weighting strategy (dataset balancing and label-smoothed cross-entropy) required to handle severe class and domain imbalance during training.

## Evidence (verbatim from paper)

> We perform separate training runs for cross-subject and cross-view evaluations, reporting balanced accuracy, overall accuracy, and macro-F1 for the 10-class task, plus sensitivity, specificity and F1-score for binary fall detection subtasks.

## Citation

```bibtex
@misc{schneider2025omnifall,
  title={OmniFall: A Unified Staged-to-Wild Benchmark for Human Fall Detection},
  author={Schneider et al. (2025)},
  year={2025},
  note={arXiv:2505.19889}
}
```

- arXiv: 2505.19889