Rgb Har Eval

---
name: rgb-har-eval
description: Evaluates the ability of a skeleton-based BLSTM model to recognize human actions from RGB-only video streams under limited labeled data conditions, comparing against methods that use depth or inertial modalities. Use when the user wants to benchmark on UTD-MHAD, KTH, or asks about evaluating this task. Reports top-1 accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1812.06544
  bibtex_key: sarker2018robust
  confidence: high
---

# rgb-har-eval

> Towards Robust Human Activity Recognition from RGB Video Stream with Limited Labeled Data — Sarker et al. (2018) (arXiv:1812.06544, 2018)

## What this evaluates

Evaluates the ability of a skeleton-based BLSTM model to recognize human actions from RGB-only video streams under limited labeled data conditions, comparing against methods that use depth or inertial modalities.

## Datasets

- **UTD-MHAD** — total 861; splits: train (-1), test (-1)
- **KTH** — total ?; splits: train (-1), test (-1)

## Metrics

- `top-1 accuracy` **(primary)** — range: percent
  - Percentage of correctly predicted action classes out of the total test samples.
- `F1 score` — range: [0, 1]
  - Harmonic mean of precision and recall across all action classes.

## Input / output format

**Input**: RGB video frames processed into skeleton key-points via OpenPose, fed into a 5-layer Bidirectional LSTM (BLSTM) model.

**Output**: Predicted action class label (classification among 27 classes for UTD-MHAD, 6 classes for KTH).

## Scoring recipe

```python
def compute_metrics(predictions, gold_labels):
    accuracy = sum(p == g for p, g in zip(predictions, gold_labels)) / len(gold_labels)
    f1 = f1_score(gold_labels, predictions, average='macro')
    ci = bootstrap_ci(accuracy, n_bootstrap=50)
    return accuracy, f1, ci
```

## Common pitfalls

- Using depth or inertial modalities instead of strictly RGB-only as specified.
- Failing to follow the 50-50 train-test split from the original UTD-MHAD paper.
- Ignoring the requirement for full subject silhouettes, which degrades pose key-point extraction reliability.

## Evidence (verbatim from paper)

> We follow 50-50 train-test split similar to [[14]]. In the experiments we only use the RGB modality to evaluate our proposed method. Fig. 4 shows the comparison among all these models on accuracy and F1 score. We report confidence interval based on 50 bootstrap trials.

## Citation

```bibtex
@misc{sarker2018robust,
  title={Towards Robust Human Activity Recognition from RGB Video Stream with Limited Labeled Data},
  author={Sarker et al. (2018)},
  year={2018},
  note={arXiv:1812.06544}
}
```

- arXiv: 1812.06544