Zero Shot Human Classification Eval

---
name: zero-shot-human-classification-eval
description: Evaluates the zero-shot transfer capability of a vision-language model on human-centric classification tasks, including activity recognition, age grouping, and emotion recognition, using pose-grounded text descriptions and subject-focused attention. Use when the user wants to benchmark on Stanford40, Emotic, LAGENDA-Body, LAGENDA-Face, UTKFace, FER+, or asks about evaluating this task. Reports top-k accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2403.06904
  bibtex_key: khan2024humanposedescriptions
  confidence: high
---

# zero-shot-human-classification-eval

> Human Pose Descriptions and Subject-Focused Attention for Improved Zero-Shot Transfer in Human-Centric Classification Tasks — Muhammad Saif Ullah Khan et al. (2024) (arXiv:2403.06904, 2024)

## What this evaluates

Evaluates the zero-shot transfer capability of a vision-language model on human-centric classification tasks, including activity recognition, age grouping, and emotion recognition, using pose-grounded text descriptions and subject-focused attention.

## Datasets

- **Stanford40** — total ?; splits: test (-1)
- **Emotic** — total ?; splits: test (-1)
- **LAGENDA-Body** — total ?; splits: test (-1)
- **LAGENDA-Face** — total ?; splits: test (-1)
- **UTKFace** — total ?; splits: test (-1)
- **FER+** — total ?; splits: test (-1)

## Metrics

- `top-k accuracy` **(primary)** — range: [0, 1]
  - Measures whether the ground-truth class label appears within the top k predicted classes by the model. k is set to 1 for age and emotion tasks, and 3 for activity recognition.

## Input / output format

**Input**: RGB image paired with a natural language pose description (text prompt).

**Output**: Predicted class label (activity, age group, or emotion) ranked by confidence score.

## Scoring recipe

```python
def top_k_accuracy(predictions, labels, k):
    correct = 0
    for pred, label in zip(predictions, labels):
        top_k_preds = pred.argsort()[-k:][::-1]
        if label in top_k_preds:
            correct += 1
    return correct / len(labels)
```

## Common pitfalls

- Age labels are originally numerical but must be explicitly binned into categorical groups (adult, teenager, kid) before evaluation.
- The evaluation is strictly zero-shot: models are trained only on the source dataset (MPII) and tested on unseen target datasets without fine-tuning.
- Different k values are used per task category (k=1 for age/emotion, k=3 for activity), so results are not directly comparable without normalization.

## Evidence (verbatim from paper)

> Tab. 2 presents our quantitative results. We report the top-k accuracy for image-based activity classification, age classification, and emotion recognition.

## Citation

```bibtex
@misc{khan2024humanposedescriptions,
  title={Human Pose Descriptions and Subject-Focused Attention for Improved Zero-Shot Transfer in Human-Centric Classification Tasks},
  author={Muhammad Saif Ullah Khan et al. (2024)},
  year={2024},
  note={arXiv:2403.06904}
}
```

- arXiv: 2403.06904