Tt Df Eval

---
name: tt-df-eval
description: Evaluates the ability of models to detect human body forgeries generated by diffusion models. It probes spatiotemporal motion inconsistencies and generalization across different generation configurations and unseen manipulation models. Use when the user wants to benchmark on TT-DF, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.08437
  bibtex_key: yang2025tt-df
  confidence: high
---

# tt-df-eval

> TT-DF: A Large-Scale Diffusion-Based Dataset and Benchmark for Human Body Forgery Detection — Yang et al. (2025) (arXiv:2505.08437, 2025)

## What this evaluates

Evaluates the ability of models to detect human body forgeries generated by diffusion models. It probes spatiotemporal motion inconsistencies and generalization across different generation configurations and unseen manipulation models.

## Datasets

- **TT-DF** — total ?; splits: train (240), val (50), test (50); repo https://github.com/HashTAG00002/TT-DF

## Metrics

- `AUC` **(primary)** — range: percent
  - Area Under the Receiver Operating Characteristic Curve. Measures the probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance based on predicted forgery scores.
- `Accuracy` — range: percent
  - Proportion of correctly classified video clips (forged vs. real) out of the total number of test clips.

## Input / output format

**Input**: Video clips consisting of 8 consecutive frames, center-cropped and scaled to 512×512 resolution. Frames are sampled pseudo-randomly during training and uniformly during evaluation.

**Output**: Binary classification prediction (forged vs. real) per video clip, typically averaged frame-wise for image-based baselines.

## Scoring recipe

```python
def compute_metrics(predictions, labels):
    # predictions: array of forgery probabilities per clip
    # labels: ground truth binary labels (0: real, 1: forged)
    acc = np.mean((predictions > 0.5).astype(int) == labels) * 100
    auc = roc_auc_score(labels, predictions) * 100
    return {'AUC': auc, 'Accuracy': acc}
```

## Common pitfalls

- Image-based baselines (Xception, BAR-Net) are adapted to video by averaging frame-wise predictions, which may fail to capture temporal inconsistencies compared to native video models.
- Test videos are clipped to fewer than 30 frames to mitigate metric fluctuations, potentially limiting the temporal context available for detection.
- Compression levels (CRF 23 vs 40) significantly affect performance, with lower-quality videos sometimes yielding better generalization due to focus on coarse motion artifacts.

## Evidence (verbatim from paper)

> For evaluation, we utilize Accuracy (Acc) and Area Under the Receiver Operating Characteristic Curve (AUC) metrics, consistent with most prior research on facial forgery detection.

## Citation

```bibtex
@misc{yang2025tt-df,
  title={TT-DF: A Large-Scale Diffusion-Based Dataset and Benchmark for Human Body Forgery Detection},
  author={Yang et al. (2025)},
  year={2025},
  note={arXiv:2505.08437}
}
```

- arXiv: 2505.08437