Groundlie360 Eval

---
name: groundlie360-eval
description: This benchmark evaluates a model's ability to detect and localize multimodal misinformation across text, speech, and video. It probes fine-grained cross-modal reasoning by requiring binary veracity classification, sub-type categorization, and precise grounding of fake content at the token, frame, and bounding-box levels. Use when the user wants to benchmark on GroundLie360, or asks about evaluating this task. Reports F1 score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.08008
  bibtex_key: [REDACTED]
  confidence: high
---

# groundlie360-eval

> A New Dataset and Benchmark for Grounding Multimodal Misinformation — Bingjian Yang et al. (2025) (arXiv:2509.08008, 2025)

## What this evaluates

This benchmark evaluates a model's ability to detect and localize multimodal misinformation across text, speech, and video. It probes fine-grained cross-modal reasoning by requiring binary veracity classification, sub-type categorization, and precise grounding of fake content at the token, frame, and bounding-box levels.

## Datasets

- **GroundLie360** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/yangbingjian/GroundLie360

## Metrics

- `F1 score` **(primary)** — range: [0, 1]
  - Harmonic mean of Precision and Recall: 2 * (Prec * Rec) / (Prec + Rec). Computed per task (binary, subtype, token, frame) and averaged macro/micro as specified.
- `Precision, Recall` — range: [0, 1]
  - Precision: fraction of predicted positives that are correct. Recall: fraction of actual positives correctly identified. Reported for classification and grounding subtasks.
- `m_tIoU, m_vIoU, vIoU@0.3, vIoU@0.5` — range: [0, 1]
  - Mean temporal IoU and mean visual IoU measure overlap between predicted and ground-truth temporal/spatial regions. vIoU@threshold reports the proportion of predictions where IoU exceeds the threshold.

## Input / output format

**Input**: Multimodal news instances containing text, audio/speech, and video. Videos are scene-segmented and uniformly sampled to 16 frames per scene. Text inputs are limited to 1024 tokens. The model receives a question-driven prompt to perform classification and grounding.

**Output**: Per instance: (1) binary veracity label (real/fake), (2) misinformation sub-type label (e.g., false title, temporal edit, CGI, false speech, contradictory, unsupported), (3) token-level text spans, (4) frame indices for temporal grounding, (5) bounding boxes for spatial grounding.

## Scoring recipe

```python
def compute_classification_metrics(preds, golds):
    tp = sum(1 for p, g in zip(preds, golds) if p == g)
    fp = sum(1 for p, g in zip(preds, golds) if p != g)
    fn = sum(1 for p, g in zip(preds, golds) if p != g)
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
    return f1, prec, rec

def compute_grounding_iou(pred_regions, gold_regions):
    ious = []
    for pred, gold in zip(pred_regions, gold_regions):
        inter = intersection_area(pred, gold)
        union = union_area(pred, gold)
        ious.append(inter / union if union > 0 else 0)
    return sum(ious) / len(ious) if ious else 0
```

## Common pitfalls

- Error propagation: Mistakes in upstream binary or sub-type classification stages directly magnify failures in downstream localization.
- Sample imbalance and ambiguity: Uneven distribution across fake types and fragmented spoken language cause significant performance drops for categories like false speech and temporal edits.
- Multi-stage pipeline dependency: Grounding accuracy is tightly coupled with classification confidence, making isolated grounding evaluation misleading without considering upstream errors.

## Evidence (verbatim from paper)

> We use different evaluation metrics depending on the task type: (1) Binary veracity classification: Precision, Recall, and F1 score. (2) Sub-type classification: Macro-averaged Precision, Recall, and F1 score. (3) Grounding tasks:(a) Textual grounding: Token-level Precision, Recall, and F1 score; (b) Video temporal grounding: Frame-level Precision, Recall, and F1 score; (c) Video spatial-temporal grounding: Mean temporal IoU (m_tIoU), mean visual IoU (m_vIoU), and vIoU@0.3 / vIoU@0.5

## Citation

```bibtex
@misc{yang2025groundlie360,
  title={A New Dataset and Benchmark for Grounding Multimodal Misinformation},
  author={Bingjian Yang et al. (2025)},
  year={2025},
  note={arXiv:2509.08008}
}
```

- arXiv: 2509.08008