Semeval2021task6 St2

---
name: semeval2021task6-st2
description: Identifies and classifies spans of persuasion techniques within unimodal text using sequence tagging. It probes the model's ability to localize and categorize persuasive spans at the token level. Use when the user wants to benchmark on SemEval-2021 Task 6 Subtask 2, or asks about evaluating this task. Reports F1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2105.09284
  bibtex_key: dimitrov2021semevaltask6
  confidence: high
---

# semeval2021task6-st2

> SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images — Dimitrov et al. (2021) (arXiv:2105.09284, 2021)

## What this evaluates

Identifies and classifies spans of persuasion techniques within unimodal text using sequence tagging. It probes the model's ability to localize and categorize persuasive spans at the token level.

## Datasets

- **SemEval-2021 Task 6 Subtask 2** — total ?; splits: test (-1)

## Metrics

- `F1` **(primary)** — range: [0, 1]
  - Micro-averaged F1 score for sequence tagging of technique spans.

## Input / output format

**Input**: Raw text snippet.

**Output**: Sequence of token-level labels indicating the start and end of persuasion technique spans.

## Scoring recipe

```python
tp = fp = fn = 0
for p, g in zip(predictions, golds):
    if p == g: tp += 1
    elif p in golds: fp += 1
    elif g in predictions: fn += 1
prec = tp / (tp + fp) if (tp + fp) > 0 else 0
rec = tp / (tp + fn) if (tp + fn) > 0 else 0
return 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
```

## Common pitfalls

- Handling BPE subword boundaries correctly is crucial; classes should be assigned as the union of subword predictions.
- Data sparseness and imbalance require careful loss design or augmentation.

## Evidence (verbatim from paper)

> Table 5 shows the evaluation results. We report our random baseline, which is based on the random selection of spans with random lengths and a random assignment of labels... Table 5: Results for Subtask 2. The systems are ordered by the official score: F1-micro.

## Citation

```bibtex
@misc{dimitrov2021semevaltask6,
  title={SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images},
  author={Dimitrov et al. (2021)},
  year={2021},
  note={arXiv:2105.09284}
}
```

- arXiv: 2105.09284