Topic Trend Detection Eval

---
name: topic-trend-detection-eval
description: Measures the change in sentiment trend towards a specific topic over time or across datasets, requiring temporal or comparative analysis. Use when the user wants to benchmark on Twitter2015-test, or asks about evaluating this task. Reports avgDiff.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1912.02387
  bibtex_key: rosenthal2015semeval
  confidence: high
---

# topic-trend-detection-eval

> SemEval-2015 Task 10: Sentiment Analysis in Twitter — Rosenthal et al. (2015) (SemEval-2015 / arXiv:1912.02387, 2015)

## What this evaluates

Measures the change in sentiment trend towards a specific topic over time or across datasets, requiring temporal or comparative analysis.

## Datasets

- **Twitter2015-test** — total ?; splits: test (-1)

## Metrics

- `avgDiff` **(primary)** — range: other
  - Average difference in sentiment scores across time periods or datasets for a given topic. Systems are ranked by this metric.
- `avgLevelDiff` — range: other
  - Average level difference metric used to rank systems; yields identical rankings to avgDiff.

## Input / output format

**Input**: Tweets associated with a topic, ordered or grouped by time/context.

**Output**: Trend score or direction for the topic.

## Scoring recipe

```python
# avgDiff and avgLevelDiff are provided by the task organizers
# Systems are ranked by avgDiff
rank = sort_by(avgDiff, descending)
# avgLevelDiff yields the same ranking
```

## Common pitfalls

- Only three teams improved over the baseline, indicating high difficulty.
- Trend detection is closely related to topic-level polarity; solving C is a prerequisite for D, and rankings differ from C.

## Evidence (verbatim from paper)

> The results for subtask D are shown in Table 13. ... Finally, note that avgDiff and avgLevelDiff yielded the same rankings.

## Citation

```bibtex
@misc{rosenthal2015semeval,
  title={SemEval-2015 Task 10: Sentiment Analysis in Twitter},
  author={Rosenthal et al. (2015)},
  year={2015},
  note={SemEval-2015 / arXiv:1912.02387}
}
```

- arXiv: 1912.02387