M Absa Eval

---
name: m-absa-eval
description: Evaluates multilingual aspect-based sentiment analysis (ABSA) models on triplet extraction (aspect term, category, sentiment) and pairwise extraction (aspect term, sentiment) across 21 languages and 7 domains. Probes cross-lingual transfer, cross-domain adaptation, and zero-shot LLM prompting capabilities. Use when the user wants to benchmark on M-ABSA, or asks about evaluating this task. Reports Micro-F1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2502.11824
  bibtex_key: wu2025mabsa
  confidence: high
---

# m-absa-eval

> M-ABSA: A Multilingual Dataset for Aspect-Based Sentiment Analysis — Wu et al. (2025) (arXiv:2502.11824, 2025)

## What this evaluates

Evaluates multilingual aspect-based sentiment analysis (ABSA) models on triplet extraction (aspect term, category, sentiment) and pairwise extraction (aspect term, sentiment) across 21 languages and 7 domains. Probes cross-lingual transfer, cross-domain adaptation, and zero-shot LLM prompting capabilities.

## Datasets

- **M-ABSA** — total ?; splits: train (-1), test (-1); repo https://github.com/swaggy66/M-ABSA

## Metrics

- `Micro-F1` **(primary)** — range: percent
  - Exact match F1: a prediction is correct if and only if all predicted sentiment elements in the pair or triplet match the gold standard. Reported as the average over 5 random seeds.

## Input / output format

**Input**: Text reviews in 21 languages across 7 domains (Coursera, Food, Hotel, Laptop, Phone, Restaurant, Sightseeing). For LLM zero-shot evaluation, structured prompts are used to extract aspect terms, categories, and sentiment polarities.

**Output**: Extracted tuples: (aspect term, sentiment polarity) for UABSA, or (aspect term, aspect category, sentiment polarity) for TASD.

## Scoring recipe

```python
correct = 0
total = 0
for pred, gold in zip(predictions, gold_labels):
    if set(pred) == set(gold):
        correct += 1
    total += 1
return (correct / total) * 100
```

## Common pitfalls

- Confusing TASD (triplet extraction) with UABSA (pairwise extraction); TASD is significantly harder and yields lower scores across all models.
- Assuming cross-lingual transfer performance scales linearly with resource availability; typological similarity (e.g., West Germanic) often matters more than raw resource count.
- Not averaging over 5 random seeds, which the authors explicitly state is required for reporting results.

## Evidence (verbatim from paper)

> We adopt Micro-F1 scores as the main evaluation metrics for all tasks. A prediction is correct if and only if all its predicted sentiment elements in the pair or triplet are correct. All the experimental results are reported using the average of 5 random seeds.

## Citation

```bibtex
@misc{wu2025mabsa,
  title={M-ABSA: A Multilingual Dataset for Aspect-Based Sentiment Analysis},
  author={Wu et al. (2025)},
  year={2025},
  note={arXiv:2502.11824}
}
```

- arXiv: 2502.11824