Eu Taxonomy Kpi Eval

---
name: eu-taxonomy-kpi-eval
description: Evaluates LLMs on extracting and predicting EU Taxonomy-compliant Key Performance Indicators from corporate sustainability reports. It probes two capabilities: multi-label classification of economic activities against regulatory definitions, and zero-shot regression of financial KPI percentages (Turnover, CapEx, OpEx) from unstructured text. Use when the user wants to benchmark on EU Taxonomy Sustainability Reports Dataset, or asks about evaluating this task. Reports multi-label text classification.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.24289
  bibtex_key: schmoll2025automated
  confidence: medium
---

# eu-taxonomy-kpi-eval

> Automated Analysis of Sustainability Reports: Using Large Language Models for the Extraction and Prediction of EU Taxonomy-Compliant KPIs — Schmoll et al. (2025) (arXiv:2512.24289, 2025)

## What this evaluates

Evaluates LLMs on extracting and predicting EU Taxonomy-compliant Key Performance Indicators from corporate sustainability reports. It probes two capabilities: multi-label classification of economic activities against regulatory definitions, and zero-shot regression of financial KPI percentages (Turnover, CapEx, OpEx) from unstructured text.

## Datasets

- **EU Taxonomy Sustainability Reports Dataset** — total 190; splits: test (-1)

## Metrics

- `multi-label text classification` **(primary)** — range: 0-1
  - Not explicitly defined in the provided section; modeled as a multi-label text classification problem where the model predicts a set of relevant Taxonomy activities from operational descriptions.
- `regression` — range: percent
  - Not explicitly defined in the provided section; modeled as a regression problem where the model predicts the numerical percentage for Turnover, CapEx, and OpEx KPIs based on report text.

## Input / output format

**Input**: Unstructured corporate sustainability reports (annual reports) containing operational descriptions and financial narrative text.

**Output**: Task 1: A predicted set of relevant EU Taxonomy economic activities. Task 2: Numerical percentage values for Turnover, CapEx, and OpEx KPIs.

## Scoring recipe

```python
# Task 1: Multi-label text classification
predicted_activities = model.predict(report_text)
gold_activities = gold_labels[report_id]
classification_score = compute_multilabel_f1(predicted_activities, gold_activities)

# Task 2: Regression
predicted_kpis = model.predict_kpis(report_text) # Turnover, CapEx, OpEx
gold_kpis = gold_kpi_values[report_id]
regression_error = mean_absolute_error(predicted_kpis, gold_kpis)
```

## Common pitfalls

- Zero-shot setting means models are not fine-tuned on this dataset, making performance highly sensitive to prompt design and context window limits.
- The 'paradox of context' indicates that providing full unstructured reports may hurt performance compared to concise metadata, contrary to typical LLM expectations.
- Model confidence scores are poorly calibrated, so high confidence does not guarantee accurate KPI predictions.

## Evidence (verbatim from paper)

> We model this as a multi-label text classification problem, where the input is a company’s operational description (e.g., from an annual report) and the output is a predicted set of relevant Taxonomy activities. ... We model this as a regression problem, where the model must predict the numerical percentage for the Turnover, CapEx, and OpEx KPIs based on the textual content of a company’s report.

## Citation

```bibtex
@misc{schmoll2025automated,
  title={Automated Analysis of Sustainability Reports: Using Large Language Models for the Extraction and Prediction of EU Taxonomy-Compliant KPIs},
  author={Schmoll et al. (2025)},
  year={2025},
  note={arXiv:2512.24289}
}
```

- arXiv: 2512.24289