Treesatai Ts Eval

---
name: treesatai-ts-eval
description: Evaluates a model's ability to perform fine-grained tree species identification using multimodal Earth observation data, specifically leveraging temporal dynamics from optical and radar time series alongside high-resolution imagery. Use when the user wants to benchmark on TreeSatAI-TS, or asks about evaluating this task. Reports weighted F1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2404.08351
  bibtex_key: astruc2024omnisat
  confidence: high
---

# treesatai-ts-eval

> OmniSat: Self-Supervised Modality Fusion for Earth Observation — Astruc et al. (2024) (arXiv:2404.08351, 2024)

## What this evaluates

Evaluates a model's ability to perform fine-grained tree species identification using multimodal Earth observation data, specifically leveraging temporal dynamics from optical and radar time series alongside high-resolution imagery.

## Datasets

- **TreeSatAI-TS** — total 50381; splits: train (-1), test (-1); repo https://github.com/gastruc/OmniSat

## Metrics

- `weighted F1` **(primary)** — range: percent
  - Weighted average of per-class F1 scores, where each class's contribution is proportional to its support (number of true instances).

## Input / output format

**Input**: Multimodal satellite tiles containing VHR RGB/NIR images, Sentinel-1 SAR time series, and Sentinel-2 optical time series, georeferenced to 60x60m areas.

**Output**: Multi-label classification vector indicating presence/absence across 20 tree species classes.

## Scoring recipe

```python
def compute_weighted_f1(gold, pred):
    from sklearn.metrics import f1_score
    return f1_score(gold, pred, average='weighted', zero_division=0) * 100
```

## Common pitfalls

- Confusing single-date baselines with time-series inputs, which drastically underperform on temporal modalities.
- Using late feature fusion instead of end-to-end multimodal fusion, which fails to capture cross-modal interactions.

## Evidence (verbatim from paper)

> We report the weighted F1 for multi-label tree species classification on TreeSatAI (TSAI) and our extended TreeSatAI-TS (TSAI-TS) dataset when fine-tuning with 10% and 100% of training labels.

## Citation

```bibtex
@misc{astruc2024omnisat,
  title={OmniSat: Self-Supervised Modality Fusion for Earth Observation},
  author={Astruc et al. (2024)},
  year={2024},
  note={arXiv:2404.08351}
}
```

- arXiv: 2404.08351