Radio Source Classification Eval

---
name: radio-source-classification-eval
description: Evaluates the transferability of self-supervised learning representations for classifying radio astronomy sources from interferometric cutout images. It benchmarks multiple SSL pretraining methods against ImageNet baselines using linear probing and full fine-tuning on downstream classification tasks. Use when the user wants to benchmark on MiraBest, RGZ, MSRS, VLASS, or asks about evaluating this task. Reports classification accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2411.14078
  bibtex_key: cecconello2024selfsupervised
  confidence: high
---

# radio-source-classification-eval

> Self-supervised learning for radio-astronomy source classification: a benchmark — Cecconello et al. (2024) (arXiv:2411.14078, 2024)

## What this evaluates

Evaluates the transferability of self-supervised learning representations for classifying radio astronomy sources from interferometric cutout images. It benchmarks multiple SSL pretraining methods against ImageNet baselines using linear probing and full fine-tuning on downstream classification tasks.

## Datasets

- **MiraBest** — total ?; splits: test (-1)
- **RGZ** — total ?; splits: test (-1)
- **MSRS** — total ?; splits: test (-1)
- **VLASS** — total ?; splits: test (-1)

## Metrics

- `classification accuracy` **(primary)** — range: percent
  - Accuracy = (number of correctly classified images) / (total number of images). Reported as mean ± standard deviation over 3-fold cross-validation or 3 independent random seeds.

## Input / output format

**Input**: Radio astronomy cutout images normalized using per-image minimum and maximum pixel values, then resized to 224×224 pixels.

**Output**: Discrete class label indicating the radio source type.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)

# Report mean and std of accuracy over 3-fold CV or 3 runs
```

## Common pitfalls

- Confusing the pretraining datasets (Curated/Uncurated) with the downstream evaluation datasets (MiraBest, RGZ, MSRS, VLASS).
- Assuming ImageNet pretraining is competitive; the benchmark shows it generally underperforms SSL on radio data except on MSRS.
- Ignoring the reporting of standard deviation; results must include mean ± std over 3 runs or 3-fold CV.

## Evidence (verbatim from paper)

> Evaluation results on the downstream tasks are reported in terms of classification accuracy. Using the above final hyperparameters, we train each SSL method with each backbone on each pretraining dataset for three times with random initialization. The only exception is that, for the Uncurated dataset, we only use ResNet-18, for timing constraints. Then, we evaluate each trained model on all downstream tasks, using 3-fold cross-validation on each task. In practice, for a given combination of SSL method, backbone, pretraining dataset and downstream dataset, we have nine values of accuracy, for which we report the corresponding mean and standard deviation.

## Citation

```bibtex
@misc{cecconello2024selfsupervised,
  title={Self-supervised learning for radio-astronomy source classification: a benchmark},
  author={Cecconello et al. (2024)},
  year={2024},
  note={arXiv:2411.14078}
}
```

- arXiv: 2411.14078