Olive Branch Learning Eval

---
name: olive-branch-learning-eval
description: Evaluates a topology-aware federated learning framework (OBL) with a satellite-assignment algorithm (CNASA) for space-air-ground integrated networks. It probes the trade-off between model accuracy and training latency under strict non-IID data distributions and varying network topologies. Use when the user wants to benchmark on MNIST, Fashion-MNIST, CIFAR-10, or asks about evaluating this task. Reports final global model accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2212.01215
  bibtex_key: [REDACTED]
  confidence: high
---

# olive-branch-learning-eval

> Olive Branch Learning: A Topology-Aware Federated Learning Framework for Space-Air-Ground Integrated Network — Fang et al. (2022) (arXiv:2212.01215, 2022)

## What this evaluates

Evaluates a topology-aware federated learning framework (OBL) with a satellite-assignment algorithm (CNASA) for space-air-ground integrated networks. It probes the trade-off between model accuracy and training latency under strict non-IID data distributions and varying network topologies.

## Datasets

- **MNIST** — total ?; splits: train (-1), test (-1); HF `mnist`
- **Fashion-MNIST** — total ?; splits: train (-1), test (-1); HF `fashion-mnist`
- **CIFAR-10** — total ?; splits: train (-1), test (-1); HF `cifar10`

## Metrics

- `final global model accuracy` **(primary)** — range: percent
  - Percentage of correctly classified samples on the held-out test set after the final global training round.
- `overall time cost of model training` — range: other
  - Total wall-clock time required to complete all global federated learning rounds, including local computation, intra-cluster aggregation, and inter-satellite communication delays.

## Input / output format

**Input**: Local image classification datasets (MNIST/Fashion-MNIST/CIFAR-10) partitioned non-IIDly across 200 IoRT devices (each holding exactly 2 classes). Models are CNNs trained locally.

**Output**: Global aggregated model parameters after each round, evaluated on a held-out test set to yield test accuracy and total training time.

## Scoring recipe

```python
def compute_metrics(predictions, gold_labels, total_time):
    correct = (predictions.argmax(dim=1) == gold_labels).sum().item()
    accuracy = correct / len(gold_labels) * 100
    return {'final global model accuracy': accuracy, 'overall time cost of model training': total_time}
```

## Common pitfalls

- The non-IID setting is highly constrained: each of the 200 devices holds data from exactly 2 classes, and geographically close devices share similar class distributions.
- Time cost is not just computation; it explicitly includes propagation delays (5 ms device-to-air, 5 ms air-to-satellite, 20 ms satellite-to-satellite) and bandwidth constraints (6000 Mbps satellite, 32 Gbps air node), which vary by topology.
- Baseline methods (GDO, CDO) are custom adaptations of FedAT and Astraea for SAGIN topologies, not standard FedAvg or SCAFFOLD implementations.

## Evidence (verbatim from paper)

> Our evaluation metrics include overall time cost of model training and final global model accuracy. ... Each dataset consists of a training set for model training and a test set for model evaluation... To mimic the challenging non-IID scenario, we assign each IoRT device samples of 2 classes, which will bring a significant negative impact on the global model accuracy.

## Citation

```bibtex
@misc{fang2022olivebranch,
  title={Olive Branch Learning: A Topology-Aware Federated Learning Framework for Space-Air-Ground Integrated Network},
  author={Fang et al. (2022)},
  year={2022},
  note={arXiv:2212.01215}
}
```

- arXiv: 2212.01215