Webfg Webinat Eval

---
name: webfg-webinat-eval
description: Evaluates fine-grained image classification performance on large-scale webly supervised datasets characterized by label noise and extreme class imbalance. It probes a model's ability to learn robust visual features and correct noisy labels through mutual peer learning or standard fine-tuning. Use when the user wants to benchmark on WebFG-496, WebiNat-5089, or asks about evaluating this task. Reports classification accuracy (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2108.02399
  bibtex_key: sun2021weblyfg
  confidence: high
---

# webfg-webinat-eval

> Webly Supervised Fine-Grained Recognition: Benchmark Datasets and An Approach — Sun et al. (2021) (arXiv:2108.02399, 2021)

## What this evaluates

Evaluates fine-grained image classification performance on large-scale webly supervised datasets characterized by label noise and extreme class imbalance. It probes a model's ability to learn robust visual features and correct noisy labels through mutual peer learning or standard fine-tuning.

## Datasets

- **WebFG-496** — total 53000; splits: train (-1), test (-1); repo https://github.com/NUST-Machine-Intelligence-Laboratory/weblyFG-dataset
- **WebiNat-5089** — total 1100000; splits: train (-1), test (-1); repo https://github.com/NUST-Machine-Intelligence-Laboratory/weblyFG-dataset

## Metrics

- `classification accuracy (%)` **(primary)** — range: percent
  - Percentage of correctly classified images out of the total test set. Calculated as (number of correct predictions / total number of samples) × 100.

## Input / output format

**Input**: RGB images of fine-grained objects (birds, aircraft, cars, or natural categories) with associated noisy web-collected labels.

**Output**: Predicted fine-grained class label for each input image.

## Scoring recipe

```python
correct = 0
total = 0
for img, true_label in test_loader:
    pred_label = model(img).argmax(dim=1)
    correct += (pred_label == true_label).sum().item()
    total += true_label.size(0)
accuracy = (correct / total) * 100
```

## Common pitfalls

- Webly datasets contain significant label noise; standard supervised baselines often fail without noise-robust techniques like peer-learning or label smoothing.
- Extreme class imbalance in WebiNat-5089 causes performance drops for standard architectures unless handled via specialized training strategies or data augmentation.
- Training time scales drastically with dataset size; single-stage vs two-stage fine-tuning strategies must be carefully chosen to balance efficiency and accuracy.

## Evidence (verbatim from paper)

> As shown in Table 4, we can notice that all six networks achieve reasonable fine-grained classification accuracy, which validates the reliability of WebFG-496.

## Citation

```bibtex
@misc{sun2021weblyfg,
  title={Webly Supervised Fine-Grained Recognition: Benchmark Datasets and An Approach},
  author={Sun et al. (2021)},
  year={2021},
  note={arXiv:2108.02399}
}
```

- arXiv: 2108.02399