Inatag Eval

---
name: inatag-eval
description: Evaluates multi-class image classification capabilities for agricultural species, genus, family, and crop/weed distinction. Probes fine-grained visual recognition and taxonomic hierarchy understanding in plant identification. Use when the user wants to benchmark on iNatAg, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2503.20068
  bibtex_key: jain2025inatag
  confidence: high
---

# inatag-eval

> iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species — Jain et al. (2025) (arXiv:2503.20068, 2025)

## What this evaluates

Evaluates multi-class image classification capabilities for agricultural species, genus, family, and crop/weed distinction. Probes fine-grained visual recognition and taxonomic hierarchy understanding in plant identification.

## Datasets

- **iNatAg** — total 2059000; splits: train (1544250), val (308850), test (205900); repo https://github.com/Project-AgML/AgML

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Percentage of correctly predicted labels out of the total number of predictions. Computed per task (species, genus, family, crop/weed) and reported as a percentage.
- `F1 Score` — range: percent
  - Harmonic mean of precision and recall. Computed per task and reported as a percentage.

## Input / output format

**Input**: RGB image resized to 384×384 or 224×224 pixels, optionally concatenated with a 32-dimensional embedding derived from normalized latitude and longitude coordinates.

**Output**: Predicted class index for species (2,059 classes), genus, family, or binary label for crop/weed (0=weed, 1=crop).

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return (correct / len(gold)) * 100

def compute_f1(predictions, gold, num_classes):
    tp = fp = fn = 0
    for p, g in zip(predictions, gold):
        if p == g:
            tp += 1
        elif p != g:
            fp += 1
            fn += 1
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    return (2 * precision * recall / (precision + recall)) * 100 if (precision + recall) > 0 else 0
```

## Common pitfalls

- The experimental dataset caps images at 1,000 per species (2,059,000 total), which differs from the full 4.7M iNatAg release mentioned in the abstract.
- Genus and family metrics are derived by mapping species predictions to taxonomic groups via a lookup table, not trained directly, which can distort hierarchy accuracy if species predictions are incorrect.
- Geospatial inputs must be normalized and passed through a fully connected layer to produce a 32-dim embedding before concatenation; raw coordinates cannot be fed directly.

## Evidence (verbatim from paper)

> All trained models were evaluated on a test set consisting of 205,900 images. Evaluation included both fine-grained species classification and binary crop/weed classification, with metrics such as accuracy, precision, recall, and F1 score. To assess taxonomic generalization, genus and family-level accuracy were computed by mapping species predictions to their respective higher-order groups using a predefined lookup table.

## Citation

```bibtex
@misc{jain2025inatag,
  title={iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species},
  author={Jain et al. (2025)},
  year={2025},
  note={arXiv:2503.20068}
}
```

- arXiv: 2503.20068