Grasp Pruning Eval

---
name: grasp-pruning-eval
description: Evaluates the test accuracy of single-shot pruning methods at initialization on image classification tasks. It measures how well a pruned sub-network can be trained and generalizes compared to baselines like SNIP and random pruning. Use when the user wants to benchmark on CIFAR-10, CIFAR-100, Tiny-ImageNet, ImageNet, or asks about evaluating this task. Reports test accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2002.07376
  bibtex_key: wang2020grasp
  confidence: high
---

# grasp-pruning-eval

> Picking Winning Tickets Before Training by Preserving Gradient Flow — Wang et al. (2020) (arXiv:2002.07376, 2020)

## What this evaluates

Evaluates the test accuracy of single-shot pruning methods at initialization on image classification tasks. It measures how well a pruned sub-network can be trained and generalizes compared to baselines like SNIP and random pruning.

## Datasets

- **CIFAR-10** — total ?; splits: train (-1), test (-1)
- **CIFAR-100** — total ?; splits: train (-1), test (-1)
- **Tiny-ImageNet** — total ?; splits: train (-1), test (-1)
- **ImageNet** — total ?; splits: train (-1), val (-1)

## Metrics

- `test accuracy` **(primary)** — range: percent
  - Percentage of correctly classified images on the held-out test set. For ImageNet, both top-1 and top-5 accuracy are reported.

## Input / output format

**Input**: Image classification dataset (images and labels) with a specified neural network architecture (e.g., VGG, ResNet) and a target pruning ratio.

**Output**: A binary mask indicating which weights to keep/prune, followed by the trained pruned network's test accuracy on the evaluation split.

## Scoring recipe

```python
def compute_test_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100
```

## Common pitfalls

- Pruning is performed at initialization, so gradients must be computed on the untrained model before any weight updates.
- Results are averaged over multiple trials (3 for CIFAR/Tiny-ImageNet, 10 for gradient norm analysis) to account for initialization variance.
- High sparsity levels (e.g., 90%+) often lead to underfitting, making convergence speed and gradient norm preservation critical factors.

## Evidence (verbatim from paper)

> To evaluate the effectiveness of GraSP on real world tasks, we test GraSP on four image classification datasets, CIFAR-10/100, Tiny-ImageNet and ImageNet, with two modern network architectures, VGGNet and ResNet... The test accuracy is reported in Table 1... We run each experiment for 3 trials for obtaining more stable results.

## Citation

```bibtex
@misc{wang2020grasp,
  title={Picking Winning Tickets Before Training by Preserving Gradient Flow},
  author={Wang et al. (2020)},
  year={2020},
  note={arXiv:2002.07376}
}
```

- arXiv: 2002.07376