Saga 3d Mesh Attack Eval

---
name: saga-3d-mesh-attack-eval
description: Evaluates the effectiveness of a spectral geometric adversarial attack on 3D mesh autoencoders by measuring how well perturbed meshes deceive a downstream classifier and evade detection. Use when the user wants to benchmark on CoMA, SMAL, or asks about evaluating this task. Reports Targeted classification accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2211.13775
  bibtex_key: stolik2022saga
  confidence: high
---

# saga-3d-mesh-attack-eval

> SAGA: Spectral Adversarial Geometric Attack on 3D Meshes — Stolik et al. (2022) (arXiv:2211.13775, 2022)

## What this evaluates

Evaluates the effectiveness of a spectral geometric adversarial attack on 3D mesh autoencoders by measuring how well perturbed meshes deceive a downstream classifier and evade detection.

## Datasets

- **CoMA** — total 10649; splits: train (8325), val (926), test (1398)
- **SMAL** — total 9918; splits: train (-1), val (-1), test (-1)

## Metrics

- `Targeted classification accuracy` **(primary)** — range: percent
  - Percentage of adversarial reconstructions that a PointNet classifier correctly labels as the target class. Calculated as (correct target predictions / total attacked pairs) * 100.
- `Untargeted classification accuracy` — range: percent
  - Percentage of adversarial reconstructions that the classifier labels as any class different from the source class. Calculated as (non-source predictions / total attacked pairs) * 100.
- `Detection accuracy` — range: percent
  - Percentage of adversarial meshes correctly identified as adversarial by a trained detector network. Lower values indicate a more successful, imperceptible attack.

## Input / output format

**Input**: 3D mesh vertex coordinates and connectivity (source shape)

**Output**: Adversarial 3D mesh (perturbed source shape) fed into a Mesh Autoencoder, followed by classification/detection of the AE's reconstruction.

## Scoring recipe

```python
def compute_metrics(reconstructions, source_classes, target_classes, detector):
    targeted_correct = sum(1 for r, t in zip(reconstructions, target_classes) if classifier.predict(r) == t)
    targeted_acc = (targeted_correct / len(reconstructions)) * 100
    
    untargeted_correct = sum(1 for r, s in zip(reconstructions, source_classes) if classifier.predict(r) != s)
    untargeted_acc = (untargeted_correct / len(reconstructions)) * 100
    
    detected_correct = sum(1 for r in reconstructions if detector.predict(r) == 'adversarial')
    detection_acc = (detected_correct / len(reconstructions)) * 100
    return targeted_acc, untargeted_acc, detection_acc
```

## Common pitfalls

- Lower detection accuracy indicates a more successful attack, which is the inverse of standard classification metrics.
- Metrics are computed on the Mesh Autoencoder's reconstruction of the adversarial mesh, not directly on the adversarial mesh itself.
- The attack is optimized per source-target pair (50 sources per class × 10 targets = 5500 pairs for CoMA), not a universal attack.

## Evidence (verbatim from paper)

> Table 1 shows the accuracy obtained from classifying the adversarial reconstructions as the target, in the targeted case, or differently from the source, in the untargeted case. The experiment included all the attacked pairs. We compare our attack with Lang et al.’s PC attack and with the clean target reconstructions.

## Citation

```bibtex
@misc{stolik2022saga,
  title={SAGA: Spectral Adversarial Geometric Attack on 3D Meshes},
  author={Stolik et al. (2022)},
  year={2022},
  note={arXiv:2211.13775}
}
```

- arXiv: 2211.13775