Cmte Xray Detection Eval

---
name: cmte-xray-detection-eval
description: Evaluates the zero-shot cross-modality transfer capability of open-vocabulary object detectors from RGB to X-ray imaging. It measures how well pre-trained RGB detectors can localize and classify objects in X-ray images without any fine-tuning or labeled X-ray data. Use when the user wants to benchmark on DET-COMPASS, PIXray, PIDray, CLCXray, DvXray, HiXray, or asks about evaluating this task. Reports AP.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2503.17071
  bibtex_key: garciafernandez2025superpowering
  confidence: high
---

# cmte-xray-detection-eval

> Superpowering Open-Vocabulary Object Detectors for X-ray Vision — Garcia-Fernandez et al. (2025) (arXiv:2503.17071, 2025)

## What this evaluates

Evaluates the zero-shot cross-modality transfer capability of open-vocabulary object detectors from RGB to X-ray imaging. It measures how well pre-trained RGB detectors can localize and classify objects in X-ray images without any fine-tuning or labeled X-ray data.

## Datasets

- **DET-COMPASS** — total ?; splits: test (-1)
- **PIXray** — total ?; splits: test (-1)
- **PIDray** — total ?; splits: test (-1)
- **CLCXray** — total ?; splits: test (-1)
- **DvXray** — total ?; splits: test (-1)
- **HiXray** — total ?; splits: test (-1)

## Metrics

- `AP` **(primary)** — range: percent
  - Average Precision computed over IoU thresholds from 0.50 to 0.95 in steps of 0.05 and across three object area sizes (small, medium, large), averaged uniformly. Standard MS COCO metric.
- `AP50` — range: percent
  - Average Precision computed at a single IoU threshold of 0.50.
- `AP75` — range: percent
  - Average Precision computed at a single IoU threshold of 0.75.

## Input / output format

**Input**: X-ray images and a fixed vocabulary of object class names. The model processes the image without any gradient updates or fine-tuning on the target modality.

**Output**: Bounding box coordinates, predicted class labels, and confidence scores for each detected object per image.

## Scoring recipe

```python
def compute_coco_ap(predictions, ground_truths):
    # predictions: list of dicts with 'bbox', 'category_id', 'score'
    # ground_truths: list of dicts with 'bbox', 'category_id', 'iscrowd'
    # Uses standard COCO API to match predictions to GTs across IoU thresholds [0.50:0.05:0.95]
    # Computes precision-recall curves per category and area size
    # Returns mean AP across all categories, IoUs, and areas
    return coco_eval.evaluate(predictions, ground_truths)['AP']
```

## Common pitfalls

- Assuming the protocol allows fine-tuning on target X-ray data (CMTE is strictly training-free).
- Using raw web-retrieved X-ray images for descriptor construction degrades performance; filtering and material transfer are required.
- Confusing Cross-Modality Transfer Evaluation (CMTE) with standard Cross-Dataset Transfer Evaluation (CDTE), which assumes the same imaging modality.

## Evidence (verbatim from paper)

> Under CMTE, an OvOD model is trained on a source RGB dataset and subsequently evaluated on target X-ray datasets without any additional training or fine-tuning. For performance assessment, we employ the standard MS COCO metrics: AP, AP50, and AP75.

## Citation

```bibtex
@misc{garciafernandez2025superpowering,
  title={Superpowering Open-Vocabulary Object Detectors for X-ray Vision},
  author={Garcia-Fernandez et al. (2025)},
  year={2025},
  note={arXiv:2503.17071}
}
```

- arXiv: 2503.17071