Excelchart400k Eval

---
name: excelchart400k-eval
description: Evaluates a model's ability to recognize and segment specific chart components (e.g., bars, lines, pie slices, legends, axis titles) within chart images using instance segmentation. Use when the user wants to benchmark on ExcelChart400K, or asks about evaluating this task. Reports mAP.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2407.21038
  bibtex_key: zheng2024advancing
  confidence: high
---

# excelchart400k-eval

> Advancing Chart Question Answering with Robust Chart Component Recognition — Hanwen Zheng et al. (2024) (arXiv:2407.21038, 2024)

## What this evaluates

Evaluates a model's ability to recognize and segment specific chart components (e.g., bars, lines, pie slices, legends, axis titles) within chart images using instance segmentation.

## Datasets

- **ExcelChart400K** — total 1698970; splits: train (1581008), val (59357), test (62605)

## Metrics

- `mAP` **(primary)** — range: [0, 1]
  - Mean Average Precision across all classes and recall levels, averaging precision values to produce a single score.

## Input / output format

**Input**: Chart image.

**Output**: Instance segmentation masks (polygons) and class labels for each detected chart component.

## Scoring recipe

```python
def compute_map(predictions, ground_truth, iou_thresh=0.5):
    # predictions and ground_truth are lists of dicts with 'mask', 'category', 'score'
    # Calculate IoU between predicted and ground truth masks
    # Match predictions to ground truth based on IoU >= threshold and highest score
    # Compute precision-recall curve per class
    # Average precision per class, then average across classes
    pass
```

## Common pitfalls

- Confusing instance segmentation masks with bounding boxes; mAP requires precise polygon overlap rather than box overlap.
- Failing to account for the 7 specific chart component categories, which may lead to incorrect class mapping or missed detections for titles and legends.

## Evidence (verbatim from paper)

> We evaluate the performance of chart component recognition with three metrics: (1) mAP (mean Average Precision) is a common metric for evaluating object detection and instance segmentation tasks. It calculates the average precision for each class across all recall levels and then averages these values to get a single score.

## Citation

```bibtex
@misc{zheng2024advancing,
  title={Advancing Chart Question Answering with Robust Chart Component Recognition},
  author={Hanwen Zheng et al. (2024)},
  year={2024},
  note={arXiv:2407.21038}
}
```

- arXiv: 2407.21038