Tusimple Eval

---
name: tusimple-eval
description: Evaluates a model's ability to detect lane boundaries in highway driving scenarios using a point-based accuracy metric over predefined row anchors. Use when the user wants to benchmark on TuSimple, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2004.11757
  bibtex_key: qin2020ultrafast
  confidence: high
---

# tusimple-eval

> Ultra Fast Structure-aware Deep Lane Detection — Qin et al. (2020) (arXiv:2004.11757, 2020)

## What this evaluates

Evaluates a model's ability to detect lane boundaries in highway driving scenarios using a point-based accuracy metric over predefined row anchors.

## Datasets

- **TuSimple** — total 6408; splits: train (3268), validation (358), test (2782)

## Metrics

- `accuracy` **(primary)** — range: percent
  - accuracy = (sum of correctly predicted lane points across all clips) / (sum of total ground truth lane points across all clips).

## Input / output format

**Input**: RGB image (original resolution 1280x720, resized to 288x800 during training)

**Output**: Predicted lane points/coordinates via row-anchor classification or regression

## Scoring recipe

```python
total_correct = 0
total_gt = 0
for clip in dataset:
    gt_points = get_gt_lane_points(clip)
    pred_points = get_pred_lane_points(clip)
    correct = count_matched_points(gt_points, pred_points)
    total_correct += correct
    total_gt += len(gt_points)
accuracy = total_correct / total_gt
```

## Common pitfalls

- The metric is point-based accuracy, not pixel-wise segmentation accuracy.
- Row anchors are fixed per dataset (TuSimple: 160-710, step 10; CULane: 260-530, step 10).
- Images are resized to 288x800 for training but evaluated at original resolution.

## Evidence (verbatim from paper)

> For TuSimple dataset, the main evaluation metric is accuracy. The accuracy is calculated by: accuracy = \sum_{clip}C_{clip} / \sum_{clip}S_{clip}, in which C_{clip} is the number of lane points predicted correctly and S_{clip} is the total number of ground truth in each clip.

## Citation

```bibtex
@misc{qin2020ultrafast,
  title={Ultra Fast Structure-aware Deep Lane Detection},
  author={Qin et al. (2020)},
  year={2020},
  note={arXiv:2004.11757}
}
```

- arXiv: 2004.11757