Runtime Efficiency Benchmark

---
name: runtime-efficiency-benchmark
description: Evaluates the computational running time, speedup gains, and parallel efficiency of median, standard deviation, and full source-finding algorithms on simulated radio interferometric images. Use when the user has predictions and gold and needs to compute Running time (seconds).
metadata:
  skill_kind: metric
  source_arxiv: 1901.04956
  bibtex_key: [REDACTED]
  confidence: high
---

# runtime-efficiency-benchmark

> Efficient Source Finding for Radio Interferometric Images — Lucas et al. (2019) (arXiv:1901.04956, 2019)

## What this evaluates

Evaluates the computational running time, speedup gains, and parallel efficiency of median, standard deviation, and full source-finding algorithms on simulated radio interferometric images.

## Datasets

- **Simulated Radio Interferometric Images** — total ?; splits: (unstated)

## Metrics

- `Running time (seconds)` **(primary)** — range: other
  - Wall-clock execution time measured in seconds for each algorithmic step or the full pipeline across varying image sizes.

## Input / output format

**Input**: 2D simulated astronomical images of sizes $2^{10}\times 2^{10}$ to $2^{16}\times 2^{16}$, containing 3 Gaussian-shaped sources.

**Output**: Execution duration in seconds, speedup ratio relative to a baseline implementation, and parallel efficiency percentage.

## Scoring recipe

```python
def compute_metrics(algorithm, image_size, baseline_time):
    start = time.perf_counter()
    result = algorithm.run(image_size)
    elapsed = time.perf_counter() - start
    speedup = baseline_time / elapsed
    parallel_efficiency = (speedup / num_cores) * 100
    return elapsed, speedup, parallel_efficiency
```

## Common pitfalls

- Timings vary due to concurrent OS tasks and must be treated as rough estimates rather than absolute constants.
- Small image sizes fit in CPU cache, causing non-linear timing variations unrelated to algorithmic complexity.
- Hyper-threading is explicitly disabled to avoid shared hardware resource contention skewing parallel efficiency.

## Evidence (verbatim from paper)

> The benchmarking tests discussed here are based on the measured running times, which provide a rough idea of the relative computational complexity of the tested functions. It is important to note that these timings may vary between distinct benchmarking instances, even on the same hardware, as they may be affected by the operating system tasks running concurrently and as well as its status.

## Citation

```bibtex
@misc{lucas2019efficient,
  title={Efficient Source Finding for Radio Interferometric Images},
  author={Lucas et al. (2019)},
  year={2019},
  note={arXiv:1901.04956}
}
```

- arXiv: 1901.04956