Research

Evaluates the sample efficiency and memory capabilities of reinforcement learning agents in partially observable environments. It probes whether a frozen language model can effectively compress historical observations to enable generalizable task solving without extensive finetuning. Use when the user wants to benchmark on RandomMaze, Minigrid (KeyCorridor), Procgen (Memory Mode), or asks about evaluating this task. Reports IQM of return.

Evaluates the robustness and generalization of reinforcement learning-based dialogue management policies across varying simulated environments. It probes how well RL algorithms handle different domain sizes, user behavior profiles, and noisy speech input channels in task-oriented spoken dialogue systems. Use when the user wants to benchmark on PyDial simulated environments, or asks about evaluating this task. Reports average success rate.

Evaluates LLMs' clinical capabilities across single-turn QA, diagnostic reasoning, and multi-turn dialogue using a urology-specific standardized patient dataset. It probes information gathering, diagnostic logic, treatment planning, and adherence to clinical workflows. Use when the user wants to benchmark on RJUA-SPs, or asks about evaluating this task. Reports Diagnosis Accuracy.

This benchmark evaluates large language models' ability to perform medical logical reasoning and urological disease diagnosis. It probes the model's capacity to handle complex, real-world clinical scenarios involving subjective patient queries and multi-disease comorbidity reasoning. Use when the user wants to benchmark on RJUA-QA, or asks about evaluating this task. Reports F1 score (diagnosis & advice).

Evaluates referring image segmentation on low-altitude drone imagery, probing the model's ability to accurately localize and segment referred objects despite challenges like category drift (tiny objects) and object drift (dense same-category scenes). Use when the user wants to benchmark on RIS-LAD, or asks about evaluating this task. Reports oIoU, mIoU.

Evaluates a model's ability to match 3D object patches and re-localize object instances in dynamically changing indoor environments. It measures feature matching robustness and 6DoF pose estimation accuracy under partial observations and contextual shifts. Use when the user wants to benchmark on 3RScan, or asks about evaluating this task. Reports Recall <0.1m, 10°.

This benchmark evaluates advanced mathematical reasoning at the International Mathematical Olympiad level. It probes two distinct capabilities: deriving a unique integer answer from complex problem statements, and constructing step-by-step deductive proofs by solving decomposed sub-problems. Use when the user wants to benchmark on RIMO, or asks about evaluating this task. Reports exact-match grading.

Evaluates a classifier's ability to predict the scientific rigour of academic papers (rated 4* vs non-4*) based solely on their abstracts and introductions. The setup tests whether linguistic patterns in early paper sections correlate with institutional rigour ratings. Use when the user wants to benchmark on REF dataset, ICLR dataset, ACL dataset, or asks about evaluating this task. Reports accuracy.

Evaluates the ability of a decoder-only latent variable model to learn geometry-respecting latent spaces on Riemannian manifolds. It probes reconstruction fidelity, preservation of intrinsic data structures (cyclical, hierarchical, phylogenetic), and downstream predictive utility of the learned latents. Use when the user wants to benchmark on Cell cycle stages (scRNA-seq), Branching diffusion process (synthetic tree), Human mitochondrial DNA (hmtDNA), or asks about evaluating this task. Repor...

Evaluates online bipartite matching algorithms for rideshare platforms on their ability to balance total trip profit and group-level fairness (subgroup representation) during peak demand hours. Use when the user wants to benchmark on NYC Yellow Cabs 2013, Synthetic Rideshare, or asks about evaluating this task. Reports competitive ratio of profit.

Probes a model's ability to segment long, unstructured legal documents into semantically coherent units and assign each sentence a specific rhetorical role label (e.g., Facts, Ratio, Arguments). This capability is fundamental for downstream legal AI applications like summarization and precedent search. Use when the user wants to benchmark on LegalEval RR Dataset, or asks about evaluating this task. Reports weighted F1 score.

Evaluates the ability of deep learning models to classify radio galaxy morphologies and detect radio sources in continuum images. It probes transfer learning, data preprocessing robustness, and handling of class imbalance in a specialized astronomical domain. Use when the user wants to benchmark on RGZ OD, or asks about evaluating this task. Reports Top-1 accuracy.

Evaluates a model's ability to perform continuous regression for Relative Geologic Time (RGT) estimation from 2D seismic images. It probes the model's capacity to learn stratigraphic continuity and structural consistency across diverse geological settings, testing generalization from synthetic labeled data to unlabeled real-world field data. Use when the user wants to benchmark on Field Seismic Dataset, Synthetic Seismic Dataset, or asks about evaluating this task. Reports regression.

Evaluates multi-modal visual grounding capabilities by requiring models to localize objects in images using both RGB and thermal infrared (TIR) modalities guided by text queries. It specifically probes robustness under complex real-world conditions such as low-light environments, small object sizes, and diverse weather/illumination variations. Use when the user wants to benchmark on RGBT-Ground, or asks about evaluating this task. Reports Acc@0.5.

Evaluates the ability of a skeleton-based BLSTM model to recognize human actions from RGB-only video streams under limited labeled data conditions, comparing against methods that use depth or inertial modalities. Use when the user wants to benchmark on UTD-MHAD, KTH, or asks about evaluating this task. Reports top-1 accuracy.

Evaluates the capability of RGB-Event fusion models to perform accurate per-pixel semantic segmentation under challenging conditions such as fast motion, varying lighting, and spatiotemporal misalignment between asynchronous modalities. Use when the user wants to benchmark on DDD17, DSEC, DELIVER, M3ED, or asks about evaluating this task. Reports mIoU.

Evaluates the robustness of Retrieval-Augmented Generation (RAG) systems against four key challenges: handling noisy or irrelevant retrieved documents, integrating information from multiple sources, rejecting queries when knowledge is insufficient, and detecting or correcting factual errors in counterfactual documents. Use when the user wants to benchmark on RGB Benchmark, or asks about evaluating this task. Reports Accuracy (ACC).

Evaluates deep learning models' ability to identify specific UAV models from radio-frequency signals by classifying time-frequency spectrograms. It probes robustness to varying signal-to-noise ratios (SNR) and sensitivity to preprocessing choices like color maps and frequency resolution. Use when the user wants to benchmark on RFUAV, or asks about evaluating this task. Reports Acc.

Measures the reconstruction fidelity of the visual tokenizer by comparing discrete latent reconstructions to original images/videos. Use when the user has predictions and gold and needs to compute rFID.

Evaluates indoor wireless transmitter localization accuracy using spatial spectrum inputs. It probes the model's ability to learn scene-agnostic spatial-spectral representations from unlabeled RF data and generalize across diverse indoor environments. Use when the user wants to benchmark on Indoor RF Localization Scenes, or asks about evaluating this task. Reports Euclidean distance (cm).

Evaluates the training efficiency, accuracy, and zero-shot generalization capability of Hamiltonian Graph Networks (RF-HGNs) on mass-spring physical systems. It benchmarks the proposed random-feature training method against standard gradient-based optimizers and existing physics-informed graph architectures. Use when the user wants to benchmark on 3D lattice mass-spring system, 2D open chain mass-spring system, 2D closed chain mass-spring system (Thangamuthu et al. [87]), or asks about evalua...

Evaluates whether a random forest model can accurately emulate subgrid atmospheric processes (vertical advection, cloud microphysics, turbulent diffusion, surface fluxes, radiative heating) from high-resolution simulation data. It further tests if the learned parameterization enables stable, long-term coarse-resolution climate simulations that reproduce key statistics like mean and extreme precipitation and ITCZ structure compared to high-resolution ground truth. Use when the user wants to be...

Evaluates the medical knowledge understanding and entity/relation coverage of AI-generated chest X-ray radiology reports by comparing structured knowledge graphs extracted from generated text against ground-truth clinical reports. It specifically probes whether models capture nuanced anatomical relationships, medical devices, and quantified measurements beyond surface-level lexical overlap. Use when the user wants to benchmark on CheXpert Plus, MIMIC-CXR, or asks about evaluating this task. R...

Evaluates multimodal models' ability to understand real-world medical photographs by answering clinician-verified multiple-choice questions across seven clinical domains. It probes capabilities in geometric perception, anatomical localization, clinical characterization, and causal reasoning. Use when the user wants to benchmark on ReXInTheWild, or asks about evaluating this task. Reports accuracy.