Research

Evaluates how well vision foundation models support robot manipulation policies in simulation and real-world environments. It probes cross-modal spatial reasoning, task generalization across diverse manipulation suites, and robustness to sensor noise and platform differences. Use when the user wants to benchmark on LIBERO, MetaWorld, or asks about evaluating this task. Reports success rate.

Evaluates the ability of reinforcement learning algorithms to learn sequential robot manipulation tasks in simulation. It probes how well methods can handle randomized initial states, multi-stage objectives, and continuous control over fixed-horizon episodes. Use when the user wants to benchmark on robosuite, or asks about evaluating this task. Reports reward.

Evaluates egocentric robot perception and navigation in crowded, unstructured environments. It probes multi-view 3D detection, 3D multi-object tracking, motion prediction, and 3D/BEV occupancy prediction using synchronized camera, LiDAR, and ultrasonic sensor data. Use when the user wants to benchmark on RoboSense, or asks about evaluating this task. Reports average precision.

Evaluates a multimodal robot narration framework's ability to select key events, generate natural language summaries, and perform failure analysis (risk estimation, localization, explanation, recovery) on real-world household robot tasks. Use when the user wants to benchmark on RoboNar, or asks about evaluating this task. Reports Accuracy on failure analysis tasks.

Evaluates object detection models' ability to generalize across diverse, real-world, domain-specific visual tasks. It probes fine-tuning performance and zero-shot transfer capabilities on crowdsourced, practitioner-curated datasets spanning multiple imaging modalities. Use when the user wants to benchmark on Roboflow 100, or asks about evaluating this task. Reports mAP@.50.

Evaluates the robustness and safety of vision-language models (VLMs) for end-to-end autonomous driving when subjected to real-world sensor corruptions (e.g., fog, rain, motion blur) and prompt corruptions (e.g., bit errors, malicious attacks). It probes the model's ability to maintain accurate trajectory prediction and low collision rates under degraded inputs. Use when the user wants to benchmark on RoboDriveBench, or asks about evaluating this task. Reports AvgL2.

This benchmark evaluates the out-of-distribution robustness of monocular depth estimation (MDE) models when exposed to real-world corruptions such as weather changes, sensor failures, and data processing anomalies. It measures how much model performance degrades relative to a clean baseline across multiple corruption types and severity levels. Use when the user wants to benchmark on KITTI-C, NYUDepth2-C, KITTI-S, or asks about evaluating this task. Reports mCE.

Evaluates a robot's ability to generalize semantic knowledge by predicting object affordances, locations, and materials in unseen environments and inferring ranks of unseen semantic triples. It probes multi-relational embedding performance for common-sense reasoning in residential robotics. Use when the user wants to benchmark on AI2Thor, or asks about evaluating this task. Reports MRR.

Evaluates the effectiveness of a large-scale bimanual manipulation dataset and a hierarchical annotation framework on Vision-Language-Action models across different robotic platforms. It probes the models' ability to generalize across task complexities, leverage multi-resolution annotations, and benefit from trajectory quality filtering. Use when the user wants to benchmark on RoboCOIN, or asks about evaluating this task. Reports success_rate.

Evaluates multimodal large language models (MLLMs) as embodied brains in robotic manipulation. It probes five cognitive dimensions: instruction comprehension, perception reasoning, generalized planning, affordance prediction, and failure analysis across diverse real-world robotic tasks. Use when the user wants to benchmark on RoboBench, or asks about evaluating this task. Reports accuracy (%).

This benchmark evaluates the ranking accuracy and sample efficiency of generalist robot policies in real-world environments. It probes whether a distributed, pairwise comparison framework can reliably approximate an exhaustive oracle ranking across diverse scenes and tasks. Use when the user wants to benchmark on RoboArena Real-World Policy Evaluation, or asks about evaluating this task. Reports Pearson correlation r.

Evaluates vision-language models' ability to perform single-step and multi-step spatial understanding and referring tasks in robotics contexts. It probes capabilities like 2D/3D relation reasoning, depth perception, and complex compositional spatial constraints in cluttered scenes. Use when the user wants to benchmark on CV-Bench, BLINK, RoboSpatial, RefSpatial-Bench, RefCOCO, or asks about evaluating this task. Reports Top-1 accuracy.

Evaluates a model's ability to jointly detect aspects, sentiments, targets, and opinions across entire product/course reviews, capturing contextual dependencies that sentence-level methods miss. It probes review-level joint extraction for both triplet (aspect-sentiment-target) and quadruple (aspect-sentiment-target-opinion) formats. Use when the user wants to benchmark on ROAST Benchmark (Amazon_FF, Coursera, Hotels, Phones, Movies), or asks about evaluating this task. Reports F1 score.

This benchmark evaluates the quality of feature importance estimators in deep neural networks by measuring how model performance degrades when ranked important features are removed and the model is retrained. It probes whether an interpretability method correctly identifies pixels that the model actually relies on for prediction. Use when the user wants to benchmark on ImageNet, Birdsnap, Food 101, or asks about evaluating this task. Reports test accuracy.

Evaluates vision-language models on visual question answering for Indian road scenes. It probes capabilities in object counting, object description, and surrounding scene description across diverse driving environments. Use when the user wants to benchmark on RoadscapesQA, or asks about evaluating this task. Reports exact-match accuracy.

Evaluates the ability to estimate traffic flow profiles for unsensed road segments by selecting similar roads based on topological embeddings or generating synthetic data. It probes how well graph-based feature similarity correlates with actual traffic pattern matching and the accuracy of generative models for sensorless traffic estimation. Use when the user wants to benchmark on Madrid Traffic Network, or asks about evaluating this task. Reports RMSE.

Evaluates graph neural networks and embedding methods for predicting traffic accident occurrences and counts on road network edges. It probes the models' ability to capture spatial-temporal dependencies, leverage graph structural features, and benefit from multitask or transfer learning across different U.S. states. Use when the user wants to benchmark on Traffic Accident Dataset, or asks about evaluating this task. Reports MAE.

Evaluates video-level road anomaly segmentation models in autonomous driving scenarios, specifically probing their ability to maintain prediction validity over time sequences and perform under real-time latency constraints. Use when the user wants to benchmark on Road Anomaly Segmentation Dataset, or asks about evaluating this task. Reports latency-aware metrics.

Evaluates a model's ability to score and rank candidate 3D RNA structural models by predicting their deviation from the true native structure. It probes the model's capacity to distinguish accurate conformations from decoys using only atomic coordinates and types. Use when the user wants to benchmark on RNA-Puzzles & FARFAR2 Decoys, or asks about evaluating this task. Reports RMSD.

This benchmark evaluates robotic manipulation policies on memory-dependent, non-Markovian tasks. It probes a model's ability to retain and utilize historical visual and state information over long horizons to complete multi-step dual-arm manipulation sequences. Use when the user wants to benchmark on RMBench, or asks about evaluating this task. Reports success rate.

Evaluates reward models' ability to correctly identify preferred responses based on substantive content rather than superficial stylistic cues. It probes sensitivity to subtle correctness differences, resistance to verbosity/style bias, and performance across diverse domains like math, code, and safety. Use when the user wants to benchmark on RM-Bench, or asks about evaluating this task. Reports Average Accuracy.

This evaluation probes the mathematical and out-of-domain reasoning capabilities of large language models trained with Reinforcement Learning with Verifiable Rewards (RLVR). It specifically tests how well entropy-aware credit assignment methods allocate learning signals across high-entropy tokens during chain-of-thought generation. Performance is measured by average accuracy and pass rate over multiple sampled reasoning paths. Use when the user wants to benchmark on AIME24, AIME25, AMC, MATH,...

Evaluates a reinforcement learning framework for dynamic search ranking that adapts to evolving user intents over multiple search iterations using sequential feedback. It also benchmarks standard learning-to-rank performance on static datasets. Use when the user wants to benchmark on TREC 2016 Dynamic Domain, TREC 2017 Dynamic Domain, MQ2007, MQ2008, or asks about evaluating this task. Reports α-NDCG.

Evaluates the trustworthiness (hallucination reduction) and helpfulness of multimodal large language models across generative, discriminative, and free-format tasks. Use when the user wants to benchmark on Object HalBench, MMHal-Bench, MHumanEval, AMBER, RefoMB, MMStar, or asks about evaluating this task. Reports response-level hallucination rate, trustworthiness win rate.