Claude Skills by qhjqhj00

Evaluates a model's ability to perform session-based next-item recommendation by capturing both spatial graph structures and temporal dynamics. It probes how well the model aggregates collaborative filtering signals and session-specific sequences to predict the subsequent item in a user's browsing session. Use when the user wants to benchmark on Tmall, Diginetica, Gowalla, RetailRocket, Nowplaying, LastFM, or asks about evaluating this task. Reports cross-entropy.

Evaluates the reasoning capabilities of language models on synthetically generated, code-verifiable tasks. It measures performance on both the custom ReSyn dataset and standard reasoning benchmarks using zero-shot generation with specific sampling parameters. Use when the user wants to benchmark on ReSyn, or asks about evaluating this task. Reports mean@4.

Evaluates the accuracy and structural consistency of retinal vascular tree annotations across pixel, vessel segment, and network levels, ensuring topological correctness and geometrical plausibility. Use when the user wants to benchmark on RETA Benchmark, or asks about evaluating this task. Reports multi_stage_annotation.

Evaluates a model's ability to segment retinal blood vessels in fundus images, focusing on preserving fine, elongated vascular structures and handling varying image resolutions and pathologies. The protocol tests robustness under strict hyperparameter consistency and standardized data splits. Use when the user wants to benchmark on DRIVE, STARE, CHASE_DB1, HRF, or asks about evaluating this task. Reports F1 score.

Evaluates the effectiveness of sparse and dense retrieval models, re-rankers, and retrieval-augmented generation (RAG) pipelines on open-domain QA, multi-hop QA, and fact verification tasks. Use when the user wants to benchmark on Natural Questions, TriviaQA, HotpotQA, 2WikiMultiHopQA, ArchivalQA, MSMARCO, WebQuestions, PopQA, ChroniclingAmericaQA, or asks about evaluating this task. Reports Top-k accuracy, Exact match (EM).

This evaluation probes how consistently large language models maintain or improve their answer quality when provided with retrieved context, specifically measuring resilience to variations in retrieval size, document order, and the risk of performance degradation compared to non-retrieval baselines. Use when the user wants to benchmark on Wikipedia QA benchmark, or asks about evaluating this task. Reports No-Degradation Rate (NDR).

Compute the RetrievalAUROC metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalAUROC, or asks how to score with RetrievalAUROC.

Compute the RetrievalFallOut metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalFallOut, or asks how to score with RetrievalFallOut.

Compute the RetrievalHitRate metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalHitRate, or asks how to score with RetrievalHitRate.

Compute the RetrievalMAP metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalMAP, or asks how to score with RetrievalMAP.

Compute the RetrievalMRR metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalMRR, or asks how to score with RetrievalMRR.

Compute the RetrievalNormalizedDCG metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalNormalizedDCG, or asks how to score with RetrievalNormalizedDCG.

Compute the RetrievalPrecision metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalPrecision, or asks how to score with RetrievalPrecision.

Compute the RetrievalPrecisionRecallCurve metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalPrecisionRecallCurve, or asks how to score with RetrievalPrecisionRecallCurve.

Compute the RetrievalRecall metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalRecall, or asks how to score with RetrievalRecall.

Compute the RetrievalRecallAtFixedPrecision metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalRecallAtFixedPrecision, or asks how to score with RetrievalRecallAtFixedPrecision.

Compute the RetrievalRPrecision metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute RetrievalRPrecision, or asks how to score with RetrievalRPrecision.

Evaluates the semantic quality of pre-trained word vectors by measuring performance improvements after applying a graph-based retrofitting method using semantic lexicons. It probes the model's ability to capture lexical relations (e.g., synonymy, hyponymy) and generalizes across different vector training methods, lexicon types, and languages. Use when the user wants to benchmark on MEN-3k, RG-65, WS-353, TOEFL, SYN-REL, SA, MC-30, or asks about evaluating this task. Reports Spearman's correla...

Evaluates an LLM's ability to generate valid chemical synthesis pathways for target molecules given reference routes and iterative feedback. Use when the user wants to benchmark on Pistachio Hard, or asks about evaluating this task. Reports solve rate.

Evaluates video anomaly detection models on dashcam footage, specifically testing their ability to detect complex traffic anomalies like collisions and skidding in dynamic, real-world driving scenes. It also benchmarks performance against standard pedestrian anomaly detection datasets to highlight challenges posed by moving cameras and contextual anomalies. Use when the user wants to benchmark on RetroTrucks, UCSD Ped1, UCSD Ped2, ShanghaiTech, or asks about evaluating this task. Reports AUC-...

Evaluates the quality of academic peer review reports across different conferences and years using a multi-dimensional framework. It measures how substantive, actionable, and well-grounded reviews are, and tracks whether these qualities decline over time. Use when the user wants to benchmark on Peer Review Campaigns (ICLR, NeurIPS, ACL), or asks about evaluating this task. Reports Q.

Evaluates the quality of academic peer reviews by measuring how well criticisms are supported by evidence (substantiation), how factually accurate the review's claims are (correctness), and how thoroughly the review covers the paper's contributions (completeness). Use when the user has predictions and gold and needs to compute correctness.

Evaluates the quality and accuracy of automated peer reviews generated by LLMs. It probes both the semantic alignment of review text against paper-specific rubrics and the precision of predicted numerical ratings and acceptance decisions. Use when the user wants to benchmark on ReviewBench, or asks about evaluating this task. Reports Rubric Overall Score.

Evaluates the ability of LLM-based reviewer agents to predict conference acceptance decisions and generate high-quality peer reviews. It probes classification accuracy against human decisions and assesses review quality through LLM-judged pairwise comparisons across multiple dimensions. Use when the user wants to benchmark on ICLR-2k dataset, or asks about evaluating this task. Reports macro-F1 (5-way).

Evaluates LLMs on simulating the academic peer review process across multi-turn dialogues. It probes the model's ability to generate relevant paper summaries, write comprehensive reviews, and make accurate acceptance or rejection decisions based on long-context interactions. Use when the user wants to benchmark on ReviewMT, or asks about evaluating this task. Reports F1-score.

Evaluates generalized speech enhancement and audio-visual speech resynthesis across multiple distortion types (denoising, separation, inpainting, video-to-speech). Measures content intelligibility, audio-video synchronization, perceptual quality, and low-level signal reconstruction. Use when the user wants to benchmark on LRS3, EasyCom, or asks about evaluating this task. Reports WER.

Evaluates the spatial reasoning and logical comprehension capabilities of multimodal large language models (MLLMs) on synthetic, spatially precise images. It probes robustness to negations, logical operators (AND/OR), adversarial object substitutions, and complex spatial relationships. Use when the user wants to benchmark on RevQA, or asks about evaluating this task. Reports performance.

Evaluates reward models on their ability to correctly rank or score LLM-generated responses across diverse domains like safety, mathematics, coding, and instruction following. It probes pairwise preference accuracy, robustness to superficial biases, cross-sample score calibration, and consistency in assigning absolute quality scores. Use when the user wants to benchmark on RewardBench, RM-Bench, PPE, JudgeBench, or asks about evaluating this task. Reports binary choice accuracy.

RewardBench2 evaluates reward models across six distinct capabilities: focus, math, safety, factuality, precise instruction following, and ties. It probes whether models can correctly rank a single high-quality response against three inferior completions, and specifically tests robustness in domains with multiple equally valid answers. The benchmark uses unseen human prompts to ensure evaluation independence from downstream post-training tests. Use when the user wants to benchmark on RewardBe...

Evaluates fine-grained visual reasoning and spatial understanding on structured domains like transit maps. It probes the model's ability to follow complex routes, count stops, and verify true/false statements about visual layouts, while also measuring generalization to broader spatial and chart reasoning benchmarks. Use when the user wants to benchmark on ReasonMap, ReasonMap-Plus, SEED-Bench-2-Plus, SpatialEval, V*Bench, HRBench, ChartQA, MMStar, or asks about evaluating this task. Reports W...

This benchmark evaluates the ability of LLM-based coding agents to autonomously implement research extensions by modifying existing AI/ML codebases based on domain-expert instructions. It probes complex, multi-step software engineering capabilities, including codebase navigation, hypothesis-driven implementation, and producing executable patches. Use when the user wants to benchmark on REXBench, or asks about evaluating this task. Reports final success rate.

Evaluates AI models' ability to generate accurate and clinically relevant radiology reports from chest X-ray images. It assesses both linguistic quality and clinical entity extraction/alignment across diverse clinical datasets. Use when the user wants to benchmark on ReXGradient, MIMIC-CXR, IU X-ray, CheXpert Plus, or asks about evaluating this task. Reports 1/RadCliQ-v1.

This benchmark evaluates video-language models on procedure-centric ultrasound understanding, specifically probing dynamic procedural reasoning, causal troubleshooting, and temporal action understanding. It measures how well models interpret visual evidence and reason through medical imaging procedures without relying on audio or static priors. Use when the user wants to benchmark on ReXSonoVQA, or asks about evaluating this task. Reports accuracy.

Evaluates multimodal models' ability to understand real-world medical photographs by answering clinician-verified multiple-choice questions across seven clinical domains. It probes capabilities in geometric perception, anatomical localization, clinical characterization, and causal reasoning. Use when the user wants to benchmark on ReXInTheWild, or asks about evaluating this task. Reports accuracy.

Evaluates the medical knowledge understanding and entity/relation coverage of AI-generated chest X-ray radiology reports by comparing structured knowledge graphs extracted from generated text against ground-truth clinical reports. It specifically probes whether models capture nuanced anatomical relationships, medical devices, and quantified measurements beyond surface-level lexical overlap. Use when the user wants to benchmark on CheXpert Plus, MIMIC-CXR, or asks about evaluating this task. R...

Evaluates whether a random forest model can accurately emulate subgrid atmospheric processes (vertical advection, cloud microphysics, turbulent diffusion, surface fluxes, radiative heating) from high-resolution simulation data. It further tests if the learned parameterization enables stable, long-term coarse-resolution climate simulations that reproduce key statistics like mean and extreme precipitation and ITCZ structure compared to high-resolution ground truth. Use when the user wants to be...

Evaluates the training efficiency, accuracy, and zero-shot generalization capability of Hamiltonian Graph Networks (RF-HGNs) on mass-spring physical systems. It benchmarks the proposed random-feature training method against standard gradient-based optimizers and existing physics-informed graph architectures. Use when the user wants to benchmark on 3D lattice mass-spring system, 2D open chain mass-spring system, 2D closed chain mass-spring system (Thangamuthu et al. [87]), or asks about evalua...

Evaluates indoor wireless transmitter localization accuracy using spatial spectrum inputs. It probes the model's ability to learn scene-agnostic spatial-spectral representations from unlabeled RF data and generalize across diverse indoor environments. Use when the user wants to benchmark on Indoor RF Localization Scenes, or asks about evaluating this task. Reports Euclidean distance (cm).

Measures the reconstruction fidelity of the visual tokenizer by comparing discrete latent reconstructions to original images/videos. Use when the user has predictions and gold and needs to compute rFID.

Evaluates deep learning models' ability to identify specific UAV models from radio-frequency signals by classifying time-frequency spectrograms. It probes robustness to varying signal-to-noise ratios (SNR) and sensitivity to preprocessing choices like color maps and frequency resolution. Use when the user wants to benchmark on RFUAV, or asks about evaluating this task. Reports Acc.

Evaluates the robustness of Retrieval-Augmented Generation (RAG) systems against four key challenges: handling noisy or irrelevant retrieved documents, integrating information from multiple sources, rejecting queries when knowledge is insufficient, and detecting or correcting factual errors in counterfactual documents. Use when the user wants to benchmark on RGB Benchmark, or asks about evaluating this task. Reports Accuracy (ACC).

Evaluates the capability of RGB-Event fusion models to perform accurate per-pixel semantic segmentation under challenging conditions such as fast motion, varying lighting, and spatiotemporal misalignment between asynchronous modalities. Use when the user wants to benchmark on DDD17, DSEC, DELIVER, M3ED, or asks about evaluating this task. Reports mIoU.

Evaluates the ability of a skeleton-based BLSTM model to recognize human actions from RGB-only video streams under limited labeled data conditions, comparing against methods that use depth or inertial modalities. Use when the user wants to benchmark on UTD-MHAD, KTH, or asks about evaluating this task. Reports top-1 accuracy.

Evaluates multi-modal visual grounding capabilities by requiring models to localize objects in images using both RGB and thermal infrared (TIR) modalities guided by text queries. It specifically probes robustness under complex real-world conditions such as low-light environments, small object sizes, and diverse weather/illumination variations. Use when the user wants to benchmark on RGBT-Ground, or asks about evaluating this task. Reports Acc@0.5.

Evaluates a model's ability to perform continuous regression for Relative Geologic Time (RGT) estimation from 2D seismic images. It probes the model's capacity to learn stratigraphic continuity and structural consistency across diverse geological settings, testing generalization from synthetic labeled data to unlabeled real-world field data. Use when the user wants to benchmark on Field Seismic Dataset, Synthetic Seismic Dataset, or asks about evaluating this task. Reports regression.

Evaluates the ability of deep learning models to classify radio galaxy morphologies and detect radio sources in continuum images. It probes transfer learning, data preprocessing robustness, and handling of class imbalance in a specialized astronomical domain. Use when the user wants to benchmark on RGZ OD, or asks about evaluating this task. Reports Top-1 accuracy.

Probes a model's ability to segment long, unstructured legal documents into semantically coherent units and assign each sentence a specific rhetorical role label (e.g., Facts, Ratio, Arguments). This capability is fundamental for downstream legal AI applications like summarization and precedent search. Use when the user wants to benchmark on LegalEval RR Dataset, or asks about evaluating this task. Reports weighted F1 score.

Evaluates online bipartite matching algorithms for rideshare platforms on their ability to balance total trip profit and group-level fairness (subgroup representation) during peak demand hours. Use when the user wants to benchmark on NYC Yellow Cabs 2013, Synthetic Rideshare, or asks about evaluating this task. Reports competitive ratio of profit.

Evaluates the ability of a decoder-only latent variable model to learn geometry-respecting latent spaces on Riemannian manifolds. It probes reconstruction fidelity, preservation of intrinsic data structures (cyclical, hierarchical, phylogenetic), and downstream predictive utility of the learned latents. Use when the user wants to benchmark on Cell cycle stages (scRNA-seq), Branching diffusion process (synthetic tree), Human mitochondrial DNA (hmtDNA), or asks about evaluating this task. Repor...

Evaluates a classifier's ability to predict the scientific rigour of academic papers (rated 4* vs non-4*) based solely on their abstracts and introductions. The setup tests whether linguistic patterns in early paper sections correlate with institutional rigour ratings. Use when the user wants to benchmark on REF dataset, ICLR dataset, ACL dataset, or asks about evaluating this task. Reports accuracy.