Claude Skills by qhjqhj00

Evaluates object-centric learning models on real-world tabletop scenes to measure their ability to segment foreground objects and background, as well as reconstruct scenes from single-image, video, or multi-view inputs. Use when the user wants to benchmark on OCTScenes-A, OCTScenes-B, or asks about evaluating this task. Reports ARI-O.

Evaluates the ability of compressed large language models to perform sequential recommendation on resource-constrained devices. It probes how well a model preserves ranking quality and recommendation accuracy after aggressive parameter compression (SVD + normalization) while maintaining low latency. Use when the user wants to benchmark on Amazon Instruments, Games, Arts, or asks about evaluating this task. Reports NDCG@10.

This benchmark probes the safety and alignment of autonomous AI agents by measuring their tendency to violate ethical, legal, or safety constraints when incentivized to optimize key performance indicators (KPIs). It evaluates whether agents prioritize task completion over moral or procedural guidelines, capturing both intentional misalignment and procedural negligence. Use when the user wants to benchmark on ODCV-Bench, or asks about evaluating this task. Reports Misalignment Rate (MR).

This benchmark evaluates a model's ability to perform multi-label classification on clinical electronic health record (EHR) notes to detect nine categories of Opioid-Related Aberrant Behaviors (ORABs). It probes the model's capacity to identify both confirmed and suggested aberrant behaviors, as well as auxiliary opioid-related signals, under conditions of significant label imbalance. Use when the user wants to benchmark on ODD, or asks about evaluating this task. Reports macro average AUPRC.

Evaluates a model's ability to classify breast MRI lesions into three clinical categories (no lesion, benign, malignant) using real-world, multi-center imaging data with high heterogeneity in scanners and protocols. It probes robustness to domain shift by comparing in-distribution cross-validation performance against an out-of-distribution test set from unseen centers. Use when the user wants to benchmark on ODELIA Breast MRI Dataset, or asks about evaluating this task. Reports Macro AUC.

Evaluates the classification accuracy and energy efficiency of a 256-neuron spiking neuromorphic processor (ODIN) on the MNIST handwritten digit dataset. It compares offline gradient-based weight training against online spike-driven synaptic plasticity (SDSP) learning, while characterizing hardware power consumption and energy per spike operation. Use when the user wants to benchmark on MNIST, or asks about evaluating this task. Reports classification accuracy.

Evaluates the ability of abstractive compression models to preserve factual correctness and answer strings when processing noisy retrieved documents in open-domain question answering. It measures how well compressed summaries retain key information to support downstream answer generation while reducing context length and inference latency. Use when the user wants to benchmark on Natural Questions (NQ), TriviaQA, PopQA, or asks about evaluating this task. Reports Exact Match (EM).

Probes an agent's ability to perform realistic, long-horizon web navigation tasks that require sustained cross-site reasoning, context maintenance across multiple tabs, and efficient action execution. It evaluates whether models can complete complex, multi-step user journeys derived from real browsing behavior within strict step budgets. Use when the user wants to benchmark on Odysseys, or asks about evaluating this task. Reports Perfect Rubrics (%).

Evaluates an LLM's ability to verify factual claims against high-volume tabular data by measuring evidence retrieval (table and data level) and claim factuality classification. It also probes whether models rely on parametric knowledge versus active retrieval and reasoning over structured data. Use when the user wants to benchmark on OECD Tabular Fact-Checking Dataset, or asks about evaluating this task. Reports Accuracy (%).

Evaluates a model's ability to perform generalized few-shot semantic segmentation on remote sensing imagery. It tests whether a model can accurately segment both previously seen (base) and new (novel) land cover classes simultaneously using only a few support examples (5-shot), probing generalization and resistance to class forgetting in low-data regimes. Use when the user wants to benchmark on OEM-GFSS, or asks about evaluating this task. Reports mIoU.

Evaluates the generalization capability of deep learning architectures on histopathology image classification, specifically probing how model capacity affects overfitting on sparse, high-resolution medical imaging data. Use when the user wants to benchmark on Oesophageal Adenocarcinomas Dataset, or asks about evaluating this task. Reports validation F1 score.

Evaluates the trade-off between improving downstream mathematical reasoning capabilities and mitigating catastrophic forgetting on general-domain knowledge benchmarks after off-policy supervised fine-tuning. It measures how well a model retains pre-trained general knowledge while learning a new specialized task. Use when the user wants to benchmark on Math500, MinervaMath, AMC23, AGIEval-Math, IMO-Bench, MMLU, MMLU-Pro, AGIEval, or asks about evaluating this task. Reports OverallAvg.

Binary classification of offensive versus non-offensive language in social media text across five languages (English, Arabic, Danish, Greek, Turkish). It probes multilingual transformer models' ability to detect hate speech and offensive content in both high-resource and low-resource settings. Use when the user wants to benchmark on SemEval-2020 Task 12 (Offenseval), or asks about evaluating this task. Reports F1-score.

Evaluates models' ability to detect offensive language in social media posts, classify the specific type of offense (e.g., insult, threat), and identify the target of the offense (individual vs. group). It tests fine-grained text classification and hierarchical annotation understanding in noisy, short-form text. Use when the user wants to benchmark on OLID, or asks about evaluating this task. Reports F1-Macro.

Evaluates Open Information Extraction (OIE) systems on their ability to correctly extract relational tuples from text, measuring precision, recall, and F2 scores under both strict and relaxed containment matching strategies. It also qualitatively classifies extraction errors to identify systematic failure modes like boundary mismatches and annotation style conflicts. Use when the user wants to benchmark on NYT-222, WEB-500, PENN-100, OIE2016, or asks about evaluating this task. Reports F2.

Compute ola13/precision_at_k via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of ola13/precision_at_k.

Evaluates models on a three-level hierarchical schema for offensive language in social media, probing the ability to detect offensiveness, categorize offense type, and identify the target of the offensive content. Use when the user wants to benchmark on Offensive Language Identification Dataset (OLID), or asks about evaluating this task. Reports macro-averaged F1-score.

Evaluates a topology-aware federated learning framework (OBL) with a satellite-assignment algorithm (CNASA) for space-air-ground integrated networks. It probes the trade-off between model accuracy and training latency under strict non-IID data distributions and varying network topologies. Use when the user wants to benchmark on MNIST, Fashion-MNIST, CIFAR-10, or asks about evaluating this task. Reports final global model accuracy.

Evaluates end-to-end OCR models on full-page transcription accuracy across diverse document types (academic papers, old scans, math, tables, multi-column layouts) and tests their ability to localize embedded images via bounding box prediction. Use when the user wants to benchmark on OlmOCR-Bench, LightOnOCR-bbox-bench, or asks about evaluating this task. Reports Overall Score.

Evaluates the ability of vision-language models and OCR tools to accurately linearize and extract structured content from complex, real-world PDFs. It probes reading order preservation, content comprehensiveness, and faithful representation of tables and equations. Use when the user wants to benchmark on olmOCR-mix-0225, or asks about evaluating this task. Reports alignment.

Evaluates advanced mathematical reasoning capabilities on Olympiad-level problems. It probes a model's ability to perform rigorous, step-by-step logical deduction and numerical verification across algebra, geometry, number theory, and combinatorics. The benchmark also assesses cross-lingual reasoning performance between English and Chinese. Use when the user wants to benchmark on OlymMATH, or asks about evaluating this task. Reports Pass@1.

This evaluation probes an agent's ability to solve complex, long-horizon mathematical problems requiring multi-step deduction, proof construction, and lemma-based reasoning. It tests performance across standard competition benchmarks (AIME, HMMT) and rigorous Olympiad-level contests (IMO, CNMO, CMO), focusing on non-geometry problems where novel proof paths are required. Use when the user wants to benchmark on AIME2025, HMMT2025 Feb, IMO2025, CNMO2025, CMO2025, or asks about evaluating this t...

This benchmark evaluates the scientific reasoning capabilities of large multimodal models (LMMs) and large language models (LLMs) on Olympiad-level mathematics and physics problems. It specifically probes bilingual (English and Chinese) text-and-image problem solving, computational correctness, and logical consistency in complex, expert-annotated scenarios. Use when the user wants to benchmark on OlympiadBench, or asks about evaluating this task. Reports micro-average accuracy.

Evaluates large vision-language models on Olympiad-level multi-image reasoning tasks across biology, chemistry, mathematics, and physics. It probes the model's ability to integrate complementary visual and textual evidence across multiple images to generate stepwise rationales and select or produce correct final answers. Use when the user wants to benchmark on OMIBench, or asks about evaluating this task. Reports accuracy.

Compute omidf/squad_precision_recall via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of omidf/squad_precision_recall.

Evaluates the kinematic feasibility and physical constraint satisfaction of retargeted humanoid motions, as well as the downstream reinforcement learning policy success rates for loco-manipulation and terrain interaction tasks. Use when the user wants to benchmark on OMOMO, In-house MoCap, LAFAN1, or asks about evaluating this task. Reports Downstream RL Policy Success Rate (%).

Evaluates the novel view synthesis and implicit scene reconstruction capabilities of NeRF-based methods on large-scale outdoor environments. It probes how well models handle diverse camera trajectories, varying lighting conditions, and different scene scales (e.g., buildings vs. cities). Use when the user wants to benchmark on OMMO, or asks about evaluating this task. Reports PSNR.

Evaluates the instruction-following and mathematical reasoning capabilities of LLMs fine-tuned with a dual-perspective preference optimization method. It measures conversational quality, adherence to instructions, and problem-solving accuracy across diverse open-ended and quantitative benchmarks. Use when the user wants to benchmark on AlpacaEval 2.0, Arena-Hard v0.1, IFEval, SedarEval, GSM8K, MATH 500, AIME 2024, AMC 2023, or asks about evaluating this task. Reports LC(%).

Evaluates large language models on rigorous, Olympiad-level mathematical reasoning across diverse domains and difficulty levels. It probes the model's ability to perform complex logical deduction, multi-step problem solving, and handle non-standard answer formats without relying on trivial or non-mathematical content. Use when the user wants to benchmark on Omni-MATH, or asks about evaluating this task. Reports accuracy.

Evaluates a general-purpose NeRF framework on downstream 3D tasks, including real-time novel view synthesis, parameter-efficient 3D scene understanding, and text-guided 3D editing. It probes the model's ability to generalize to unseen scenes and adapt to various geometric and appearance tasks with minimal fine-tuning. Use when the user wants to benchmark on DTU, ScanNet, or asks about evaluating this task. Reports PSNR.

Evaluates the safety alignment and refusal capabilities of Audio-Visual Large Language Models (OLLMs) when exposed to harmful unimodal, dual-modal, and omni-modal inputs. It specifically probes whether models maintain consistent safety boundaries across modality combinations and reveals vulnerabilities in cross-modal comprehension-aware safety. Use when the user wants to benchmark on Omni-SafetyBench, or asks about evaluating this task. Reports Safety-score.

Evaluates the ability of AI-generated image detectors to generalize across different semantic domains (human, animal, object, scene) and resist modern, photorealistic generative models. It probes whether models rely on content-agnostic artifacts versus semantic features for robust real-vs-fake classification. Use when the user wants to benchmark on GenImage, Chameleon, Mirage-Test, or asks about evaluating this task. Reports Accuracy (%).

Evaluates multimodal large language models' ability to perform visual-to-clinical reasoning on brain imaging data. It probes capabilities ranging from basic anatomical identification to complex multi-stage clinical decision-making and prognosis prediction. Use when the user wants to benchmark on OmniBrainBench, or asks about evaluating this task. Reports accuracy.

Evaluates end-to-end document parsing capabilities, including text recognition, formula recognition, table structure extraction, and reading order prediction across diverse document types and languages. Use when the user wants to benchmark on OmniDocBench, or asks about evaluating this task. Reports OverallEdit.

This evaluation protocol assesses the ability of omni-modal large language models to avoid hallucinating non-existent audio or visual content when presented with contradictory or incomplete multimodal inputs. It specifically probes whether models can correctly identify real elements while resisting false affirmations of missing ones across text, vision, and audio modalities. Use when the user wants to benchmark on AVHBench, CMM, or asks about evaluating this task. Reports F1 Score, Hallucinat...

Evaluates embodied agent reasoning by testing how well models infer capability gaps, dynamic tool acquisition, and coordination needs from environmental constraints. Probes the ability to ground abstract reasoning in physical reality under partial observability. Use when the user wants to benchmark on EAR-Bench, or asks about evaluating this task. Reports Success Rate (SR).

Evaluates human fall detection and action recognition capabilities across controlled (staged) and uncontrolled (wild) video domains. It probes a model's ability to classify a 10-class activity taxonomy, detect binary fall/fallen states, and segment action timelines, while measuring generalization gaps between in-distribution and out-of-distribution settings. Use when the user wants to benchmark on CMDFall, UP-Fall, Le2i, GMDCSA24, EDF, OCCU, CaucaFall, MCFD, OOPS-Fall, or asks about evaluatin...

Evaluates optical flow estimation models on synthetic omnidirectional human motion data. It probes the model's ability to handle fisheye distortions, domain-randomized environments, and varying amounts of fine-tuning data. Use when the user wants to benchmark on OmniFlow, or asks about evaluating this task. Reports optical flow error.

Evaluates native omni-modal AI agents' ability to perform multi-hop cross-modal reasoning across video, audio, and image inputs. It probes their capacity to integrate external tools (web search, browser, code execution) for evidence gathering and to produce verifiable open-form answers under varying task difficulties. Use when the user wants to benchmark on OmniGAIA, or asks about evaluating this task. Reports Pass@1.

Evaluates a unified multimodal model's capabilities across visual understanding, text-to-image generation, instruction-based image editing, and in-context generation. It probes compositional prompt following, long-prompt adherence, edit accuracy versus preservation, and subject consistency across single, multiple, and scene contexts. Use when the user wants to benchmark on MMBench, MMMU, MM-Vet, GenEval, DPG-Bench, Emu-Edit, GEdit-Bench-EN, ImgEdit-Bench, OmniContext, or asks about evaluating...

Evaluates genomic foundation models on diverse in-silico tasks including RNA structure prediction, plant DNA regulation, cross-species genomic understanding, and regulatory element classification. It probes the models' ability to generalize across nucleic acid types, species, and complex sequence motifs. Use when the user wants to benchmark on RGB, PGB, GUE, GB, or asks about evaluating this task. Reports macro F1.

This benchmark evaluates a model's ability to perform language-based object detection using dynamic, open-vocabulary label spaces. It specifically probes handling of free-form text descriptions, negative examples (descriptions referring to zero objects), and multi-instance references within a single image. Use when the user wants to benchmark on OmniLabel, or asks about evaluating this task. Reports harmonic_mean_AP.

Evaluates multilingual automatic speech recognition (ASR) capabilities across 1,600+ languages, including zero-shot generalization to previously unsupported languages. It measures transcription accuracy under varying resource conditions and compares performance against established baselines like Whisper, USM, and MMS. Use when the user wants to benchmark on MMS-Lab, FLEURS, MLS, Common Voice 22, or asks about evaluating this task. Reports CER.

This benchmark probes an agent's ability to perform long-horizon question answering over audio-video streams. It specifically tests fine-grained multimodal retrieval, multi-turn tool calling, and budget-aware reasoning when key evidence is scattered across time. Use when the user wants to benchmark on OmniVideoBench, WorldSense, Daily-Omni, or asks about evaluating this task. Reports accuracy.

Evaluates multimodal retrieval capabilities across text, image, video, and audio modalities, including composed queries. It probes the model's ability to align heterogeneous media types and rank relevant candidates under varying modality combinations. Use when the user wants to benchmark on Extended M-BEIR, MMEBv2, ACM (Audio-Centric Multimodal Benchmark), or asks about evaluating this task. Reports Recall@5.

Evaluates the semantic alignment and factual fidelity of automatically generated scientific image captions. It probes whether dense, context-aware captions can replace visual inputs for downstream reasoning tasks and how well they capture complex scientific figures compared to raw human-written captions. Use when the user wants to benchmark on OmniScience, AI2D, MMMU, MM-MT-Bench, MSEarth, or asks about evaluating this task. Reports cross-modal relevance score.

This benchmark evaluates vision-language models on advanced spatial reasoning capabilities, specifically probing dynamic reasoning, complex spatial logic, spatial interaction, and perspective-taking. It measures how well models understand and manipulate spatial relationships, temporal changes, and viewpoint shifts beyond basic object recognition. Use when the user wants to benchmark on OmniSpatial, or asks about evaluating this task. Reports accuracy.

Evaluates a foundation model's ability to process native-resolution astronomical spectra of variable lengths without resampling, and assesses its zero-shot, few-shot, and supervised performance on stellar property estimation and source classification tasks across diverse spectroscopic surveys. Use when the user wants to benchmark on OmniSpectra Multi-Survey Corpus, or asks about evaluating this task. Reports Mean-Squared Error.

Evaluates the out-of-the-box predictive performance of tree-based models, neural networks, and foundation models on a large-scale collection of real-world tabular datasets. It also analyzes how dataset metafeatures (e.g., size, feature distribution, target skewness) correlate with model success to identify which model category excels under specific data conditions. Use when the user wants to benchmark on OmniTabBench, or asks about evaluating this task. Reports performance score.

Evaluates multimodal large language models' ability to jointly reason across visual and audio modalities in long-duration videos. It probes capabilities like cross-modal alignment, temporal dependency modeling, and understanding of low-semantic acoustic cues such as music and ambient sounds. Use when the user wants to benchmark on OmniVideoBench, or asks about evaluating this task. Reports accuracy.