Claude Skills by qhjqhj00

Evaluates archival domain adaptation capabilities across four distinct tasks. It probes a model's ability to predict document retention periods, classify open access status, determine confidentiality levels, and correct post-OCR text errors in Chinese archival records. Use when the user wants to benchmark on AMBLE, or asks about evaluating this task. Reports F1 score, Levenshtein Distance.

Evaluates the clinical reasoning capabilities and on-device runtime efficiency of various LLMs using the AMEGA benchmark. It measures response accuracy via an LLM-as-a-judge scoring system and tracks inference throughput and thermal throttling effects across different mobile hardware configurations. Use when the user wants to benchmark on AMEGA, or asks about evaluating this task. Reports AMEGA score.

Evaluates fine-tuned Whisper models for Amharic speech-to-text recognition by measuring transcription accuracy at word and character levels, alongside n-gram overlap. It also probes the impact of homophone normalization and zero-shot generalization on low-resource language ASR performance. Use when the user wants to benchmark on FLEURS Amharic, BDU Speech Corpus, Mozilla Common Voice v17.0 Amharic, or asks about evaluating this task. Reports WER.

Evaluates long-sequence narrative understanding and cultural variation in Amharic using story-based question answering. Probes both multiple-choice and generative QA capabilities across different Ethiopian regional folktales. Use when the user wants to benchmark on AmharicStoryQA, or asks about evaluating this task. Reports accuracy.

Probes long-horizon agentic planning, cross-image grounding, and uncertainty-driven question selection. Models must iteratively ask constrained Yes/No/Unsure questions to identify a hidden target from a gallery of visually similar dress images while strictly tracking constraints and avoiding prohibited attributes. Use when the user wants to benchmark on AMIGO, or asks about evaluating this task. Reports identification success.

Evaluates large language models' ability to solve high school and IMO-level mathematics competition problems. It probes complex mathematical reasoning, problem-solving under strict constraints, and the model's capacity to scale reasoning effort with test-time compute. Use when the user wants to benchmark on AMO-Bench, or asks about evaluating this task. Reports AVG@32.

Evaluates a model's ability to infer occluded (amodal) 3D geometry and predict physically stable configurations in cluttered tabletop scenes. It further tests downstream robotic manipulation success (grasping, pushing, rearranging) under varying levels of visual occlusion. Use when the user wants to benchmark on ShapeNet, MuJoCo Cluttered Tabletop Benchmark, or asks about evaluating this task. Reports Chamfer distance.

Evaluates a model's ability to predict multi-layered pixel-level motion fields that explicitly account for both visible and occluded regions of objects (amodal optical flow), along with associated masks and semantic labels. It also assesses the utility of these predictions for downstream panoptic tracking. Use when the user wants to benchmark on AmodalSynthDrive, or asks about evaluating this task. Reports AFQ.

Evaluates the downstream performance of pretrained text encoders on a suite of natural language understanding and reading comprehension benchmarks via standard single-task fine-tuning. Use when the user wants to benchmark on GLUE, SQuAD 2.0, or asks about evaluating this task. Reports AVG.

Evaluates the ability of reprogrammed language models to classify antimicrobial peptide (AMP) sequences into binary categories (toxic vs. non-toxic, or AMP vs. non-AMP) using limited labeled data. Use when the user wants to benchmark on AMP Dataset, or asks about evaluating this task. Reports Test Accuracy.

Evaluates a physics-based character's ability to learn stylized locomotion and complex task execution (e.g., navigating targets, avoiding obstacles) by imitating unstructured motion datasets. It probes the model's capacity to compose disparate skills, generalize across gaits, and maintain high-fidelity motion tracking without manual motion planning. Use when the user wants to benchmark on AMP Motion Datasets, or asks about evaluating this task. Reports normalized task return.

Evaluates a language model's ability to perform analogical reasoning and select the correct word pair from multiple choices under temperature scaling. Use when the user wants to benchmark on Analogy Multiple Choice, or asks about evaluating this task. Reports accuracy.

Evaluates the scoring accuracy of interpretable automated scoring frameworks on educational assessment items across three domains. It measures how well LLM-extracted features and ordinal logistic regression align with human raters while adhering to strict interpretability constraints. Use when the user wants to benchmark on Educational Assessment Items (Science, Reading Informational Text, Reading Literature), or asks about evaluating this task. Reports QWK.

Evaluates Named Entity Recognition (NER) capabilities across five regional dialects of the Bangla language. It probes a model's ability to correctly identify and classify entities (Person, Location, Organization, Role, Food) in dialect-specific text where linguistic features and vocabulary differ significantly from standard Bangla. Use when the user wants to benchmark on ANCHOLIK-NER, or asks about evaluating this task. Reports F1-score.

Compute the anderson_ksamp metric — provided by scipy.stats. Use when the user has predictions and ground-truth and needs to compute anderson_ksamp, or asks how to score with anderson_ksamp.

Compute the anderson metric — provided by scipy.stats. Use when the user has predictions and ground-truth and needs to compute anderson, or asks how to score with anderson.

Evaluates the ability of Graph Neural Networks to classify Android applications as benign or malicious, and to identify specific malware families or categories, by learning topological patterns from function call graphs. Use when the user wants to benchmark on Malnet-Tiny, Drebin, or asks about evaluating this task. Reports Accuracy.

Evaluates the ability of UI control agents to execute mobile app tasks by predicting correct actions given textual screen representations and instruction history. It probes in-domain and out-of-domain generalization, and measures how model performance scales with the volume of training demonstrations. Use when the user wants to benchmark on AndroidControl, or asks about evaluating this task. Reports step-wise accuracy.

Evaluates a GUI agent's ability to perform long-horizon, multi-app tasks in a mobile environment. It probes the agent's planning and skill-retrieval capabilities across complex, real-world application scenarios. Use when the user wants to benchmark on AndroidLH, or asks about evaluating this task. Reports task success rate.

This benchmark evaluates the ability of autonomous multimodal agents to navigate and interact with real-world Android applications to complete programmatic user instructions. It probes UI understanding, precise touch interaction, state tracking, and error recovery in a dynamic mobile environment. Use when the user wants to benchmark on AndroidWorld, or asks about evaluating this task. Reports Success Rate (SR).

Probes the zero-shot generalization capability of mobile agents trained via online reinforcement learning across increasingly challenging unseen scenarios in Android environments, including new task instances, UI templates, and entirely new applications. It measures how well learned interaction policies transfer to novel contexts without additional supervised fine-tuning. Use when the user wants to benchmark on AndroidWorld-Generalization, or asks about evaluating this task. Reports Success R...

Evaluates fine-grained compositional reasoning over untrimmed videos by requiring models to interpret spatio-temporal scene graphs and answer complex questions involving attributes, actions, and temporal relationships. Use when the user wants to benchmark on ANetQA, or asks about evaluating this task. Reports accuracy.

Compute angelina-wang/directional_bias_amplification via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of angelina-wang/directional_bias_amplification.

Evaluates automated end-to-end video dubbing systems by testing their ability to generate synchronized English audio from Japanese source video, specifically probing prosody matching, timing alignment, and multi-speaker isolation capabilities. Use when the user wants to benchmark on Anim-400K, or asks about evaluating this task. Reports MUSHRA.

This benchmark evaluates the ability of deep learning models to estimate 3D pose and shape of diverse mammal species from single images. It probes cross-species generalization, synthetic-to-real transfer, and the adaptation of human-centric pose estimation architectures to non-human anatomies. Use when the user wants to benchmark on Animal3D, or asks about evaluating this task. Reports S-MPJPE.

This benchmark evaluates video generation models on character-centric animation capabilities, specifically probing IP preservation, motion expressiveness, deformation accuracy, and multi-angle consistency. It operationalizes animation principles into measurable dimensions to identify gaps missed by standard realism-focused benchmarks. Use when the user wants to benchmark on AnimationBench, or asks about evaluating this task. Reports AnimationBench score.

Evaluates Vision Language Models' ability to perceive and categorize basic UI animation types from short video clips. It probes motion perception and recognition of primitive visual effects like movement, rotation, scaling, color change, fading, blurring, and morphing. Use when the user wants to benchmark on AniMINT, or asks about evaluating this task. Reports accuracy.

Evaluates the quality and controllability of AI-generated animation videos, specifically probing character consistency, style consistency, and distortion detection. It addresses the unique challenges of non-photorealistic content, exaggerated motion, and artistic coherence that standard video benchmarks often miss. Use when the user wants to benchmark on AniSora Benchmark, or asks about evaluating this task. Reports character consistency.

This benchmark evaluates approximate nearest neighbor (ANN) search algorithms by measuring the trade-off between search quality (recall) and computational efficiency (queries per second, index size, and build time). It probes how well different algorithmic families perform across diverse high-dimensional datasets and distance metrics, revealing robustness and approximation capabilities. Use when the user wants to benchmark on SIFT, GIST, GLOVE, NYTimes, Rand-Euclidean, SIFT-Hamming, Word2Bits...

Measures the data loading throughput and epoch iteration time for large-scale biological datasets. It benchmarks how efficiently a loader can fetch and prepare mini-batches from disk compared to existing frameworks. Use when the user wants to benchmark on Tahoe100M, 1000 Genomes GRCh38, Single-cell microscopy images, or asks about evaluating this task. Reports samples/sec.

This benchmark evaluates the detection accuracy and computational efficiency of classical machine learning, tree-based, and deep learning anomaly detection algorithms across diverse multivariate and univariate datasets. It probes how well different models handle class imbalance, varying anomaly prevalence, and differing requirements for labeled anomaly data during training. The evaluation also measures training time and resource consumption to assess real-world deployment feasibility. Use whe...

Evaluates unsupervised and semi-supervised time series anomaly detection pipelines across multiple real-world and benchmark datasets. It measures how well different models identify known anomalous segments in telemetry, production traffic, and synthetic signals. Use when the user wants to benchmark on NAB, NASA, YAHOO, or asks about evaluating this task. Reports F1 score.

Evaluates anomaly detection algorithms on a large corpus of synthetic datasets systematically varied along four dimensions: point difficulty, semantic variation, relative frequency, and feature relevance. It probes algorithm robustness, generalization across diverse anomaly-generating processes, and the impact of experimental design choices on reported performance. Use when the user wants to benchmark on Synthetic Anomaly Detection Corpus, or asks about evaluating this task. Reports AUC.

Evaluates the robustness and calibration stability of anomaly detection models across heterogeneous cloud telemetry datasets under strict no-leakage conditions. Probes how architectures handle distribution shift, high dimensionality, and label sparsity without test-time label access. Use when the user wants to benchmark on NAB, Microsoft Cloud Monitoring, Exathlon, IBM Console dataset, or asks about evaluating this task. Reports normalized NAB score.

This benchmark evaluates log-based anomaly detection models by measuring their ability to classify log sequences as normal or anomalous. It specifically probes how well different model paradigms (classical ML, supervised/unsupervised deep learning, and LLM-based) generalize when trained on code-guided synthetic data augmentation across varying augmentation ratios. Use when the user wants to benchmark on HDFS, Zookeeper, or asks about evaluating this task. Reports F1-score.

This evaluation probes a semi-supervised anomaly detection model's ability to identify rare or visually distinct objects in highly imbalanced image datasets. It measures how effectively the model ranks anomalies at the top of its predictions using limited initial labels and iterative active learning cycles. Use when the user wants to benchmark on miniImageNet, GalaxyMNIST, Galaxy Zoo 2 (Kaggle Challenge subset), or asks about evaluating this task. Reports AUROC.

This benchmark evaluates the robustness of LLM-based tutoring models against adversarial student agents designed to elicit final answers. It measures how easily tutors disclose solutions under various attack strategies and tracks the dialogue length required for answer leakage across math, multiple-choice, and coding domains. Use when the user wants to benchmark on GSM8K, MMLU, HumanEval, or asks about evaluating this task. Reports answer leakage rate.

Measures the causal influence of activation-based interventions (linear directions or multidimensional cones) on an LLM's factual reasoning. It quantifies how effectively steering or ablating specific neural subspaces switches model outputs from truthful to untruthful across a set of propositional prompts. Use when the user has predictions and gold and needs to compute Answer Switching Rate (ASR).

Evaluates multi-perspective answer summarization for community question answering, probing content selection, perspective clustering, abstractive summarization, and factual consistency/coverage. Use when the user wants to benchmark on AnswerSumm, or asks about evaluating this task. Reports F1, ROUGE-1/2/L.

Evaluates out-of-distribution generalization of protein language models and sequence CNNs for therapeutic antibody design across different antigen targets and generative models. It probes robustness to covariate shifts, label shifts, and assay biases in molecular sequence data. Use when the user wants to benchmark on Antibody DomainBed, or asks about evaluating this task. Reports accuracy.

Evaluates a model's ability to predict anticancer drug responses (IC50 scores) between drugs and cell lines. It probes the model's capacity to perform weighted link prediction/regression on a multimodal graph combining drug and cell line similarities. Use when the user wants to benchmark on CCLE Dataset, or asks about evaluating this task. Reports MSE.

Evaluates the ability of image editing models to follow natural language instructions to modify images while preserving unedited regions and maintaining semantic/visual consistency. It probes alignment with complex editing intents, content preservation, and robustness across diverse editing types including implicit and visual-conditioned tasks. Use when the user wants to benchmark on Emu Edit Test, MagicBrush, AnyEdit-Test, or asks about evaluating this task. Reports CLIPim.

Evaluates the ability of text-to-image models to accurately render specified multilingual text (English and Chinese) in arbitrary shapes and positions while maintaining visual realism and seamless background integration. Use when the user wants to benchmark on AnyText-benchmark, or asks about evaluating this task. Reports Sen. ACC.

Evaluates a unified diffusion transformer model's ability to generate high-fidelity audio and music conditioned on diverse modalities (text, video, image, audio). It measures acoustic similarity, generation quality/diversity, and cross-modal semantic alignment across multiple standard audio generation benchmarks. Use when the user wants to benchmark on AudioCaps, VGGSound, AVVP, MusicCaps, V2M-bench, or asks about evaluating this task. Reports FAD.

Evaluates an agent's ability to retrieve and invoke relevant APIs from a large-scale pool to resolve user queries. It probes hierarchical API retrieval, self-reflective error recovery, and the capacity to handle context limits when dealing with thousands of available tools. Use when the user wants to benchmark on ToolBench (filtered), AnyToolBench, or asks about evaluating this task. Reports pass rate.

Compute anz2/iliauniiccocrevaluation via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of anz2/iliauniiccocrevaluation.

Evaluates automatic post-editing (APE) models by measuring how effectively they correct machine-translated text to align with human references. It probes the system's ability to fix translation artifacts, preserve source semantics, and adapt to different domains and translation technologies. Use when the user wants to benchmark on WMT'18 SMT, SubEdits, MLQE-PE, or asks about evaluating this task. Reports BLEU.

Evaluates the effectiveness of automatically generated prompts (instructions) from the APE framework compared to human-designed or baseline prompts across various natural language processing tasks. Use when the user wants to benchmark on Instruction Induction, BIG-Bench Instruction Induction (BBII), MultiArith, GSM8K, or asks about evaluating this task. Reports zero-shot execution accuracy.

Evaluates the ability of NLP models to detect hate speech in Korean text. It specifically probes domain-agnostic generalizability and resistance to common inductive biases like text length or topic distribution. Use when the user wants to benchmark on APEACH, or asks about evaluating this task. Reports accuracy.

Evaluates models on complex, real-world professional reasoning tasks across four domains (investment banking, management consulting, big law, primary care). It probes document analysis, multi-step reasoning, and domain-specific judgment under practical constraints. Use when the user wants to benchmark on APEX-v1.0, or asks about evaluating this task. Reports autograded scores.