Claude Skills by qhjqhj00

Evaluates text fidelity and typography accuracy in AI-generated images by measuring spelling, case sensitivity, repetition, and structural inconsistencies against reference prompts. Use when the user has predictions and gold and needs to compute ABHINAW Score.

Compute abidlabs/mean_iou via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of abidlabs/mean_iou.

Compute abidlabs/mean_iou2 via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of abidlabs/mean_iou2.

Detects abnormal driving behaviors in naturalistic driving data using event-level safety indicators and motion features. It evaluates a semi-supervised machine learning model's ability to distinguish between normal and anomalous driving events based on vehicle dynamics and temporal proximity metrics. Use when the user wants to benchmark on Naturalistic Driving Dataset, or asks about evaluating this task. Reports F1-score.

Evaluates molecular generative models by assessing their ability to recreate known ligands, predict drug-target affinity, and bind to target proteins via molecular docking. It probes the biological relevance and structural fidelity of de novo generated molecules across multiple protein targets. Use when the user wants to benchmark on ABRAHAM, or asks about evaluating this task. Reports ROOM recreation metric.

Probes the ability of depth estimation models to accurately predict distances for specific semantic classes, particularly focusing on thin structures like wires and cables. It measures class-specific absolute relative error to highlight performance on challenging, low-pixel-count obstacles relevant to drone navigation. Use when the user has predictions and gold and needs to compute AbsRel_class.

Probes an LLM's ability to identify granular sentiment toward specific topics within a text, including custom labels like 'not mentioned' and 'wished for'. It tests fine-grained aspect-level classification rather than overall review sentiment. Use when the user wants to benchmark on TravelBench ABSA, or asks about evaluating this task. Reports F1-score.

Evaluates large language models' ability to abstain from answering when questions are unanswerable or when uncertain, while maintaining accuracy on answerable questions. It measures how well models balance abstention with correct answer selection under different prompting strategies and uncertainty calibration methods. Use when the user wants to benchmark on Abstain-QA, or asks about evaluating this task. Reports Abstention Rate (AR).

Evaluates multimodal models' ability to comprehend and reason over synthetic abstract images, including charts, tables, road maps, dashboards, relation graphs, flowcharts, visual puzzles, and planar layouts. Use when the user wants to benchmark on Synthetic Abstract Image Benchmark, or asks about evaluating this task. Reports accuracy.

Evaluates a model's ability to predict multi-modal future trajectories of agents given historical positions. It probes the model's capacity to capture social interactions, scene constraints, and long-term motion dynamics across diverse environments. Use when the user wants to benchmark on ETH, UCY, Stanford Drone Dataset (SDD), STATS SportVU NBA, Intersection Drone Dataset (inD), TrajNet++, or asks about evaluating this task. Reports TopK ADE, TopK FDE.

Evaluates the ability of sentiment analysis models to classify short social media posts and movie reviews into discrete sentiment categories. It probes how well supervised and unsupervised word representations capture task-specific sentiment orientation. Use when the user wants to benchmark on ACA, Stanford Sentiment Treebank (SST), or asks about evaluating this task. Reports Accuracy.

Evaluates an LLM's ability to perform long-context summarization and synthesize related work sections by retrieving and reasoning over heterogeneous academic memory chunks. Use when the user wants to benchmark on AcademicEval-abstract, AcademicEval-related, or asks about evaluating this task. Reports F1 score.

This benchmark evaluates audio-visual singing voice separation models by measuring how accurately they isolate target singing voices from mixed audio accompanied by video. It probes the model's ability to leverage visual motion cues (face landmarks) to separate overlapping or low-volume singing voices across different languages and volume conditions. Use when the user wants to benchmark on Acappella, or asks about evaluating this task. Reports SDR.

Evaluates a zero-shot text-to-speech system's ability to generate speech with high-fidelity target accents while preserving the reference speaker's voice. It probes the model's capacity to disentangle accent characteristics from speaker identity using continuous embeddings, measuring both objective acoustic similarity and subjective listener preference across inherent and cross-accent generation tasks. Use when the user wants to benchmark on Common Voice v17.0 (English), VCTK, LibriTTS-R (cle...

Evaluates ASR models on African-accented clinical speech to measure how well they transcribe medical named entities (MNEs) like drug names, diagnoses, and lab results. It specifically probes the gap between standard word-level accuracy and clinically relevant entity recognition. Use when the user wants to benchmark on AfriSpeech, or asks about evaluating this task. Reports M-WER.

This benchmark probes systematic disability bias in large language models by comparing responses to neutral queries versus disability-aware queries. It measures whether explicitly mentioning a disability degrades response quality across sentiment, social perception, and factual accuracy dimensions. Use when the user wants to benchmark on AccessEval, or asks about evaluating this task. Reports Bias Degradation Rate ($\Delta_M$).

Evaluates a model's ability to estimate long-range dense optical flow between distant video frames, specifically testing robustness to large motions and severe occlusions. It measures how well the model accumulates flow over multiple steps while correcting misalignments and occlusion artifacts. Use when the user wants to benchmark on CVO, HS-Sintel, or asks about evaluating this task. Reports EPE.

Evaluates machine learning models' ability to detect accounting fraud using financial statement ratios across different industry sectors. It probes the models' predictive accuracy, sensitivity to fraud cases, and robustness to class imbalance and industry-specific data distributions. Use when the user wants to benchmark on SIC Industry Financial Fraud Dataset, or asks about evaluating this task. Reports Accuracy.

Compute the accuracy_score metric — provided by scikit-learn. Use when the user has predictions and ground-truth and needs to compute accuracy_score, or asks how to score with accuracy_score.

Probes the pairwise ranking accuracy of an AI judge system when evaluating generated commit messages against a heuristic ground truth derived from multiple automatic text generation metrics. Use when the user has predictions and gold and needs to compute accuracy.

Probes the physical plausibility and spatiotemporal flow preservation of data-driven weather forecasting models by directly measuring errors in horizontal (advection) and vertical (convection) atmospheric motions, rather than relying on pixel-wise accuracy metrics that reward blurriness. Use when the user has predictions and gold and needs to compute ACE.

Probes molecular representation models on their ability to predict chemical properties and classify molecular structures. It evaluates how well pre-trained embeddings capture task-relevant chemical motifs when probed with linear classifiers or regressors. Use when the user wants to benchmark on MoleculeNet, Photoswitch, Synthetic Toxicity Benchmark, or asks about evaluating this task. Reports %AUCROC, MAE.

This evaluation protocol assesses the mathematical reasoning and code generation capabilities of large language models. It probes the model's ability to solve competitive math problems and implement algorithms for coding contests under strict generation constraints. Use when the user wants to benchmark on AIME2024, AIME2025, MATH500, HMMT2025 Feb, BRUMO2025, LiveCodeBench v5, LiveCodeBench v6, Codeforces (LiveCodeBench Pro), EvalPlus, or asks about evaluating this task. Reports avg@k.

Evaluates the ability of pretrained models to recognize named entities in open-domain text, specifically probing generalization when name regularity and mention coverage are manipulated. It measures how well models rely on contextual patterns versus superficial spelling cues. Use when the user wants to benchmark on ACE2005, or asks about evaluating this task. Reports Micro-F1.

Evaluates machine translation metrics on their ability to correctly rank good translations above incorrect ones across specific linguistic error phenomena. It probes metric robustness to fine-grained translation errors like hallucination, omission, and real-world knowledge failures. Use when the user wants to benchmark on ACES, or asks about evaluating this task. Reports Kendall's tau-like correlation.

Evaluates the execution speed and dispatching efficiency of different interpreter implementations (AST vs. bytecode variants) for a simple imperative language (ACOL) in Prolog. Use when the user wants to benchmark on ACOL Interpreter Benchmarks, or asks about evaluating this task. Reports geometric_mean_runtime.

Evaluates language and reasoning models on open-ended, generative planning tasks derived from PDDL domains. It probes capabilities like action applicability, reachability, progression, justification, and next-action prediction without predefined answer choices. Use when the user wants to benchmark on ACPBench Hard, or asks about evaluating this task. Reports accuracy.

Evaluates a model's ability to identify acronym boundaries and their corresponding long forms in scientific text, and to disambiguate ambiguous acronyms by selecting the correct long form from a candidate set. Use when the user wants to benchmark on SciAI, SciAD, or asks about evaluating this task. Reports Macro F1.

Tests translating a user's natural language command into a structured executable action (function, arguments, status) for GUI interaction. It bridges high-level intent with precise, structured API-like calls. Use when the user wants to benchmark on GUI-360°-Bench, or asks about evaluating this task. Reports Step success rate.

Evaluates a text-to-image model's ability to customize generated images with a specific subject's appearance while accurately transferring a target action from an exemplar image, without appearance leakage or subject deformation. Use when the user wants to benchmark on ActionBench, or asks about evaluating this task. Reports total accuracy.

Evaluates the effectiveness of active learning policies in selecting a minimal subset of prompts to accurately predict an LLM's overall benchmark score, thereby reducing evaluation costs while maintaining predictive fidelity. Use when the user wants to benchmark on HuggingFace Open LLM Leaderboard, MMLU, HELM-Lite, AlpacaEval 2.0, Chatbot Arena, or asks about evaluating this task. Reports absolute differences.

Evaluates the ability of large vision-language models to mitigate object-existence hallucinations by dynamically allocating visual computation based on uncertainty. It probes fine-grained perception, object counting, spatial reasoning, and color recognition under adaptive visual grounding. Use when the user wants to benchmark on POPE, MME, CHAIR, or asks about evaluating this task. Reports POPE Accuracy.

Evaluates the accuracy of 3D geometry reconstruction from multi-view images using active pattern projection. It measures how closely the predicted point cloud matches the ground truth geometry and assesses robustness under varying view counts and camera-projector baselines. Use when the user wants to benchmark on NeRF derivative (synthetic), Real-world capture (RealSense D415), or asks about evaluating this task. Reports Chamfer Distance (mm).

Evaluates a generative model's ability to synthesize realistic human mobility patterns by producing activity chains and location trajectories conditioned on socio-demographic and household attributes. It probes the model's capacity to capture temporal dynamics, activity type distributions, transition probabilities, and household interdependencies at both the sequence and system levels. Use when the user wants to benchmark on Household Travel Survey (HTS) / NHTS, or asks about evaluating this ...

Evaluates a model's ability to recognize human activities in real-time by simultaneously learning from skeletal pose data and object attributes. It probes the integration of multi-modal cues (color, shape, distance, or object probabilities) for accurate and efficient activity classification in robotics scenarios. Use when the user wants to benchmark on Cornell Activity Dataset (CAD-60), MSR Daily Activity 3D Dataset, or asks about evaluating this task. Reports accuracy.

Evaluates fine-grained temporal and compositional alignment in video-text models by testing their ability to distinguish between videos and captions that contain subtle structural disruptions. It probes sensitivity to temporal reordering, action word replacement, and segment-level misalignment, as well as the model's robustness to combined disruptions. Use when the user wants to benchmark on ActivityNet-Comp, YouCook2-Comp, or asks about evaluating this task. Reports binary classification acc...

This benchmark evaluates a model's ability to perform speech role-playing by generating persona-consistent, emotionally grounded audio responses. It specifically probes the model's capacity for accurate voice impersonation, precise content delivery, and alignment with target emotional prosody in a conversational context. Use when the user wants to benchmark on ActorMindBench, or asks about evaluating this task. Reports RP-MOS.

Evaluates the predictive accuracy and decision-making value of ad targeting policies using different information sets (contextual, geographical, behavioral). It specifically tests whether geographical and behavioral data act as complements or substitutes in improving click-through rates, accounting for user exposure history. Use when the user wants to benchmark on Real-world Ad Impression Dataset, or asks about evaluating this task. Reports IPS policy value.

Evaluates multimodal large language models on autonomous driving tasks under adverse weather and complex scenes. It probes base and advanced visual perception, relational understanding, event reasoning, and the coherence of hierarchical chain-of-thought reasoning. Use when the user wants to benchmark on AD^2-Bench, or asks about evaluating this task. Reports Avg-S.

Evaluates visual anomaly detection models for autonomous driving by measuring their ability to detect and precisely localize defects or hazards in road scenes. It probes the trade-off between detection accuracy, pixel-level localization precision, and computational efficiency for onboard deployment. Use when the user wants to benchmark on AD4AD (AnoVox), or asks about evaluating this task. Reports P-AP.

Evaluates a reinforcement learning-based adaptive JPEG compression framework for cloud computer vision services. It measures how effectively the system balances image file size reduction against the accuracy degradation of downstream black-box vision models, while accounting for end-to-end latency overhead compared to standard JPEG baselines. Use when the user wants to benchmark on ImageNet, DNIM, or asks about evaluating this task. Reports relative top-5 accuracy.

Evaluates the inference speedup and output consistency of adaptive layer parallelism for LLM decoding compared to standard autoregressive generation. Use when the user wants to benchmark on HumanEval, or asks about evaluating this task. Reports speedup.

Evaluates multi-task model merging methods on image classification tasks by measuring average accuracy across multiple datasets, generalization to unseen tasks, and robustness to image corruptions. Use when the user wants to benchmark on Image Classification Bench (SUN397, Cars, RESISC45, EuroSAT, SVHN, GTSRB, MNIST, DTD), or asks about evaluating this task. Reports Avg Acc.

Evaluates the privacy vulnerability of adapter-based federated large language models against gradient inversion attacks. It measures how accurately an adversary can reconstruct private training text from shared adapter gradients under varying batch sizes, model architectures, and defensive mechanisms. Use when the user wants to benchmark on CoLA, SST, Rotten Tomatoes, or asks about evaluating this task. Reports ROUGE-1.

This evaluation probes the effectiveness of diversity-based adaptive test selection strategies for black-box LLM applications. It measures how quickly and reliably different prioritization methods detect failures in prompt templates compared to random baselines, while also assessing the diversity of generated outputs. Use when the user wants to benchmark on BBH & P3 Prompt Templates, or asks about evaluating this task. Reports APFD.

Evaluates retrieval and answer generation methods across structured financial, legal, and medical documents. It probes how well different architectures handle varying query complexities, cross-references, and domain-specific structural requirements. Use when the user wants to benchmark on Controlled Multi-Domain Corpus, FinanceBench, or asks about evaluating this task. Reports Quality.

Evaluates the training efficiency and convergence accuracy of sparse deep learning optimizers on large-scale, high-dimensional multi-class classification tasks with extreme label sparsity. Use when the user wants to benchmark on Amazon-670k, Delicious-200k, or asks about evaluating this task. Reports time-to-accuracy.

Evaluates a model's ability to dynamically adjust its reasoning depth based on problem difficulty, balancing computational efficiency against task accuracy. It also measures safety alignment by assessing the model's harmless response rate on adversarial or harmful prompts. Use when the user wants to benchmark on MATH500, AIME2024, AMC2023, Olympiad Bench, GSM8K, BeaverTails, HarmfulQA, or asks about evaluating this task. Reports pass@1 accuracy.

Evaluates Vision-Language Models' ability to dynamically select between text-only and tool-augmented reasoning modes, and assesses the quality, efficiency, and final accuracy of their reasoning processes across multimodal domains. Use when the user wants to benchmark on AdaptMMBench, or asks about evaluating this task. Reports Accuracy.

Evaluates the algorithmic and system efficiency of the Adasum distributed gradient combiner compared to naive gradient averaging across different hardware interconnects and model scales. It probes the ability of synchronous SGD to scale to large effective batch sizes while maintaining convergence accuracy and reducing time-to-accuracy. Use when the user wants to benchmark on ImageNet, SQuAD 1.1, MNIST, or asks about evaluating this task. Reports epochs_to_target_accuracy.