Claude Skills by qhjqhj00

This benchmark evaluates computer vision models on three core tasks using drone footage of wild baboons: object detection, multi-object tracking, and fine-grained behavioral recognition. It probes a model's ability to handle extreme scale variation, heavy occlusion, and temporal context in natural, uncontrolled wildlife environments. Use when the user wants to benchmark on BaboonLand, or asks about evaluating this task. Reports Top-1 accuracy.

Evaluates the lexical and syntactic competence of self-supervised spoken language models using child-centered, developmentally plausible speech data. It probes whether models can acquire language-like representations from ecologically valid, in-the-wild audio recordings compared to clean audiobooks or text-based inputs. Use when the user wants to benchmark on BabySLM, or asks about evaluating this task. Reports lexical accuracy.

Evaluates fundamental visual reasoning capabilities in multimodal large language models independent of linguistic priors. It probes early-vision abilities such as visual tracking, spatial perception, fine-grained discrimination, and visual pattern recognition through image-based tasks. Use when the user wants to benchmark on BabyVision, or asks about evaluating this task. Reports Avg@3.

Evaluates neural machine translation models on English-German, German-English, English-Latvian, and Latvian-English translation tasks. It probes the effectiveness of iterative back-translation (wake-sleep extension) compared to standard back-translation and baseline MLE training across supervised and semi-supervised domain adaptation scenarios. Use when the user wants to benchmark on WMT 2017, TED (IWSLT 2014), or asks about evaluating this task. Reports BLEU (SACREBLEU v1.2.3).

This benchmark probes an agent's ability to recover from harmful states in real-world computer use environments by backtracking or remediating to a safe operational state. It evaluates how well agents align with human preferences during recovery under varying resource constraints (step limits). Use when the user wants to benchmark on BackBench, or asks about evaluating this task. Reports Bradley-Terry rating.

Evaluates the fine-tuning performance of lightweight, pre-trained CNN and attention-based backbones across diverse image classification domains, including natural images, remote sensing, medical histopathology, and plant imaging. It probes how well different architectures generalize under data-scarce conditions and whether ImageNet pre-training accuracy correlates with downstream task performance. Use when the user wants to benchmark on CIFAR-10, CIFAR-100, Tiny ImageNet, Stanford Dogs, Flowe...

This benchmark evaluates the designability, structural diversity, and novelty of generated protein backbones across varying lengths. It measures how well diffusion models can produce foldable and structurally distinct protein scaffolds. Use when the user wants to benchmark on Protein Backbone Generation Benchmark, or asks about evaluating this task. Reports scRMSD.

Probes the interdependence between vision backbone architectures and optimization algorithms by measuring how different backbones perform when paired with various optimizers across classification and detection tasks. It evaluates whether architectural design dictates optimal optimizer choice and how this coupling affects transfer learning and hyperparameter robustness. Use when the user wants to benchmark on CIFAR-100, ImageNet-1K, COCO, or asks about evaluating this task. Reports Top-1 accur...

Evaluates language models' vulnerability to backdoor attacks and the effectiveness of detection and purification defenses. It probes whether a model can correctly classify clean text while resisting trigger-induced misclassifications, and whether a defense can identify poisoned samples without degrading benign task performance. Use when the user wants to benchmark on SST-2, YELP, AG’s News, or asks about evaluating this task. Reports AUC.

This benchmark evaluates the robustness and effectiveness of multimodal backdoor attacks and defense mechanisms across image, text, and audio modalities. It specifically probes how well defenses maintain clean accuracy while suppressing attack success rates under varying noise conditions and label corruption. Use when the user wants to benchmark on CIFAR-10, SST-2, SpeechCommands, or asks about evaluating this task. Reports ASR, accuracy.

This benchmark evaluates the fidelity and reliability of explainable AI (XAI) attribution methods in identifying backdoor triggers versus natural image features. It tests whether attribution techniques can consistently highlight injected trigger patterns across different visibility levels and attack types, while remaining invariant to clean input distributions. Use when the user wants to benchmark on CIFAR-10, GTSRB, ImageNet 2012, or asks about evaluating this task. Reports trigger recall.

Evaluates the safety and alignment of embodied LLMs by measuring their susceptibility to voice-based and text-based adversarial prompts that induce harmful physical actions, privacy violations, or fraud. It probes cascading jailbreaks, safety misalignment between language and action, and gaps in physical world knowledge. Use when the user wants to benchmark on BadRobot Physical Action Benchmark, or asks about evaluating this task. Reports MSR (Manipulate Success Rate).

BAGEL probes language models' specialized knowledge of animal natural history, including taxonomy, morphology, behavior, habitat, vocalization, and ecological interactions. It evaluates closed-book fact recall and reasoning across diverse source domains (encyclopedic, scientific literature, ecological databases, and bioacoustics) without providing source passages at inference time. Use when the user wants to benchmark on BAGEL, or asks about evaluating this task. Reports accuracy.

Compute the balanced_accuracy_score metric — provided by scikit-learn. Use when the user has predictions and ground-truth and needs to compute balanced_accuracy_score, or asks how to score with balanced_accuracy_score.

Compute Baleegh/Fluency_Score via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of Baleegh/Fluency_Score.

Evaluates audio-language alignment, reasoning, and instruction-following capabilities of audio-aware large language models. It probes the model's ability to answer audio-based questions, perform semantic reasoning, detect hallucinations, and follow complex multimodal instructions. Use when the user wants to benchmark on ClothoAQA, Synonym-Hypernym Test, MMAU, MMAR, SAKURA, Audio Hallucination Benchmark, Instruction-Following Benchmark, or asks about evaluating this task. Reports accuracy, wei...

Evaluates Arabic large language models across 14 diverse NLP categories, including creative writing, question answering, reading comprehension, logic, and machine translation. It probes the models' ability to handle complex Arabic morphology, long-form generation, and task-specific reasoning. Use when the user wants to benchmark on BALSAM, or asks about evaluating this task. Reports LLM as a judge.

Evaluates a model's ability to generate coherent, faithful Bangla text conditioned on a set of extracted keywords, testing sequence-to-sequence generation capabilities in a low-resource language setting. Use when the user wants to benchmark on Bangla Key2Text, or asks about evaluating this task. Reports generation_quality.

Evaluates large language models' ability to solve mathematical Olympiad problems in Bangla and English. It probes multilingual reasoning, step-by-step problem solving, and the impact of retrieval-augmented generation and fine-tuning on low-resource language math tasks. Use when the user wants to benchmark on BDMO dataset, Test dataset, or asks about evaluating this task. Reports accuracy.

Evaluates the capability of NLP models to classify sentiment in Bangla text. It compares classical machine learning, CNN, FastText, and transformer-based architectures to determine which model family performs best on low-resource Bangla sentiment tasks. Use when the user wants to benchmark on Multiple publicly available Bangla sentiment datasets, or asks about evaluating this task. Reports accuracy.

Probes multilingual vision-language models' ability to understand and reason about Bengali cultural concepts across regional dialects and historically linked languages. It measures how well models maintain cultural grounding when faced with linguistic variation, testing both visual captioning and structured question-answering capabilities. Use when the user wants to benchmark on BanglaVerse, or asks about evaluating this task. Reports accuracy (%).

Evaluates the ability of models to classify Bangla book reviews into three sentiment categories (Positive, Neutral, Negative). It probes product-specific sentiment analysis in a low-resource language, testing both contextual understanding and robustness to class imbalance and lexical overlap. Use when the user wants to benchmark on BANGLABOOK, or asks about evaluating this task. Reports weighted average F1-score.

Evaluates AI agents' ability to execute end-to-end investment banking workflows, requiring multi-file deliverable generation (Excel, PowerPoint, reports) using specialized financial tools and data sources. It probes financial judgment, cross-artifact consistency, tool-use fidelity, and professional presentation standards under realistic constraints. Use when the user wants to benchmark on BankerToolBench, or asks about evaluating this task. Reports rubric score.

Evaluates the ability of quantum machine learning models to classify synthetic financial transactions as fraudulent or benign based on demographic, merchant, and transactional features. It probes the models' capacity to handle imbalanced binary classification tasks and extract discriminative patterns from tabular financial data. Use when the user wants to benchmark on BankSim, or asks about evaluating this task. Reports F1 score.

Evaluates a model's ability to retrieve relevant legal passages and answer legal questions that require multi-hop or analogical reasoning, characterized by low lexical overlap between queries and documents. Use when the user wants to benchmark on Bar Exam QA, or asks about evaluating this task. Reports Recall@10.

Evaluates the robustness of autonomous driving behavior planners (MCTS, RL, IDM, MOBIL) in interactive multi-agent traffic. It probes how well models handle prediction inaccuracies, parameter variations, and complex merging constraints without fine-tuning. Use when the user wants to benchmark on BARK Sampling Scenarios, INTERACTION, or asks about evaluating this task. Reports collision_rate.

Evaluates the reproducibility and standardization of evaluation protocols in recommender systems. It probes both candidate item matching (ranking) and click-through rate (CTR) prediction tasks using standardized data splits, hyperparameter configurations, and common industry metrics to ensure fair and comparable model performance. Use when the user wants to benchmark on Criteo, MovieLens, or asks about evaluating this task. Reports NDCG@K, AUC.

Evaluates a denoising sequence-to-sequence pre-trained model across discriminative comprehension, abstractive text generation, dialogue response, and machine translation tasks to measure cross-task generalization and generation quality. Use when the user wants to benchmark on SQuAD 1.1, SQuAD 2.0, GLUE, CNN/DailyMail, XSum, ConvAI2, ELI5, WMT'16 RO-EN, or asks about evaluating this task. Reports ROUGE.

BARTScore evaluates the quality of generated text by treating evaluation as a conditional text generation task. It measures the likelihood of a hypothesis given a source text, or a reference given a hypothesis, using pre-trained sequence-to-sequence models. This approach enables unsupervised, multi-perspective assessment of fluency, factuality, and informativeness without relying on human annotations or simple n-gram overlap. Use when the user has predictions and gold and needs to compute Spe...

Compute bascobasculino/mot-metrics via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of bascobasculino/mot-metrics.

Evaluates multimodal human activity recognition capabilities in basketball training scenarios by classifying complex dynamic movements from synchronized physiological, inertial, and video sensor data. Use when the user wants to benchmark on BasketHAR, or asks about evaluating this task. Reports F1-score.

Evaluates multimodal large language models on close-ended visual question answering and open-ended generation tasks in Basque and English. It probes visual reasoning, language proficiency, and the impact of training data composition and backbone LLM choice on low-resource language performance. Use when the user wants to benchmark on VQAv2, A-OKVQA, PixMoCapQA, BertaQA, Wildvision, or asks about evaluating this task. Reports Accuracy.

This benchmark evaluates Basque language models across classical NLP tasks including topic classification, stance detection, coreference detection, and natural language inference. It measures both task-specific performance and overall linguistic competence in a low-resource agglutinative language setting. Use when the user wants to benchmark on BasqueGLUE, or asks about evaluating this task. Reports Avg.

This benchmark evaluates audio language models on musical understanding and semantic reasoning across four domains: structural segmentation, lyric transcription, musicological analysis, and artist collaboration. It probes the model's ability to process long-form audio and perform temporal, hierarchical, and musicological reasoning over vocal and structural attributes. Use when the user wants to benchmark on BASS, or asks about evaluating this task. Reports IWER (Normalized Word Error Rate).

Evaluates the accuracy and robustness of event-based optical flow estimation models. It probes the model's ability to predict dense 2D motion fields from sparse, asynchronous event streams, handling varying temporal resolutions and occlusions. Use when the user wants to benchmark on DSEC-Flow, MVSEC, or asks about evaluating this task. Reports EPE.

This benchmark evaluates a model's ability to understand coarse driving actions and predict bidirectional control transitions between human drivers and automated driving systems. It probes multimodal fusion capabilities by testing whether models can leverage synchronized video, vehicle telemetry, and route context to forecast handovers and takeovers under varying time horizons. Use when the user wants to benchmark on BATON, or asks about evaluating this task. Reports Accuracy, AUPRC.

Evaluates a controllable text-to-speech model's ability to generate intelligible speech and accurately convey specific emotional tones based on text instructions. It probes zero-shot cross-lingual generalization and instruction-following capabilities in speech synthesis. Use when the user wants to benchmark on Seed-TTS, Emotion dataset, or asks about evaluating this task. Reports Emotion Classification Accuracy.

Evaluates a genetic algorithm enhanced with an LRU strategy for estimating battery swap demand and optimizing 24-hour charging schedules. It probes the algorithm's ability to minimize charging costs while maintaining high user satisfaction and computational efficiency under real-world demand fluctuations. Use when the user wants to benchmark on ST-EVCDP series, UrbanEV series, or asks about evaluating this task. Reports optimization rate (r_opt).

Evaluates EFCE solvers on a parametric sequential conflict-resolution game where players place ships and fire shots. It probes the solver's ability to construct incentive-compatible correlation plans that maximize social welfare through deterrence and punishment mechanisms. Use when the user wants to benchmark on Battleship, or asks about evaluating this task. Reports Social Welfare (SW).

Evaluates the likelihood of different binary black hole formation channels (CEE, CHE, SMT) given gravitational wave strain data by comparing Bayesian evidence and prior odds. Use when the user has predictions and gold and needs to compute Bayes factor ($\mathcal{B}$), Odds ratio ($\mathcal{O}$).

Evaluates a Bayesian statistical inversion method for estimating optical flow fields and quantifying their uncertainty from image pairs, compared against deterministic baselines. Use when the user wants to benchmark on Synthetic benchmark flow fields, Middlebury dataset, or asks about evaluating this task. Reports reconstruction accuracy.

Evaluates the ability of Bayesian optimization methods to efficiently search discrete spaces (molecules, arithmetic expressions) by maximizing or minimizing a black-box objective function over a limited budget of oracle calls. It probes how well a model aligns its latent representation with the objective landscape to guide search. Use when the user wants to benchmark on Guacamol, TDC DRD3, Arithmetic Expression, or asks about evaluating this task. Reports objective value.

Evaluates multilingual translation quality and cross-lingual reasoning across high-resource and low-resource languages. It probes the model's ability to align languages and transfer capabilities from high-resource to low-resource settings without extensive low-resource instruction data. Use when the user wants to benchmark on Flores-101, WMT22, Belebele, XNLI, GSM8K, or asks about evaluating this task. Reports BLEU (sacrebleu), COMET.

Evaluates a model's zero-shot in-context learning capability on reasoning-heavy multiple-choice tasks. It compares self-generated demonstrations against direct prompting and chain-of-thought baselines to measure accuracy gains. Use when the user wants to benchmark on BIG-Bench Hard (BBH), or asks about evaluating this task. Reports accuracy.

Evaluates the feasibility of identifying host galaxies for binary black hole mergers using next-generation gravitational wave detector networks by comparing estimated localization volumes against theoretical stellar mass and metallicity thresholds. Use when the user wants to benchmark on Grid I: Galaxy Catalogue Injections, Grid II: Maximum & Minimum Sky Sensitivity Injections, or asks about evaluating this task. Reports localization_volume.

This protocol evaluates how well aggregate and per-task benchmark performance can be predicted from scaled compute using scaling law fits. It probes the monotonicity and predictability of LLM capabilities across compute scaling, distinguishing between stable scaling trends and emergent or non-monotonic behaviors. Use when the user wants to benchmark on BIG-Bench Hard (BBH), MMLU, or asks about evaluating this task. Reports mean absolute error.

Evaluates whether simulated gravitational-wave observations (detection rates and chirp mass distributions) can distinguish between different compact binary population synthesis models of binary black hole formation under realistic detector sensitivities and observing durations. Use when the user wants to benchmark on Simulated aLIGO O1/O2 BBH detections, or asks about evaluating this task. Reports posterior probability.

Tests the impact of prompt structure and logical validity on language model reasoning performance. Specifically, it compares answer-only, standard chain-of-thought, and logically invalid chain-of-thought prompting strategies on complex reasoning tasks. Use when the user wants to benchmark on BIG-Bench Hard, or asks about evaluating this task. Reports accuracy.

Evaluates social bias in question-answering models by measuring accuracy and a bias score across ambiguous and disambiguated contexts. It probes whether models rely on stereotypes when context is under-informative and whether correct answers align with harmful biases. Use when the user wants to benchmark on BBQ, or asks about evaluating this task. Reports accuracy.

Evaluates the ability of retrieval models to match statutory article questions to the correct legal articles in Dutch and French. It benchmarks both zero-shot dense/lexical models and fine-tuned language-specific models on a parallel bilingual dataset. Use when the user wants to benchmark on bBSARD, or asks about evaluating this task. Reports R@k, MAP@k, MRR@k, nDCG@k.