Claude Skills by qhjqhj00

This benchmark evaluates a model's ability to detect time-dependent and physical mistakes in robotic task executions from video. It specifically probes temporal reasoning, semantic task violation detection, and sim-to-real generalization by comparing frame-level anomaly predictions against ground-truth annotations. Use when the user wants to benchmark on BridgeData V2, Multi-robot dataset, or asks about evaluating this task. Reports F1.

Evaluates deepfake detectors on distinguishing real from synthetic audio and video, and on attributing synthetic audio to specific TTS generators. It probes robustness to post-processing (DTW alignment, augmentation) and video compression. Use when the user wants to benchmark on TIMIT-TTS, or asks about evaluating this task. Reports AUC.

Evaluates the efficiency and accuracy of LLM benchmarking by testing how well a small, strategically selected subset of examples predicts overall model performance on standard evaluation scenarios. Use when the user wants to benchmark on HELM, MMLU, AlpacaEval 2.0, Open LLM Leaderboard, or asks about evaluating this task. Reports estimation error.

Evaluates the deployment efficiency and accuracy of neural networks on commodity microcontrollers (MCUs) under strict memory and latency constraints. It measures inference speed, memory footprint, and task-specific accuracy across vision, audio, and anomaly detection workloads. Use when the user wants to benchmark on TinyMLPerf (VWW, KWS, AD), or asks about evaluating this task. Reports Accuracy (%), Latency (ms).

Evaluates a compact audio-language model's ability to perform music information retrieval (genre and instrument classification), generate descriptive music captions, and answer complex multiple-choice questions about musical theory and structure. Use when the user wants to benchmark on GTZAN, Medley-Solos-DB, MusicCaps, MuChoMusic, or asks about evaluating this task. Reports classification accuracy.

Evaluates multimodal language models' ability to perform agentic reasoning with images, specifically requiring dynamic visual manipulation and tool-use to solve complex tasks like rotation, jigsaw assembly, and instrument reading. It probes whether models can iteratively process, crop, or transform visual inputs to extract information or solve spatial problems. Use when the user wants to benchmark on TIR-Bench, or asks about evaluating this task. Reports accuracy.

Evaluates semantic segmentation models for day-and-night cloud detection using thermal infrared imagery, specifically testing how auxiliary environmental features improve segmentation accuracy and how well models transfer across different satellite sensors and resolutions. Use when the user wants to benchmark on Landsat Main, or asks about evaluating this task. Reports mIoU.

Evaluates source-free domain adaptive object detection (SF-DAOD) and unsupervised domain adaptation (UDA) across natural and medical imaging domains. Probes a model's ability to align features and reduce pseudo-label noise when adapting to a target domain without access to target labels during training. Use when the user wants to benchmark on Cityscapes, Foggy Cityscapes, KITTI, SIM10k, BDD100k, RSNA-BSD1K, INBreast, DDSM, or asks about evaluating this task. Reports mAP.

Evaluates the ability of machine learning models to detect fraudulent financial transactions in real-time using aggregated transaction network features and basic attributes. It probes how well different feature engineering and classification approaches handle severe label imbalance and temporal data splits. Use when the user wants to benchmark on Ant Financial Transaction Dataset, or asks about evaluating this task. Reports F1 Score.

Evaluates LLM safety alignment against stateless multi-turn adversarial attacks. It measures how often models generate unsafe responses and which specific risk categories they fail on when subjected to iterative, context-independent prompt injection. Use when the user wants to benchmark on ModifiedMasterKeyJailbreakQuestions, or asks about evaluating this task. Reports Unsafe Response Rate.

Evaluates large language models on Bangla language capabilities, specifically probing world knowledge, commonsense reasoning, physical reasoning, and reading comprehension. The benchmark uses multiple-choice and yes/no question formats to measure how well models understand and generate text in a low-resource language context. Use when the user wants to benchmark on Bangla MMLU, CommonsenseQA BN, OpenBookQA BN, PIQA BN, BoolQ BN, or asks about evaluating this task. Reports normalized accuracy.

Evaluates large language models' proficiency in Tibetan across general knowledge comprehension and safety-critical domains. It probes the models' ability to handle low-resource language tasks, complex reasoning, and culturally sensitive alignment compared to English baselines. Use when the user wants to benchmark on Ti-MMLU, Ti-SafetyBench, or asks about evaluating this task. Reports Accuracy (ACC).

Evaluates Named Entity Recognition (NER) capabilities on Tagalog news text, specifically measuring performance across Person, Organization, and Location entities using supervised learning and zero-shot LLM prompting. Use when the user wants to benchmark on TLUNIFIED-NER, or asks about evaluating this task. Reports F1-score.

Tests an LLM's ability to iteratively design transition metal complexes (TMCs) by maximizing specific properties (polarisability) or expanding multi-objective Pareto frontiers. Use when the user wants to benchmark on Pd(II) square planar complex space, or asks about evaluating this task. Reports Pareto frontier quality.

Evaluates foundation models' multitask language understanding and reasoning capabilities in Traditional Chinese across diverse academic subjects including STEM, social sciences, humanities, and other domains. Use when the user wants to benchmark on TMMLU+, or asks about evaluating this task. Reports average accuracy (%).

Evaluates natural language-based tracking on 2000 YouTube and surveillance videos, testing the model's ability to follow and adapt to language descriptions over time. Use when the user wants to benchmark on TNL2K, or asks about evaluating this task. Reports AUC.

Evaluates long-term vision-language tracking capability by measuring localization accuracy over extended video sequences while dynamically updating natural language descriptions to handle appearance changes and occlusions. Use when the user wants to benchmark on TNLLT, or asks about evaluating this task. Reports PR.

Evaluates a model's ability to perform multi-turn task-oriented dialogue by jointly tracking dialogue states and generating task-completing responses. It probes how well the system understands user intents, fills correct slots across multiple domains, and fulfills explicit user requests. Use when the user wants to benchmark on MultiWOZ2.0/2.1, In-Car, or asks about evaluating this task. Reports Comb.

Evaluates the ability of task-oriented dialogue systems to generate natural language responses while maintaining entity consistency and completing user goals across multiple domains. It probes end-to-end dialogue generation, dialogue state tracking, and response quality under both automated simulation and human evaluation. Use when the user wants to benchmark on DSTC8 Track 1 End-to-End Multi-Domain Dialogue Challenge, MultiWOZ 2.0 benchmark, or asks about evaluating this task. Reports Succes...

Evaluates the privacy leakage of a token-level perturbation mechanism by measuring how easily an adversary can recover original tokens from their privatized embeddings. It probes the robustness of the privacy-preserving noise injection against nearest-neighbor-based inversion attacks. Use when the user has predictions and gold and needs to compute token embedding inversion accuracy.

Evaluates how different tokenizer configurations (pre-tokenizer, fitting corpus, vocabulary size) affect downstream BERT performance on tasks requiring robustness vs. sensitivity to language variation. Use when the user wants to benchmark on AV (Authorship Verification), PAN, CORE, NUCLE, Dialect, GLUE, GLUE+typo, or asks about evaluating this task. Reports accuracy, F1.

This benchmark evaluates the robustness of language model tokenizers against real-world input perturbations, including orthographic errors, script variations, homoglyphs, diacritics, and stylistic changes across five languages. It isolates the impact of tokenizer design by testing identical model architectures with different tokenization strategies. Use when the user wants to benchmark on TokSuite, or asks about evaluating this task. Reports relative performance drop.

Probes a model's ability to perform first- and second-order Theory of Mind reasoning by tracking agents' true and false beliefs about object locations. It specifically tests whether models can distinguish objective reality from subjective mental states while resisting heuristic shortcuts. Use when the user wants to benchmark on ToMi, or asks about evaluating this task. Reports aggregate accuracy.

Evaluates large language models' Theory of Mind capabilities by testing their ability to infer mental states (beliefs, intentions, emotions) across multiple orders of reasoning using story-based narratives. The benchmark probes whether models can accurately track character perspectives and answer questions about what different agents know or believe in complex social scenarios. Use when the user wants to benchmark on TOMBENCH, or asks about evaluating this task. Reports accuracy.

Evaluates deep learning-based network intrusion detection on IoT traffic, comparing hardware accelerators (Edge TPU vs ARM CPU) for classification accuracy, inference speed, and energy efficiency. Use when the user wants to benchmark on ToN-IoT, or asks about evaluating this task. Reports classification accuracy.

Evaluates zero-shot and cross-dataset generalization of a tongue segmentation model adapted from SAM. It probes the model's ability to segment tongue regions in medical images without task-specific fine-tuning on the target datasets. Use when the user wants to benchmark on TongueSet1, BioHit (TongueSet2), Webset (TongueSet3), or asks about evaluating this task. Reports mIoU.

Evaluates foundation models' ability to decompose complex instructions, reason over subgoals, and dynamically select/call external APIs or tools to complete tasks across diverse domains like translation, mathematics, web search, and data processing. Use when the user wants to benchmark on MLQA, ASDiv, MathQA, RealTimeQA, HotpotQA, WebShop, ALFWorld, Curated (Map), Curated (Weather), Curated (Stock), Curated (Slides), Curated (Tables), Curated (KGs), Curated (Cooking), Curated (Movie), Curated...

Evaluates an LLM's ability to autonomously invoke and coordinate multiple tools (search, code, calculator, etc.) for complex reasoning tasks. It probes both computational reasoning (math) and knowledge-intensive reasoning (open-domain QA) under a multi-tool collaborative setting. Use when the user wants to benchmark on AIME2024, AIME2025, MATH500, MATH, GSM8K, GAIA, HLE, WebWalker, HotpotQA, 2WikiMultihopQA, Musique, Bamboogle, or asks about evaluating this task. Reports LLM-based judging acc...

Evaluates a language model's ability to generalize tool-use capabilities to unseen tools and APIs through multi-turn interaction, parameter selection, and final response generation. It measures how well compact models trained on simulated data can adapt to real-world and out-of-dataset tool scenarios without task-specific fine-tuning. Use when the user wants to benchmark on ToolAlpaca Evaluation Set, GPT4Tools Test Set, or asks about evaluating this task. Reports Overall.

This benchmark evaluates an LLM's ability to understand API documentation, select the correct tool, and generate valid arguments to fulfill a user's goal. It probes tool manipulation capabilities across a diverse set of 8 real-world applications, ranging from simple single-call tasks to complex multi-step reasoning. Use when the user wants to benchmark on ToolBench, or asks about evaluating this task. Reports success rate.

Evaluates the safety and helpfulness of language model agents interacting with tools in a simulated environment. It measures how well an automated emulator and evaluator align with human judgments, and quantifies agent failure rates under standard and adversarial conditions. Use when the user wants to benchmark on ToolEmu Agent Trajectories, or asks about evaluating this task. Reports Cohen's κ (Quadratic-weighted).

Evaluates large language models' tool-use and function-calling capabilities, specifically probing multi-turn dialogues and agentic workflows such as search and memory retrieval. Use when the user wants to benchmark on BFCL-v4, τ-Bench, τ²-Bench, or asks about evaluating this task. Reports BFCL-v4 Overall.

Evaluates whether LLMs can correctly answer questions that require interacting with external tools, rather than relying on pre-trained knowledge. It probes tool selection, multi-step tool chaining, and reasoning over execution traces in an open-ended setting. Use when the user wants to benchmark on ToolQA, or asks about evaluating this task. Reports success rate.

Evaluates LLM tool-use capabilities in a stateful, conversational, and interactive setting. It probes the model's ability to handle implicit state dependencies, canonicalize arguments, handle insufficient information, and maintain efficiency across single/multiple tool calls and user turns. Use when the user wants to benchmark on ToolSandbox, or asks about evaluating this task. Reports average similarity score.

Compute the top_k_accuracy_score metric — provided by scikit-learn. Use when the user has predictions and ground-truth and needs to compute top_k_accuracy_score, or asks how to score with top_k_accuracy_score.

Predicts the sentiment polarity associated with a specific topic within a tweet, requiring topic-aware sentiment classification and contextual disambiguation. Use when the user wants to benchmark on Twitter2015-test, or asks about evaluating this task. Reports macro-averaged F1.

Measures the change in sentiment trend towards a specific topic over time or across datasets, requiring temporal or comparative analysis. Use when the user wants to benchmark on Twitter2015-test, or asks about evaluating this task. Reports avgDiff.

Evaluates open-domain conversational question answering with topic switching, requiring models to maintain context across multiple turns and dynamically retrieve relevant documents to answer evolving questions. Use when the user wants to benchmark on TOPIOCQA, or asks about evaluating this task. Reports F1.

Evaluates histopathology image classification for ovarian and breast cancer diagnosis using topological deep learning features combined with CNNs. Probes the model's ability to differentiate cancer subtypes and benign/malignant cases from microscopic tissue tiles. Use when the user wants to benchmark on UBC-OCEAN, BREAKHIS, or asks about evaluating this task. Reports Balanced Accuracy.

Evaluates fairness-aware link prediction models on synthetic graphs with controlled topological biases. It probes how structural properties like assortativity, heterogeneity, and class imbalance impact fairness metrics (SP, EO) and predictive accuracy (Hit@10, AUC). Use when the user wants to benchmark on Opinion use case, Friendship use case, Collab use case, Real datasets (Collab, Polblogs, Facebook), or asks about evaluating this task. Reports Statistical Parity (SP), Equalized Odds (EO).

Evaluates a model's ability to perform dialogue user request summarization, specifically extracting and condensing a user's intent and key constraints from a multi-turn task-oriented conversation into a single paragraph. Use when the user wants to benchmark on ToQB (Task-oriented Queries Benchmark), or asks about evaluating this task. Reports key_slot_verification.

Evaluates machine learning classifiers' ability to predict tornado occurrences up to five days in advance using historical meteorological grid data. It measures detection capability and false alarm rates under a strict temporal train-test split simulating real-world forecasting. Use when the user wants to benchmark on Custom Tornado Forecasting Dataset, or asks about evaluating this task. Reports POD.

Evaluates machine learning models for detecting tornadoes using full-resolution polarimetric weather radar imagery. It probes the ability of classifiers to distinguish tornadic signatures from non-tornadic weather patterns across varying difficulty levels and threshold settings. Use when the user wants to benchmark on TorNet, or asks about evaluating this task. Reports AUC (ROC).

Measures the end-to-end serving latency of an LLM inference system deployed over heterogeneous edge networks using speculative decoding. It probes how well pipeline parallelism, adaptive batching, and wireless resource allocation reduce total time-to-output compared to sequential or fixed-strategy baselines. Use when the user has predictions and gold and needs to compute total latency.

Compute the TotalVariation metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute TotalVariation, or asks how to score with TotalVariation.

Evaluates zero-shot and fine-tuned time series forecasting capabilities on real-world observability telemetry and general-purpose benchmarks. Probes model robustness to high-dimensional, nonstationary, multivariate series with skewed distributions and varying temporal intervals. Use when the user wants to benchmark on Boom, Boomlet, GIFT-Eval, LSF, or asks about evaluating this task. Reports CRPS.

Evaluates an agent's capacity to navigate through street-view environments using natural language instructions and resolve complex spatial descriptions to locate a hidden target object within a panoramic image. Use when the user wants to benchmark on Touchdown, or asks about evaluating this task. Reports pixel distance.

This evaluation probes a model's ability to classify text content according to a user-defined toxicity taxonomy. It measures how closely the model's predictions align with gold labels generated through a multi-model voting process, and tests generalization to out-of-domain categories. Use when the user wants to benchmark on ToVo, or asks about evaluating this task. Reports consensus rate.

This evaluation probes the multilingual vision-language capabilities of models across text recognition, cultural understanding, multimodal translation, and video reasoning. It specifically tests cross-lingual generalization and cultural grounding in both image and video domains across high- and low-resource languages. Use when the user wants to benchmark on ALM-Bench, OCRBench, cc-OCR, TextVQA, CoMMuTE, Multi30K, ViMUL-Bench, or asks about evaluating this task. Reports accuracy.

Evaluates molecular toxicity prediction capabilities across diverse AI architectures (descriptor-based models, neural networks, tabular transformers, and zero-shot LLMs) on a standardized chemical safety benchmark. Use when the user wants to benchmark on Tox21 Challenge dataset, or asks about evaluating this task. Reports performance.