Claude Skills by qhjqhj00

Evaluates handwritten text recognition (HTR) systems on historical Arabic manuscripts. It probes the model's ability to accurately transcribe cursive text at both the page and line levels, handling contextual character variations and layout structures. Use when the user wants to benchmark on Muharaf, or asks about evaluating this task. Reports CER.

This benchmark evaluates a model's ability to perform universal information extraction (NER, RE, EE) and fine-grained cross-modal grounding (segmentation/tracking) across text, image, audio, and video modalities in a unified zero-shot setting. It probes the model's capacity to align semantic information with visual/auditory content and handle modality-shared versus modality-specific scenarios without task-specific fine-tuning. Use when the user wants to benchmark on MUIE, or asks about evalua...

Evaluates deep reinforcement learning algorithms on continuous control tasks. It probes capabilities in handling high-dimensional state/action spaces, partial observability, sensor noise, delayed actions, and hierarchical decision-making across physics-based simulations. Use when the user wants to benchmark on DeepMind Control Suite (MuJoCo Tasks), or asks about evaluating this task. Reports reward.

Evaluates the multimodal reasoning and understanding capabilities of MLLMs across diverse domains including mathematics, chart interpretation, scientific/medical images, and hallucination detection. It measures how well models generate step-by-step reasoning paths and reflect on errors to produce correct answers. Use when the user wants to benchmark on MathVista, MMStar, MMMU, ChartQA, DynaMath, HallBench, MM-Math, MMEsum, or asks about evaluating this task. Reports Average Benchmark Score.

Evaluates models' ability to process and extract information from long documents (minimum 10,000 tokens) across multiple NLP tasks including question answering, summarization, classification, and translation. It specifically probes long-context dependency handling and real-world document understanding capabilities. Use when the user wants to benchmark on MuLD Benchmark, NarrativeQA, HotpotQA, OpenSubtitles, or asks about evaluating this task. Reports results.

Evaluates the end-to-end latency, success rate, and cost-efficiency of a multi-agent LLM tutoring system under varying concurrency levels across different cloud inference throughput tiers. It isolates the impact of shared vs. priority vs. provisioned inference pools on response time variance and system reliability. Use when the user wants to benchmark on ITAS Student Query Corpus, or asks about evaluating this task. Reports end-to-end latency.

Evaluates the generalization and task-specific performance of a multi-anatomy X-ray foundation model across diverse downstream tasks including image retrieval, disease classification, anatomical segmentation, lesion localization, and clinical report generation. Use when the user wants to benchmark on CheXpert, dXR, PTX / SIIM-ACR, MURA, JSRT, VinDr-RibCXR, PAX-Ray++, MS-CXR, IU-XRay, Bone Fracture Detection, or asks about evaluating this task. Reports AUROC.

Evaluates the emotional intelligence (EI) capabilities of spoken dialogue models in multi-turn interactive settings. It probes basic emotion understanding, advanced emotion support, paralinguistic analysis, and style inference across both Chinese and English dialogues. Use when the user wants to benchmark on MULTI-Bench, or asks about evaluating this task. Reports accuracy.

Evaluates deep learning models on their ability to classify multi-type cancer histopathological images across six distinct cancer categories. It probes the model's capacity to learn discriminative morphological features and generalize across heterogeneous medical imaging conditions. Use when the user wants to benchmark on Multi-Cancer Histopathology Dataset (Kaggle), or asks about evaluating this task. Reports accuracy.

Evaluates the cross-institutional generalizability of deep learning models for multi-disease chest X-ray classification. It probes whether training on diverse, weakly-labeled radiology datasets improves prediction performance for specific pathologies when tested on held-out medical sites. Use when the user wants to benchmark on NIH, CheXpert, Shifa International Hospital (SIH), or asks about evaluating this task. Reports AUC.

This benchmark evaluates zero-shot cross-lingual transfer and multi-label classification capabilities on legal documents. It probes how well models trained in one language generalize to others, while handling highly skewed label distributions and temporal concept drift across 23 EU languages. Use when the user wants to benchmark on MultiEURLEX, or asks about evaluating this task. Reports mean R-Precision (mrp).

Evaluates multi-hop question answering capabilities across diverse reasoning types, including implicit commonsense/arithmetic reasoning, explicit composition/comparison, and fact verification. It tests the model's ability to synthesize information from retrieved evidence and generate step-by-step explanations. Use when the user wants to benchmark on STRATEGYQA, FERMI, QUARTZ, HOTPOTQA, 2WIKIMQA, BAMBOOGLE, FEVEROUS, or asks about evaluating this task. Reports F1.

Evaluates optical flow estimation models on synthetic single- and multi-human video sequences. It probes the model's ability to handle complex human poses, occlusions, and fine-grained motion on small body parts like fingers and hands. Use when the user wants to benchmark on SHOF, MHOF, or asks about evaluating this task. Reports EPE.

Evaluates an LLM's ability to identify multiple concurrent toxicity categories in real-world prompts using a fine-grained 15-category taxonomy. It probes fine-grained safety alignment, multi-label classification under ambiguous annotations, and the model's robustness to sparse or noisy supervision signals. Use when the user wants to benchmark on Q-A-MLL, H-X-MLL, R-A-MLL, or asks about evaluating this task. Reports mean Average Precision.

Evaluates abstractive models on multi-document summarization, measuring how well they condense multiple source articles into a single coherent summary. It probes coverage, redundancy control, and fluency under strict input length constraints. Use when the user wants to benchmark on Multi-News, DUC 2004, or asks about evaluating this task. Reports R-1, R-2, R-SU.

Evaluates vision models on diagnostic and prognostic classification of Oral Squamous Cell Carcinoma using high-magnification histopathology images. It probes multi-task learning capabilities, feature fusion across multiple tissue slices, and the impact of stain normalization and image resolution on clinical prediction accuracy. Use when the user wants to benchmark on Multi-OSCC, or asks about evaluating this task. Reports AUC.

This protocol evaluates how accurately a statistical estimation method can reconstruct the full performance distribution and specific quantiles of large language models across hundreds of prompt templates, using a fraction of the standard evaluation budget. It probes the robustness of LLM performance metrics against arbitrary prompt selection and measures the efficiency of borrowing strength across prompts and examples. Use when the user wants to benchmark on MMLU, BIG-bench Hard, LMentry, or...

This evaluation protocol probes the robustness of large language models to instruction phrasing by measuring performance across multiple semantically equivalent prompts. It assesses whether model rankings and absolute scores remain stable when the same task is presented with different instruction templates. Use when the user wants to benchmark on LMentry, BIG-bench Lite, BIG-bench Hard, or asks about evaluating this task. Reports exact match evaluation.

This evaluation probes the scalability, asymptotic optimality, and computational efficiency of sampling-based motion planners in high-dimensional multi-robot configuration spaces. It measures how quickly algorithms find initial feasible paths, converge to optimal costs, and succeed under increasing dimensionality and robot counts. Use when the user wants to benchmark on 2 Disk Robots among 2D Polygons, Many Disk Robots among 2D Polygons, Dual-arm Manipulator (Motoman SDA10F), Motoman Tabletop...

Evaluates a model's ability to perform next-item recommendation by leveraging both current session context and historical multi-session information. It probes how well the model handles varying session lengths (short vs. long) and filters out noise from irrelevant historical sessions. Use when the user wants to benchmark on Delicious, Reddit, or asks about evaluating this task. Reports Recall@20.

This benchmark evaluates an LLM's ability to resolve software engineering issues across multiple programming languages. It probes capabilities in long-context reasoning, multi-file code patching, and fault localization by requiring models to generate executable fixes for real-world repository issues. Use when the user wants to benchmark on Multi-SWE-bench, or asks about evaluating this task. Reports Resolved Rate (%).

Assesses an agent's ability to execute complex, multi-turn voice-driven tasks requiring tool invocation, memory, and reasoning across varying difficulty levels. Use when the user wants to benchmark on Custom Multi-Turn Voice Tasks, or asks about evaluating this task. Reports Avg. Success.

Evaluates a model's ability to resolve multi-turn dialogue context into standalone questions and subsequently generate correct SQL queries. It measures both the quality of context resolution (utterance rewrite) and the accuracy of semantic parsing across individual questions and full dialogue interactions. Use when the user wants to benchmark on SParC, CoSQL, TASK, CANARD, or asks about evaluating this task. Reports Question Match, Interaction Match.

Evaluates active learning strategies for multi-view 3D pose estimation by measuring annotation efficiency. It probes how well geometric consistency and self-training can reduce the number of required human annotations while maintaining low 3D keypoint error. Use when the user wants to benchmark on CMU Panoptic, InterHand2.6M, or asks about evaluating this task. Reports 3D Mean Key Point Error (MKPE).

Evaluates the robustness of deep multi-view clustering models when input data is corrupted by randomly injected noise at varying proportions. It measures how effectively the model can identify noisy samples, rectify them, and produce accurate cluster assignments across multiple feature views. Use when the user wants to benchmark on BBCSport, WebKB, Reuters, UCI-digit, Caltech101, STL10, or asks about evaluating this task. Reports ACC.

Evaluates 3D visual grounding models on their ability to localize zero, single, or multiple objects in 3D scenes based on natural language descriptions. It tests whether models can correctly identify target objects while handling ambiguous references, distractors, and zero-target cases. Use when the user wants to benchmark on Multi3DRefer, or asks about evaluating this task. Reports Acc@0.5.

Evaluates the ability of LLM-powered multi-agent systems to collude and execute financial fraud in simulated social platform environments. It probes how agents coordinate through public and private channels, adapt to content warnings, and amplify fraud risks based on interaction depth and activity levels. Use when the user wants to benchmark on MultiAgentFraudBench, or asks about evaluating this task. Reports fraud_success.

Evaluates speech anti-spoofing detection and API source attribution capabilities on a large-scale dataset of synthetic speech generated by 30 distinct APIs. It probes model robustness to domain shifts, generalization to unseen spoofing sources, and fine-grained source identification in realistic, heterogeneous environments. Use when the user wants to benchmark on MultiAPI Spoof, or asks about evaluating this task. Reports EER.

Evaluates text-to-image generation models on their ability to synthesize images from multiple reference images and text prompts. It probes adherence to complex instructions, consistency with reference attributes, and robustness to domain mismatches, scale discrepancies, rare concepts, and multilingual text. Use when the user wants to benchmark on MultiBanana, or asks about evaluating this task. Reports MultiBanana score.

Tests the ability of retrieval models to rank previously fact-checked claims relevant to a given social media post. It probes crosslingual and monolingual claim retrieval capabilities, evaluating how well models handle multilingual text, varying post lengths, and different fact-check ratings. Use when the user wants to benchmark on MultiClaim, or asks about evaluating this task. Reports S@K.

Compute the MulticlassAccuracy metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassAccuracy, or asks how to score with MulticlassAccuracy.

Compute the MulticlassAUROC metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassAUROC, or asks how to score with MulticlassAUROC.

Compute the MulticlassAveragePrecision metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassAveragePrecision, or asks how to score with MulticlassAveragePrecision.

Compute the MulticlassCalibrationError metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassCalibrationError, or asks how to score with MulticlassCalibrationError.

Compute the MulticlassCohenKappa metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassCohenKappa, or asks how to score with MulticlassCohenKappa.

Compute the MulticlassConfusionMatrix metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassConfusionMatrix, or asks how to score with MulticlassConfusionMatrix.

Compute the MulticlassEER metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassEER, or asks how to score with MulticlassEER.

Compute the MulticlassExactMatch metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassExactMatch, or asks how to score with MulticlassExactMatch.

Compute the MulticlassF1Score metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassF1Score, or asks how to score with MulticlassF1Score.

Compute the MulticlassFBetaScore metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassFBetaScore, or asks how to score with MulticlassFBetaScore.

Compute the MulticlassHammingDistance metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassHammingDistance, or asks how to score with MulticlassHammingDistance.

Compute the MulticlassHingeLoss metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassHingeLoss, or asks how to score with MulticlassHingeLoss.

Compute the MulticlassJaccardIndex metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassJaccardIndex, or asks how to score with MulticlassJaccardIndex.

Compute the MulticlassLogAUC metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassLogAUC, or asks how to score with MulticlassLogAUC.

Compute the MulticlassMatthewsCorrCoef metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassMatthewsCorrCoef, or asks how to score with MulticlassMatthewsCorrCoef.

Compute the MulticlassNegativePredictiveValue metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassNegativePredictiveValue, or asks how to score with MulticlassNegativePredictiveValue.

Compute the MulticlassPrecision metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassPrecision, or asks how to score with MulticlassPrecision.

Compute the MulticlassPrecisionAtFixedRecall metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassPrecisionAtFixedRecall, or asks how to score with MulticlassPrecisionAtFixedRecall.

Compute the MulticlassPrecisionRecallCurve metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassPrecisionRecallCurve, or asks how to score with MulticlassPrecisionRecallCurve.

Compute the MulticlassRecall metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute MulticlassRecall, or asks how to score with MulticlassRecall.