Claude Skills by qhjqhj00

Evaluates patch-level histopathology classification across four organ types (Skin, Colorectal, Thorax, Breast). It probes a model's ability to correctly identify tissue morphologies using both a central patch and its surrounding contextual patches. Use when the user wants to benchmark on SPIDER, or asks about evaluating this task. Reports accuracy.

This benchmark evaluates the robustness of text-to-SQL models when natural language questions contain real-world synonyms replacing schema-related terms. It probes whether models rely on rigid lexical matching or can generalize to paraphrased queries while preserving the underlying database schema and target SQL query. Use when the user wants to benchmark on Spider, Spider-Syn, or asks about evaluating this task. Reports exact matching accuracy.

Evaluates multimodal long-context reasoning and figure/table comprehension on scientific papers. Tests direct question answering with images, full paper context, and chain-of-thought retrieval capabilities. Use when the user wants to benchmark on SPIQA, or asks about evaluating this task. Reports L3Score.

Evaluates the ability of a sparse linear decomposition method (SpLiCE) to reconstruct CLIP image embeddings while preserving semantic interpretability and downstream task performance. It probes how well concept-based representations align with human captions and zero-shot classification benchmarks compared to random or learned baselines. Use when the user wants to benchmark on CIFAR100, MIT States, CelebA, MSCOCO, ImageNetVal, or asks about evaluating this task. Reports cosine similarity.

Evaluates a multi-task supervised compression model for split computing across image classification, object detection, and semantic segmentation. It measures predictive accuracy alongside system-level metrics like end-to-end latency and energy consumption on resource-constrained edge devices with simulated wireless links. Use when the user wants to benchmark on ILSVRC 2012, COCO 2017, PASCAL VOC 2012, or asks about evaluating this task. Reports model accuracy.

Evaluates conversational question answering models on both clean text and noisy ASR transcripts, measuring their ability to maintain performance under speech recognition errors and leverage data distillation techniques. Use when the user wants to benchmark on CoQA, Spoken-CoQA, or asks about evaluating this task. Reports F1.

Evaluates the ability of speech-based models to perform mathematical reasoning directly from audio inputs. It probes three distinct capabilities: direct arithmetic computation, contextual word-problem solving across single- and multi-step scenarios, and application of domain-specific mathematical knowledge in spoken language. Use when the user wants to benchmark on Spoken-MQA, or asks about evaluating this task. Reports accuracy.

Evaluates dialog state tracking performance by measuring how accurately an agent predicts and maintains the current state of a multi-turn spoken conversation. Use when the user wants to benchmark on SpokenWOZ, or asks about evaluating this task. Reports Joint Goal Accuracy (JGA).

Evaluates video-language models' ability to recognize and report content encoded purely in temporal sequences of noise-like frames, probing their temporal pattern recognition and susceptibility to 'time-blindness' despite strong spatial reasoning. Use when the user wants to benchmark on SpookyBench, or asks about evaluating this task. Reports exact-match accuracy.

Compute sportlosos/sescore via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of sportlosos/sescore.

Evaluates multi-object tracking performance in sports scenes, specifically probing a model's ability to maintain track identities under fast, variable-speed motion and highly similar player appearances. Use when the user wants to benchmark on SportsMOT, or asks about evaluating this task. Reports HOTA.

Probes video question answering capabilities, specifically focusing on temporal reasoning, action causality, counterfactual inference, and fine-grained motion understanding within professional sports contexts. Use when the user wants to benchmark on Sports-QA, or asks about evaluating this task. Reports accuracy.

Evaluates the stability of safety alignment in text-to-image diffusion models after benign fine-tuning. It probes whether models suffer silent safety failures where utility remains high but safety degrades under distribution shifts like multilingual or domain-specific adaptation. Use when the user wants to benchmark on ViSU, I2P, RAB, or asks about evaluating this task. Reports R.

This benchmark evaluates the ability of computer vision models to estimate dense scene flow, optical flow, and stereo disparity at ultra-high resolutions with fine structural details. It specifically probes how well methods handle high-frequency textures, non-rigid motion, unmatched regions, and sky areas where traditional benchmarks often lack detail. Use when the user wants to benchmark on Spring, or asks about evaluating this task. Reports 1px outlier rate.

Evaluates the ability of signal quality indices (SQIs) to predict downstream task performance on medical time series. It measures how well an SQI correlates with and separates high-quality from low-quality signal segments for specific tasks like R-peak detection and atrial fibrillation classification. Use when the user wants to benchmark on Glasgow University database (GUDb), MIT-BIH Atrial Fibrillation Database (MIT-BIH AF), Deepbeat test subset, or asks about evaluating this task. Reports o...

Evaluates an LLM's ability to translate SQL queries across different database schemas while preserving structural integrity and semantic meaning. It measures mapping success, structural fidelity, execution validity, and the semantic alignment between generated SQL and natural language questions. Use when the user wants to benchmark on BIRD, SPIDER, or asks about evaluating this task. Reports Structural Alignment.

This evaluation compares the interactive analytics performance of four SQL-on-Hadoop systems (Impala, Drill, Spark SQL, Phoenix) by measuring query response times and resource utilization. It characterizes how each system's optimizer and execution engine handle join orders, operator selection, and data scanning across different storage formats and scaling configurations. Use when the user wants to benchmark on Unspecified SQL workloads (text/parquet), or asks about evaluating this task. Repor...

This benchmark evaluates a model's ability to translate natural language questions into correct SQL queries across diverse, real-world database schemas. It specifically probes the model's capacity to handle complex, multi-operation statements and cross-domain syntax structures that are often underrepresented in traditional benchmarks. Use when the user wants to benchmark on SQL-Synth, or asks about evaluating this task. Reports execution accuracy (EX).

Evaluates an LLM's ability to generate correct SQL queries for transforming building energy data schemas. It measures how well different prompt strategies and iterative optimization handle complex schema mappings, pivoting, and aggregation in real-world smart building datasets. Use when the user wants to benchmark on Building Energy Data Transformation Benchmark, or asks about evaluating this task. Reports Execution Accuracy.

Measures a model's ability to extract precise answer spans from a given context paragraph in response to a natural language question, testing reading comprehension and span prediction. Use when the user wants to benchmark on SQuAD 1.1/2.0, or asks about evaluating this task. Reports F1.

Evaluates a model's ability to extract and rank answer candidates from a given context for phrase-indexed question answering. It probes both the quality of candidate retrieval and the accuracy of final answer selection against gold spans. Use when the user wants to benchmark on SQuAD v1.1, FQuAD, or asks about evaluating this task. Reports exact-match.

Measures extractive question answering capability by requiring the model to identify a text span in a passage that answers a given question. It tests precise token-level span prediction and contextual understanding. Use when the user wants to benchmark on SQuAD v1.1, or asks about evaluating this task. Reports exact-match (EM).

Extends extractive QA by allowing questions that have no answer in the passage, testing the model's ability to abstain or predict a null span. It evaluates robustness against unanswerable questions. Use when the user wants to benchmark on SQuAD v2.0, or asks about evaluating this task. Reports F1.

Compute the SQuAD metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute SQuAD, or asks how to score with SQuAD.

Probes a model's ability to perform extractive reading comprehension while correctly identifying when a question cannot be answered from the provided context. It forces models to distinguish between answerable and unanswerable questions, testing knowledge gap detection and resistance to semantically relevant distractors. Use when the user wants to benchmark on SQuAD 2.0, or asks about evaluating this task. Reports F1.

Evaluates long-document, question-focused summarization quality through structured human ratings and automatic metric correlation. It probes a model's ability to generate accurate, comprehensive, and high-quality summaries that align with human preferences rather than relying on surface-level n-gram overlap. Use when the user wants to benchmark on SQuALITY, or asks about evaluating this task. Reports Human Rating (1-100).

Evaluates a novel 'Squashing' activation function against standard alternatives (ReLU, Sigmoid, Tanh) on synthetic 2D classification tasks and the Fashion-MNIST image classification benchmark. It measures how well continuously differentiable logical approximations perform compared to conventional non-linearities in terms of convergence speed and final classification accuracy. Use when the user wants to benchmark on Fashion-MNIST, Synthetic 2D Classification, or asks about evaluating this task...

Evaluates a model's ability to extract the smallest verbatim evidence block from a single tool observation given a focused query, while maximizing compression and correctly returning empty output for true negatives. Use when the user wants to benchmark on Squeez Benchmark, or asks about evaluating this task. Reports F1.

Evaluates sequential recommendation models across accuracy, fairness, stability, and efficiency dimensions. It tests whether models can correctly rank items based on user interaction history and assesses their robustness, bias, and computational cost. Use when the user wants to benchmark on Yelp, ML-100K, Beauty, or asks about evaluating this task. Reports Recall@5.

Evaluates symbolic regression methods on their ability to recover known physical laws from tabular data, testing both predictive accuracy and structural interpretability while probing robustness against irrelevant dummy variables. Use when the user wants to benchmark on SRSD-Feynman, or asks about evaluating this task. Reports R^2 > 0.999.

Evaluates the recommendation accuracy and unlearning effectiveness of session-based recommendation models after deleting a portion of training sessions. It measures how well the model retains predictive performance while successfully preventing the inference of removed items. Use when the user wants to benchmark on Amazon Beauty, Amazon Games, Steam, or asks about evaluating this task. Reports NDCG@K.

Evaluates machine translation quality estimation metrics on under-resourced African languages by comparing their predicted scores against human-annotated Direct Assessment (DA) judgments. It probes a model's ability to correlate with human perception of translation adequacy across diverse language pairs, including both reference-based and reference-free settings. Use when the user wants to benchmark on SSA-MTE, or asks about evaluating this task. Reports Spearman correlation.

Evaluates real-time object detection capability by predicting bounding boxes and class scores directly from multi-scale feature maps, eliminating traditional proposal generation steps. It measures how well the model localizes and classifies objects across varying scales and aspect ratios under strict latency constraints. Use when the user wants to benchmark on PASCAL VOC2007, or asks about evaluating this task. Reports mAP.

Evaluates a model's ability to generate structured, human-centric scene graphs from images by jointly predicting verb predicates and fine-grained semantic role-value pairs for persons and objects. It probes multi-concurrent action understanding, affordance reasoning, and structured visual representation learning. Use when the user wants to benchmark on SSG dataset, Action Genome dataset, or asks about evaluating this task. Reports accuracy.

Probes constrained-manifold spatial reasoning by requiring models to rank structural components based on geometric, topological, and physical constraints in complex 3D engineering scenes. It tests compositional spatial operations like mental rotation, occlusion handling, and force-path reasoning, revealing gaps in structural grounding and 3D constraint consistency. Use when the user wants to benchmark on SSI-Bench, or asks about evaluating this task. Reports Taskwise Accuracy.

Evaluates the transfer learning capability of self-supervised learning (SSL) pre-trained models on diverse histopathology datasets. It probes domain-specific representation learning by measuring performance on image classification and nuclei instance segmentation tasks under linear probing and fine-tuning protocols. Use when the user wants to benchmark on BACH, CRC, PCam, MHIST, CoNSeP, or asks about evaluating this task. Reports top-1 accuracy.

Evaluates a model's ability to predict binding affinity between drug molecules and target proteins. It probes regression accuracy, correlation strength, and ranking consistency across varying data scarcity and generalization settings. Use when the user wants to benchmark on BindingDB, DAVIS, KIBA, or asks about evaluating this task. Reports Concordance Index (CI).

Evaluates musical reasoning capabilities across rhythm, chords, intervals, and scales using sheet music problems. It tests both textual and visual (staff notation) modalities to measure how well models recognize musical elements and perform logical deductions. Use when the user wants to benchmark on Synthetic Sheet Music Reasoning Benchmark (SSMR-Bench), or asks about evaluating this task. Reports accuracy.

Evaluates the capability of deep search agents to answer complex factual and multi-hop questions using retrieval-augmented generation and multi-turn reasoning. It probes the agent's ability to dynamically adjust search strategies, verify information via RAG, and synthesize accurate answers under constrained tool-use budgets. Use when the user wants to benchmark on NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, Musique, Bamboogle, or asks about evaluating this task. Reports pass@1 accuracy.

Evaluates vision-language models on spatial understanding and general question answering using image-text pairs. It probes capabilities such as object existence, attribute recognition, action identification, counting, positional reasoning, and object identification, specifically testing how well models leverage depth information and spatial reasoning. Use when the user wants to benchmark on SSRBench, or asks about evaluating this task. Reports accuracy.

Evaluates single-step retrosynthesis capability by predicting reactant molecules from a given target product, testing both in-distribution chemical knowledge and out-of-distribution generalization. Use when the user wants to benchmark on USPTO-50K-test, URSA-expert-2026, or asks about evaluating this task. Reports Unique.

Evaluates a model's ability to classify the sentiment of movie review phrases into five fine-grained categories, measuring classification accuracy and error rates. The benchmark probes hierarchical sentiment understanding at the phrase level rather than the full sentence level. Use when the user wants to benchmark on Stanford Sentiment Treebank (SST), or asks about evaluating this task. Reports Error Rate (Fine-Grained).

Evaluates a model's ability to perform sentiment classification on constituent trees, testing both fine-grained (5-class) and binary sentiment prediction at the sentence root and phrase levels. Use when the user wants to benchmark on Stanford Sentiment Treebank, TREC, or asks about evaluating this task. Reports accuracy.

Evaluates text classification performance on sentence sentiment analysis and news topic categorization. Probes the model's ability to capture non-linear, non-consecutive word interactions (e.g., negation, long-range dependencies) for accurate document/sentence-level prediction. Use when the user wants to benchmark on Stanford Sentiment Treebank (Fine-grained), Stanford Sentiment Treebank (Binary), Sogou Chinese News Corpora, or asks about evaluating this task. Reports accuracy.

Evaluates the binary sentiment classification capability of hybrid quantum-classical language models on both short and long text sequences. It probes whether adaptive quantum routing and attention mechanisms provide measurable accuracy gains over purely classical or purely quantum baselines on standard NLP benchmarks. Use when the user wants to benchmark on SST-2, IMDB, or asks about evaluating this task. Reports Accuracy.

Evaluates the classification performance of SSVEP-based Brain-Computer Interface algorithms using Riemannian geometry on EEG covariance matrices. It probes the robustness of different covariance estimators and online/offline classification pipelines under varying trial lengths, latency delays, and outlier conditions. Use when the user wants to benchmark on SSVEP BCI dataset (12 subjects), or asks about evaluating this task. Reports classification accuracy.

Evaluates multimodal models' ability to detect unintended content, structural, and low-level appearance changes in image-to-image transitions. It probes fine-grained visual reasoning and pixel-level alignment capabilities by asking models to assess fidelity across three distinct dimensions. Use when the user wants to benchmark on StableI2I-Bench, or asks about evaluating this task. Reports accuracy.

Evaluates named entity recognition capabilities on software programming texts. It specifically probes the model's ability to identify fine-grained code-related entities like variable names, libraries, and data structures in StackOverflow posts. Use when the user wants to benchmark on StackOverflow NER corpus, or asks about evaluating this task. Reports F1.

Assesses an LLM's capacity to abstract scene-level events into concise, free-form descriptions without schema constraints. It evaluates whether the generated events form a coherent, non-redundant structure and remain factually grounded in the screenplay text. Use when the user wants to benchmark on STAGE-ES, or asks about evaluating this task. Reports Event-Structure Consistency.

Evaluates an LLM's ability to role-play as a specific movie character using memory-grounded agent frameworks. It probes consistency with the character's persona, speaking style, and narrative facts across interactive dialogues. Use when the user wants to benchmark on STAGE-ICRP, or asks about evaluating this task. Reports Persona Consistency.