Claude Skills by qhjqhj00

Compute the pearsonr metric — provided by scipy.stats. Use when the user has predictions and ground-truth and needs to compute pearsonr, or asks how to score with pearsonr.

Compute the PearsonsContingencyCoefficient metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute PearsonsContingencyCoefficient, or asks how to score with PearsonsContingencyCoefficient.

Evaluates multimodal large language models on their ability to selectively forget specific person or event concepts while preserving general knowledge. It probes cross-concept interference, unlearning efficacy, and the trade-off between forgetting targeted data and maintaining model utility. Use when the user wants to benchmark on PEBench, or asks about evaluating this task. Reports Efficacy.

Evaluates deep learning architectures for multi-class segmentation of pediatric brain tumors on MRI scans. It probes the model's ability to accurately delineate tumor sub-regions (whole tumor, enhanced tumor, cystic component, edema) and assesses cross-domain generalizability to adult glioma data. Use when the user wants to benchmark on PED BraTS 2024, CBTN, BraTS Adult Glioma 2023, or asks about evaluating this task. Reports lesion-wise Dice.

This benchmark evaluates the ability of weakly supervised multiple instance learning models to classify pediatric brain tumors from whole-slide histopathology images. It probes fine-grained diagnostic discrimination across varying class granularities (2 to 7 classes) under conditions of class imbalance and limited data. Use when the user wants to benchmark on Pediatric brain tumor WSI dataset, or asks about evaluating this task. Reports Macro F1.

This benchmark evaluates a model's ability to predict whether a learner will engage with a specific educational video fragment based on their historical interaction sequence and the fragment's content features. It probes sequential behavior modeling and content-based recommendation in informal, self-directed learning environments. Use when the user wants to benchmark on PEEK, or asks about evaluating this task. Reports F1-measure.

Evaluates zero-shot generalization and visual/semantic robustness of robot manipulation policies when transferred to new real-world setups, visual clutter, and unseen object configurations. Use when the user wants to benchmark on Franka Sim-to-Real Custom Setup, BRIDGE-v2, or asks about evaluating this task. Reports success rate.

This protocol evaluates the linguistic and content-level properties of academic peer review reports to assess how LLM assistance influences review quality, complexity, and aspect coverage over time. Use when the user wants to benchmark on ICLR & NeurIPS Peer Reviews, or asks about evaluating this task. Reports aspect_mentions.

This benchmark evaluates the ability of models to detect toxic sentences within academic peer reviews, where toxicity manifests as subtle emotive, rhetorical, or unconstructive language rather than overt abuse. It measures alignment with human judgments and assesses whether models can revise toxic sentences while preserving the original critique. Use when the user wants to benchmark on Peer Review Toxic Detection Dataset, or asks about evaluating this task. Reports Cohen's Kappa.

Evaluates a profiling framework's ability to measure granular, end-to-end latency and network asymmetry across heterogeneous hardware and live wireless networks in robotic systems. It probes how well the tool captures component-level timing, inference variance, and transmission delays in real-world deployments. Use when the user wants to benchmark on ImageNet, Waymo Open Dataset, Franka Emika Panda Teleoperation Setup, or asks about evaluating this task. Reports end-to-end latency.

Evaluates the ability of various LLM text detection methods to distinguish between human-written and AI-generated peer reviews, while also assessing their robustness to hybrid (human idea + AI text) workflows. Use when the user wants to benchmark on PeerPrism, or asks about evaluating this task. Reports accuracy.

Evaluates a robot policy's ability to perform contact-rich manipulation by jointly reasoning over visual and haptic feedback. It measures how well a learned representation improves sample efficiency, generalizes across peg geometries, and recovers from perturbations during peg insertion tasks. Use when the user wants to benchmark on Custom Peg Insertion Environment, or asks about evaluating this task. Reports sum of rewards achieved in an episode, normalized by the highest attainable reward.

Evaluates a model's ability to predict the next item in a session-based recommendation task by capturing evolving user preferences and mitigating preference drift over time. Use when the user wants to benchmark on Yoochoose, Diginetica, LastFM, PHEME, or asks about evaluating this task. Reports P@20.

Evaluates multi-step verifiable reasoning and agentic iteration on constraint-satisfaction puzzles. It probes a model's ability to plan, execute moves, check constraints step-by-step, and course-correct over long contexts. Use when the user wants to benchmark on Pencil Puzzle Bench, or asks about evaluating this task. Reports success rate.

Evaluates the quality and generalization capability of a large-scale, commercially licensed speech recognition dataset by training an acoustic model on it and measuring word error rate on standard read-speech benchmarks. Use when the user wants to benchmark on The People's Speech, Librispeech, or asks about evaluating this task. Reports Word Error Rate (WER).

Evaluates a model's ability to predict whether a given peptide-protein pair interacts (binary classification) and to localize binding residues on both the peptide and protein sequences. It also assesses the model's capacity to generate target-specific peptide sequences that improve structural binding affinity over native templates. Use when the user wants to benchmark on Test167, LEADS-PEP, Test251, or asks about evaluating this task. Reports AUROC.

Evaluates the accuracy of per-object depth estimation for vehicles in autonomous driving scenarios. It measures how well a model predicts the depth of individual objects given monocular RGB images and 2D bounding boxes. Use when the user wants to benchmark on Waymo Open Dataset, KITTI Detection Dataset, KITTI MOT Dataset, or asks about evaluating this task. Reports delta<1.25.

Evaluates multimodal video models on core perception skills and reasoning types across six computational tasks, including tracking, temporal localization, and video question answering. It probes zero-shot and few-shot generalization on real-world videos with dense annotations. Use when the user wants to benchmark on Perception Test, or asks about evaluating this task. Reports top-1 accuracy.

This benchmark evaluates long-horizon, perception-centric video reasoning in multimodal LLMs. It requires models to gather visual evidence across temporally separated segments and integrate multiple compositional constraints (e.g., object recognition, temporal tracking, spatial inference) to answer complex questions. Use when the user wants to benchmark on PerceptionComp, or asks about evaluating this task. Reports accuracy.

Evaluates a vision-language process reward model's ability to detect step-level visual grounding and reasoning errors in structured multimodal reasoning traces. The benchmark specifically probes whether the model can distinguish between correct and subtly mutated perception steps that are designed to be challenging for automated error detection. Use when the user wants to benchmark on PerceptionProcessBench, or asks about evaluating this task. Reports step-level correctness.

Evaluates a 3D vision-language model's ability to answer questions about indoor scenes and generate dense captions for 3D instances. It probes fine-grained spatial reasoning, object attribute recognition, and scene understanding by comparing generated text against ground-truth annotations using standard natural language generation metrics. Use when the user wants to benchmark on ScanNet, or asks about evaluating this task. Reports CiDEr.

Compute the PermutationInvariantTraining metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute PermutationInvariantTraining, or asks how to score with PermutationInvariantTraining.

This protocol evaluates language model memorisation and training data contamination by measuring how well the model predicts benchmark text compared to out-of-distribution baselines. Lower perplexity on benchmark passages relative to a clean baseline indicates the model has likely seen the text during training. Use when the user has predictions and gold and needs to compute perplexity.

This benchmark evaluates a model's ability to recognize the emotional relationship (e.g., intimate, hostile, neutral) between two interacting characters in drama videos. It probes multi-modal fusion capabilities by requiring the model to integrate visual, audio, and textual cues to classify pairwise interactions. Use when the user wants to benchmark on ERATO, or asks about evaluating this task. Reports Micro-F1.

Evaluates whether LLMs can capture true personalized user preferences by ranking items or users in groups, while explicitly controlling for confounding factors like user rating bias and item quality. It probes the model's ability to perform comparative reasoning rather than simple rating prediction. Use when the user wants to benchmark on PerRecBench, or asks about evaluating this task. Reports Kendall’s tau.

Evaluates training-free, one-shot instance segmentation in dense, cluttered, and occluded scenes. It probes the model's ability to localize and segment specific target instances using few exemplars and point prompts, while handling high object density and overlapping objects. Use when the user wants to benchmark on PerSense-D, COCO-20i, COCO-20d, LVIS-92i, LVIS-92d, or asks about evaluating this task. Reports mIoU.

Evaluates the training efficiency, scalability, and convergence of a hybrid deep learning recommender system against baselines on click-through rate (CTR) prediction tasks. It measures end-to-end training time to reach target AUC, final test AUC for statistical efficiency, and training throughput across varying model scales up to 100 trillion parameters. Use when the user wants to benchmark on Taobao-Ad, Avazu-Ad, Criteo-Ad, Kwai-Video, Criteo-Syn, or asks about evaluating this task. Reports ...

Evaluates the instruction-following capability of Persian large language models across multiple NLP tasks, including paraphrasing, sentiment analysis, and textual entailment. Use when the user wants to benchmark on parsinlu queryparaphrasing, Digikala SentimentAnalysis, FarsTail, ParsinluEntailment, or asks about evaluating this task. Reports ROUGE-L F1.

Evaluates the quality and cross-lingual transferability of machine-translated Persian named entity recognition datasets by measuring model performance against original English benchmarks. It probes how well translation-based dataset generation preserves entity boundaries and labels across languages with different scripts and linguistic structures. Use when the user wants to benchmark on CoNLL 2003, OntoNotes 5.0, NCBI Disease, WNUT 2017, or asks about evaluating this task. Reports F1.

Evaluates retrieval-augmented generation (RAG) pipelines for Persian text across general, scientific, and formal domains. It probes the ability of sentence embeddings to retrieve relevant context and large language models to generate accurate, faithful, and relevant answers based on that context. Use when the user wants to benchmark on PQuad, Scientific-Specialized, Organizational Report, or asks about evaluating this task. Reports Context Recall.

Evaluates person and body part detection accuracy using standard object detection metrics, while assessing a self-monitoring framework's ability to reduce false negatives and false positives through part-based plausibility checks. Use when the user wants to benchmark on DensePose, MS-COCO, Pascal VOC, or asks about evaluating this task. Reports AP@0.5.

Evaluates a dialogue agent's ability to generate or rank contextually appropriate next utterances while maintaining consistency with a given personal profile. It probes the model's capacity for persona-conditioned chit-chat and its ability to infer or reflect speaker interests during conversation. Use when the user wants to benchmark on PersonaChat, or asks about evaluating this task. Reports Hits@1.

Evaluates an embodied agent's ability to navigate and ground objects based on user-specific ownership semantics provided only in text. It tests long-term memory, spatial reasoning, and the capacity to interpret personalized queries without relying on visual object cues. Use when the user wants to benchmark on PersONAL, or asks about evaluating this task. Reports success_rate.

Evaluates large language models' ability to track dynamic user profile evolution over time and generate personalized responses to in-situ queries. It probes long-context memory, preference tracking, and contextual alignment across interleaved multi-session conversations. Use when the user wants to benchmark on PersonaMem, or asks about evaluating this task. Reports multiple-choice selection.

This benchmark evaluates the capability of NLP models to extract structured business process elements and their relationships from unstructured natural language text. It specifically probes entity recognition (activities, actors, gateways, data) and relation detection (flow, usage, performer/recipient) under varying information availability assumptions. Use when the user wants to benchmark on PET, or asks about evaluating this task. Reports F1.

Evaluates protein language models across 33 downstream tasks (fitness, localization, PPI, solubility, and structure prediction) to measure how sub-word tokenization and vocabulary size affect representation quality and task performance. Use when the user wants to benchmark on PETA Benchmark Suite, or asks about evaluating this task. Reports Spearman correlation.

Evaluates the accuracy and latency-constrained performance of vision-language models on long-video understanding tasks. It measures how well models preserve temporal reasoning and answer questions about extended video sequences under strict computational and time budgets. Use when the user wants to benchmark on LongVideoBench, VideoMME, EgoSchema, MVBench, or asks about evaluating this task. Reports accuracy.

Evaluates the ability of probability flow matching to infer stochastic gene regulatory dynamics and cell differentiation trajectories from time-resolved single-cell omics data. It probes interpolation accuracy, generalization to unseen initial conditions, and recovery of biologically validated gene-gene regulatory interactions. Use when the user wants to benchmark on 2D Ornstein-Uhlenbeck process, Multistable Waddington-like landscape, Ex vivo Hematopoiesis scRNA-seq, or asks about evaluating...

Evaluates the ability of statistical and learning-based detectors to identify sparse PFM-1 landmines in UAV-captured hyperspectral imagery, emphasizing performance under severe class imbalance and varying background clutter. Use when the user wants to benchmark on UAV Hyperspectral Imagery (PFM-1 Landmine Scene), or asks about evaluating this task. Reports AP.

Evaluates the expressivity and predictive performance of permutation-sensitive Graph Neural Networks (PG-GNN) on synthetic substructure counting tasks and real-world graph classification/regression benchmarks. It probes the model's ability to capture pairwise node correlations and higher-order substructures (triangles, 4-cliques) compared to standard permutation-invariant GNNs. Use when the user wants to benchmark on Erdős-Rényi random graphs, Random regular graphs, TUDataset (PROTEINS, NCI1,...

Evaluates the predictive accuracy of GNN models and the effectiveness of post-hoc explanation methods on synthetic and real-world graph classification and node classification tasks. It probes whether a parameterized explainer can learn global explanatory motifs end-to-end and generalize inductively without retraining. Use when the user wants to benchmark on BA-Shapes, BA-Community, Tree-Cycles, Tree-Grid, BA-2motifs, MUTAG, or asks about evaluating this task. Reports Accuracy.

Evaluates layout-aware document understanding models on Chinese pharmaceutical shipping documents. It probes semantic entity recognition, entity linking, and reading order prediction, specifically testing robustness to dense tabular layouts and long-range semantic dependencies. Use when the user wants to benchmark on PharmaShip, or asks about evaluating this task. Reports F1.

Evaluates whether traditional tabular reinforcement learning hardness metrics (MDP diameter, suboptimality gaps, effective horizon) can predict the sample efficiency and performance of deep RL agents across different observation modalities and environment scales. Use when the user wants to benchmark on Pharos Benchmark, or asks about evaluating this task. Reports cumulative regret.

Evaluates the ability of neural phase pickers to detect P- and S-wave arrivals in continuous seismic waveforms across multi-station networks. It probes detection accuracy, timing precision, and generalization to out-of-distribution earthquake sequences under varying signal-to-noise conditions. Use when the user wants to benchmark on NCEDC 2020 Test Set, 2019 Ridgecrest Sequence, or asks about evaluating this task. Reports F1 score.

Evaluates a model's ability to detect seismic events and classify phase types (P vs S) from raw waveform windows, particularly under varying amounts of labeled training data. It probes the effectiveness of self-supervised pretraining in learning generalizable seismic features compared to randomly initialized baselines. Use when the user wants to benchmark on ETHZ, GEOFON, STEAD, or asks about evaluating this task. Reports AUC.

Evaluates the effectiveness of different phase recovery and source separation algorithms on audio mixtures. It probes how well models maintain phase consistency and reconstruct audio quality under blind and oracle conditions, particularly when time-frequency bins overlap. Use when the user wants to benchmark on Audio source separation mixtures (synthetic harmonics, piano notes, MIDI excerpt), or asks about evaluating this task. Reports SDR.

Evaluates deep neural networks on phenotypic drug discovery tasks using high-content screening images. It probes the model's ability to deconvolve mechanisms of action, molecular targets, and compound identities from cellular phenotypes, as well as zero-shot compound retrieval for CRISPR perturbations. Use when the user wants to benchmark on Pheno-CA, or asks about evaluating this task. Reports accuracy.

Evaluates language models on reasoning, common sense, logical reasoning, knowledge retrieval, and coding capabilities across a diverse suite of standard academic benchmarks. Use when the user wants to benchmark on MMLU, HellaSwag, ANLI, GSM-8K, MATH, MedQA, AGIEval, TriviaQA, Arc-C, Arc-E, PIQA, SociQA, BigBench-Hard, WinoGrande, OpenBookQA, BoolQ, CommonSenseQA, TruthfulQA, HumanEval, MBPP, GPQA, MT Bench, or asks about evaluating this task. Reports accuracy.

Evaluates large language models on reasoning-specific capabilities including mathematics, scientific QA, coding, algorithmic planning, and spatial reasoning. It probes the model's ability to generate step-by-step solution traces and produce correct final answers under varying decoding temperatures and run counts. Use when the user wants to benchmark on AIME, GPQA Diamond, OmniMATH, LiveCodeBench, Codeforces, or asks about evaluating this task. Reports pass@1 accuracy.

Probes the vulnerability of multi-modal large language models to inference-time adversarial image perturbations that hijack response preferences (e.g., personality, opinions, contrastive biases) without model retraining. It measures how effectively optimized images steer model outputs toward attacker-specified targets across text-only, multi-modal, and universal perturbation settings. Use when the user wants to benchmark on Anthropic Model-Written Evaluation Datasets (Advanced AI Risk & Hallu...