Claude Skills by qhjqhj00

This benchmark evaluates the effectiveness of feature augmentation modules for detecting Ponzi scheme accounts on the Ethereum blockchain. It probes a model's ability to classify account nodes as legitimate or malicious based on transaction graph structures and temporal behavior patterns. Use when the user wants to benchmark on Ethereum Ponzi dataset, or asks about evaluating this task. Reports micro-F1.

Evaluates object perception and hallucination in LVLMs by prompting models to identify whether specific objects are present in an image. It measures how often models correctly affirm or deny object existence without generating false positives. Use when the user wants to benchmark on POPE, or asks about evaluating this task. Reports Acc.

Tests object perception and hallucination on images without captions, evaluating whether LVLMs can ground object detection purely from visual input without textual priors. Use when the user wants to benchmark on POPE-NoCaps, or asks about evaluating this task. Reports Acc.

This evaluation probes the robustness of vision-language computer agents against adversarial visual distractions (pop-ups) injected into GUI environments. It measures how often agents are tricked into interacting with malicious overlays and how these distractions degrade their ability to complete legitimate user tasks. Use when the user wants to benchmark on OSWorld, VisualWebArena, or asks about evaluating this task. Reports Attack Success Rate (ASR).

Evaluates large language models' ability to perform quantitative reasoning and structured decision-making in financial portfolio optimization. It probes whether models can correctly apply convex optimization principles under varying constraints and multi-criteria objectives. Use when the user wants to benchmark on PortBench, or asks about evaluating this task. Reports accuracy.

Evaluates multimodal models on portrait composition understanding and generation. It probes the ability to predict aesthetic scores, reason about fine-grained composition attributes, answer image-grounded questions, and generate portraits that adhere to explicit spatial and compositional constraints. Use when the user wants to benchmark on PortraitCraft, or asks about evaluating this task. Reports SRCC.

Evaluates neural models on a Portuguese-language benchmark derived from English GLUE and SuperGLUE tasks, probing capabilities in grammatical acceptability, sentiment, paraphrase detection, semantic similarity, natural language inference, reading comprehension, and causal reasoning. Use when the user wants to benchmark on CoLA, SST-2, MRPC, QQP, STS-B, WiC, MNLI, QNLI, RTE, WNLI, WSC, CB, AXb, AXg, BoolQ, MultiRC, ReCoRD, COPA, or asks about evaluating this task. Reports single-number perform...

Evaluates the ability of self-supervised visual representations to capture geometric pose information and semantic content. It probes absolute and relative pose estimation accuracy, as well as semantic classification performance, across in-domain, out-of-domain, and real-world settings. Use when the user wants to benchmark on Carvana, Synthetic dataset [8], or asks about evaluating this task. Reports relative pose estimation accuracy.

Evaluates the robustness of human and animal pose estimation models when subjected to real-world image corruptions such as blur, noise, compression, lighting changes, and occlusion masks. It measures how much model accuracy degrades relative to clean-image performance across varying corruption severities. Use when the user wants to benchmark on COCO-C, OCHuman-C, AP10K-C, or asks about evaluating this task. Reports mRR.

Evaluates a model's ability to estimate 6-DOF camera pose (translation and rotation) from a single monocular image across indoor and outdoor environments. It probes the network's robustness to challenging conditions like motion blur, low light, and dynamic objects, as well as its generalization to unseen scenes and varying training baselines. Use when the user wants to benchmark on 7 Scenes, Cambridge Landmarks, or asks about evaluating this task. Reports localization error.

Evaluates how well automated metrics and vision-language models can identify granular errors (attribute/relation misattachments) in detailed image descriptions and correctly rank paired descriptions against human judgments. Use when the user has predictions and gold and needs to compute macro F1, Spearman rank ρ.

Compute posicube/mean_reciprocal_rank via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of posicube/mean_reciprocal_rank.

Assesses whether large language models can generate story endings that align with free-form instructions provided alongside a narrative context. It measures both instruction-following accuracy and the model's ability to produce distinct endings for different instructions. Use when the user wants to benchmark on Possible Stories, or asks about evaluating this task. Reports IFSM.

Evaluates the ability of text-to-image models to accurately render specified textual elements within aesthetically designed posters. It measures how well generated images preserve the exact characters, words, and layout instructions from the input prompt. Use when the user wants to benchmark on PosterCraft Test Prompts, or asks about evaluating this task. Reports Text F-score.

Evaluates multimodal large language models' ability to generate accurate, abstractive summaries from complex, visually dense scientific posters. It probes layout understanding, visual-textual integration, and hierarchical summarization capabilities by measuring how well models extract and synthesize information from combined image and text inputs. Use when the user wants to benchmark on PosterSum, or asks about evaluating this task. Reports ROUGE-L.

Evaluates the adversarial robustness of tool-using agentic AI against two orthogonal attack surfaces: breadth attacks that poison retrieval results to induce epistemic drift, and depth attacks that inject structural traps into information graphs to cause navigational collapse. It also probes agent susceptibility to linguistic credibility cues and hedging. Use when the user wants to benchmark on Potemkin-S2, Potemkin-Phantoms, Potemkin-Claims, or asks about evaluating this task. Reports DR.

Compute the power_divergence metric — provided by scipy.stats. Use when the user has predictions and ground-truth and needs to compute power_divergence, or asks how to score with power_divergence.

Evaluates the zero-shot and fine-tuning performance of time-series foundation models and deep learning baselines on deterministic and probabilistic power system forecasting tasks. It probes capabilities including horizon sensitivity, multivariate covariate handling, and generalization to unseen geographic sites. Use when the user wants to benchmark on ARPA-E PERFORM, or asks about evaluating this task. Reports nMAE.

Evaluates reinforcement learning agents for real-time power grid topology control under adversarial attacks and dynamic loads. It probes the agent's ability to maintain grid stability, minimize operational costs, and avoid blackouts across multiple challenging scenarios. Use when the user wants to benchmark on L2RPN NeurIPS 2020 (Robustness track) Offline, L2RPN NeurIPS 2020 (Robustness track) Online, L2RPN WCCI 2020 Offline, or asks about evaluating this task. Reports survival steps, scenari...

Evaluates protein language model architectures for predicting binding affinity in multi-chain protein-protein complexes. It probes how well different architectural designs capture inter-chain interactions compared to simple sequence or embedding concatenation. Use when the user wants to benchmark on PPB-Affinity, or asks about evaluating this task. Reports Spearman ρ.

Evaluates the transferability and emergent capabilities of generalist and specialist time-series foundation models on diverse physiological tasks using PPG and cross-modal signals. Probes classification (e.g., arrhythmia, mental load, activity recognition) and regression (e.g., vital signs, blood chemistry, blood pressure) performance across clinical and ambulatory settings. Use when the user wants to benchmark on Stanford AF, Simband, Real World PPG, MIMIC-III, Sleep-EDF, or asks about evalu...

Evaluates the fidelity of synthesizing ECG signals from PPG inputs and measures the downstream utility of the generated signals for cardiac and physiological task analysis. Use when the user wants to benchmark on WESAD, CAPNO, DALIA, BIDMC, MIMIC, PPG-BP, Cuffless-BP, or asks about evaluating this task. Reports RMSE.

Evaluates reinforcement learning algorithms on continuous control and pixel-based Atari tasks to measure sample efficiency, stability, and final performance. It probes the ability of policy optimization methods to learn effective control policies across diverse physics simulators and arcade games. Use when the user wants to benchmark on OpenAI Gym (MuJoCo), Roboschool, Arcade Learning Environment, or asks about evaluating this task. Reports average total reward of the last 100 episodes.

Evaluates a model's ability to answer free-form scientific questions about unseen protein sequences using zero-shot multimodal reasoning. It probes biochemical property extraction, functional annotation, and cross-modal alignment between protein embeddings and natural language. Use when the user wants to benchmark on Pika-DS, or asks about evaluating this task. Reports mw MALE.

Compute prajwall/mse via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of prajwall/mse.

Evaluates cross-lingual document alignment (CLDA) techniques by measuring chunk/sentence-level alignment accuracy (intrinsic) and the resulting document-level machine translation quality (extrinsic) across English and 11 Indic languages. Use when the user wants to benchmark on Pralekha, or asks about evaluating this task. Reports F1 Score, DocCOMET.

Evaluates the generalization capability of a language model during the pre-training phase by measuring the average cross-entropy loss on a held-out validation corpus. Lower values indicate that the model has better learned the underlying token distribution and converges more effectively under the given architectural and training configurations. Use when the user has predictions and gold and needs to compute pre-training validation loss.

Evaluates a model's ability to identify and rank key moments (shots) in soccer match videos for summarization. It measures how well the model selects representative content when constrained to match the exact duration of a human-curated highlight summary. Use when the user has predictions and gold and needs to compute F1 Score@$T$.

Compute the precision_score metric — provided by scikit-learn. Use when the user has predictions and ground-truth and needs to compute precision_score, or asks how to score with precision_score.

Compute the Precision metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute Precision, or asks how to score with Precision.

Compute the PrecisionAtFixedRecall metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute PrecisionAtFixedRecall, or asks how to score with PrecisionAtFixedRecall.

Compute the PrecisionRecallCurve metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute PrecisionRecallCurve, or asks how to score with PrecisionRecallCurve.

This benchmark evaluates a model's ability to dynamically adapt to evolving user preferences by conditioning on natural language preferences inferred from interaction history. It probes recommendation accuracy, fine- and coarse-grained preference steering, sentiment following, and history consolidation across multiple e-commerce and gaming datasets. Use when the user wants to benchmark on Amazon Beauty, Amazon Sports and Outdoors, Amazon Toys and Games, Steam, or asks about evaluating this ta...

Measures an agent's capability to retain and adhere to user preferences during long, multi-turn conversations. It tests whether the model can maintain consistency without external reminders or with explicit preference cues. Use when the user wants to benchmark on PrefEval, or asks about evaluating this task. Reports preference retention accuracy.

Evaluates a model's ability to classify the sense of a preposition in context. It tests cross-lingual context representation and semi-supervised learning for fine-grained lexical disambiguation. Use when the user wants to benchmark on Web-reviews corpus, SemEval corpus, or asks about evaluating this task. Reports accuracy.

Evaluates a privacy-regulated speech-to-speech translation framework that replaces voice cloning with matching to pre-consented preset voices. Probes classifier robustness, cross-lingual speech naturalness, and inference efficiency across multilingual scenarios. Use when the user wants to benchmark on RAVDESS, CGDD, CAFE, EmoDB, CREMA-D, or asks about evaluating this task. Reports NISQA.

Evaluates multi-objective hyperparameter optimization algorithms on deep learning benchmarks, measuring their ability to find high-quality Pareto fronts of validation error and training cost under varying prior conditions and budget constraints. Use when the user wants to benchmark on Yahpo-Gym & PD1 HPO Benchmarks, or asks about evaluating this task. Reports mean dominated hypervolume.

Evaluates a pre-trained seismic model's multi-task capability on single-station waveforms, specifically phase picking (Pg, Sg, Pn, Sn), P-wave polarization classification, and seismic event type classification. The protocol tests generalization across temporal splits and transfer learning on local data to mitigate dataset imbalance. Use when the user wants to benchmark on CSNCD, or asks about evaluating this task. Reports recall.

Evaluates the quality of Semantic Role Labeling (SRL) systems by measuring precision and recall for predicate senses and argument annotations. It specifically probes a model's step-dependent error propagation by penalizing argument scores when the associated predicate sense is incorrect, while also handling discontinuous and reference arguments. Use when the user has predictions and gold and needs to compute PriMeSRL-Eval.

Probes an LLM's ability to align generated responses with a set of natural language constitutional principles without parameter fine-tuning. It measures both overall conformance quality and the reduction of critical principle violations through an inference-time self-correction pipeline. Use when the user wants to benchmark on SafeRLHF, HH-RLHF, or asks about evaluating this task. Reports 5-Point Likert Score Ranking.

Evaluates large language models' ability to reason about medical ethics using the Principlism framework (autonomy, non-maleficence, beneficence, justice). It probes both theoretical knowledge of ethical principles and their practical application to complex, open-ended clinical dilemmas. Use when the user wants to benchmark on PrinciplismQA, or asks about evaluating this task. Reports Knowledge accuracy, Practice score.

Evaluates the effectiveness of various prior-based loss functions (low-level boundary/distance and high-level shape/size constraints) for medical image segmentation across diverse anatomical structures and imaging modalities. Use when the user wants to benchmark on WMH, ISLES, Atrium, Colon, Spleen, Hippocampus, Prostate, ACDC, or asks about evaluating this task. Reports Dice score.

Evaluates the ability of sentiment lexicons or models to assign accurate real-valued polarity scores to individual terms, measuring rank correlation with gold standards. Use when the user wants to benchmark on Term test set, or asks about evaluating this task. Reports Kendall's τ coefficient.

Evaluates fine-grained, multi-aspect-aware paper-to-paper retrieval by decomposing long-form query papers into aspect-specific views and segmenting candidate papers into section-level representations for targeted retrieval. Use when the user wants to benchmark on SciFullBench, PatentFullBench, or asks about evaluating this task. Reports Recall@K.

Probes LLM hallucinations across four dimensions (knowledge missing, knowledge errors, reasoning errors, and instruction-following errors) by isolating error sources through controlled, task-specific queries. It evaluates model reliability and guides optimization by measuring error rates across memory, instruction, and reasoning generation stages. Use when the user wants to benchmark on PRISM, or asks about evaluating this task. Reports H-Score.

Evaluates large multimodal models' ability to detect, correct, and reason over real-world multimodal inconsistencies in scientific papers. It probes inter-modal mismatch detection, structured reasoning, and robustness to linguistic shortcuts versus genuine visual grounding. Use when the user wants to benchmark on PRISMM-Bench, or asks about evaluating this task. Reports Accuracy (%).

Evaluates the trade-offs between privacy preservation, model utility, and computational/energy costs in hybrid privacy-preserving vision systems. It probes how combining federated learning with differential privacy or secure multi-party computation affects convergence, classification accuracy, and resource consumption across different neural architectures. Use when the user wants to benchmark on Alzheimer MRI Classification, ISIC Skin Lesion Classification, or asks about evaluating this task....

Assesses an LLM agent's ability to understand and follow privacy norms while performing real-world tasks. It measures both helpfulness and the rate at which sensitive information is incorrectly exposed. Use when the user wants to benchmark on PrivacyLens, or asks about evaluating this task. Reports privacy leakage rate.

Evaluates the power efficiency, throughput, and real-time inference performance of embedded AI hardware platforms running multitask and single-task deep neural networks for automotive vision tasks. Use when the user wants to benchmark on COCO test2017, or asks about evaluating this task. Reports FPS, inference time, memory usage, energy efficiency (Wtotal, W/fps).

Evaluates the skill of a probabilistic Random Forest model in forecasting severe thunderstorms (tornadoes, large hail, damaging winds) 4–8 days in advance using ensemble meteorological data. It probes the model's calibration, discrimination, and spatial coverage compared to human-generated SPC outlooks. Use when the user wants to benchmark on SPC Severe Weather Reports & GEFSv12 Reforecast, or asks about evaluating this task. Reports Brier Skill Score (BSS).