Claude Skills by qhjqhj00

Evaluates a model's ability to predict IDH mutation status (mutant vs. wild-type) in glioma patients using multi-modal MRI-derived structural brain networks. Use when the user wants to benchmark on TCIA + In-house Cohort, or asks about evaluating this task. Reports accuracy.

Evaluates large language models' ability to disambiguate whether a given phrase is used idiomatically or literally within a specific context. It probes zero-shot, few-shot, and cross-lingual prompting capabilities, measuring how well models generalize to idiomatic expressions without task-specific fine-tuning. Use when the user wants to benchmark on SemEval 2022 Task 2a, FLUTE, MAGPIE, or asks about evaluating this task. Reports macro F1.

Evaluates the quality and utility of a large-scale synthetic identity document dataset for fraud detection. It measures metadata diversity, visual fidelity to real documents, stealthiness of forged modifications, and downstream model accuracy. Use when the user wants to benchmark on IDNet, or asks about evaluating this task. Reports SSIM.

Evaluates an AutoML-based intrusion detection system's ability to classify network traffic as benign or malicious across multiple attack types. It probes the framework's robustness to class imbalance and its efficiency in real-time network environments. Use when the user wants to benchmark on CICIDS2017, 5G-NIDD, or asks about evaluating this task. Reports F1-score.

Evaluates machine learning classifiers for network intrusion detection on imbalanced, high-dimensional traffic data. Probes the model's ability to distinguish benign from malicious traffic across binary and multilabel settings using standard classification metrics. Use when the user wants to benchmark on UNSW-NB15, CIC-IDS2017, CIC-IDS2018, or asks about evaluating this task. Reports Accuracy.

Evaluates the ability of individual machine learning classifiers and ensemble strategies to detect network intrusions and classify traffic types. It probes model robustness, precision-recall trade-offs, and computational efficiency across diverse real-world network traffic datasets with varying attack profiles. Use when the user wants to benchmark on RoEduNet-SIMARGL2021, CICIDS-2017, or asks about evaluating this task. Reports F1 Score.

Evaluates intrusion detection systems for resource-constrained IoT and cloud environments by measuring classification accuracy, computational efficiency, and model confidence. It probes the ability of AutoML pipelines to balance detection performance against training time, inference latency, and memory footprint. Use when the user wants to benchmark on CICIDS2017, IoTID20, or asks about evaluating this task. Reports F1-score.

This benchmark evaluates machine learning-based anomaly detection systems for smart grid cybersecurity, focusing on both detection performance and model explainability. It probes how well intrusion detection methods generalize across diverse operational datasets while providing interpretable feature importance and robustness to data noise. Use when the user wants to benchmark on Power System dataset, CIDDS-002 dataset, or asks about evaluating this task. Reports explanation sensitivity (Expl....

Compute idsedykh/codebleu via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of idsedykh/codebleu.

Compute idsedykh/codebleu2 via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of idsedykh/codebleu2.

Compute idsedykh/megaglue via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of idsedykh/megaglue.

Compute idsedykh/metric via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of idsedykh/metric.

Evaluates the accuracy and diversity of end-to-end sequential recommendation models by testing their ability to predict the next item in a user's behavior sequence while maintaining item diversity in the recommendation list. Use when the user wants to benchmark on ML100K, ML1M, or asks about evaluating this task. Reports Recall.

This evaluation probes a model's ability to detect financial fraud in a federated, privacy-preserving setting using quantum-enhanced neural networks. It measures classification performance on imbalanced transaction data while assessing robustness against simulated quantum hardware noise. Use when the user wants to benchmark on IEEE-CIS Fraud Detection, or asks about evaluating this task. Reports binary classification accuracy.

Evaluates binary financial fraud detection performance across diverse model architectures (LSTM, Transformer, XGBoost, GNN, and ensembles) on highly imbalanced transaction data. Probes threshold-independent discrimination (AUC-ROC, PR-AUC) and threshold-dependent detection accuracy (F1, Precision, Recall, MCC) under stratified cross-validation and temporal holdout conditions. Use when the user wants to benchmark on IEEE-CIS Financial Fraud Detection Dataset, or asks about evaluating this task...

Evaluates the zero-shot open information extraction (OIE) capability of pre-trained language models by measuring their ability to extract subject-predicate-object triples from text without task-specific training or fine-tuning. It probes whether LMs inherently store rich, open-world relational knowledge that can be accessed via attention mechanisms. Use when the user wants to benchmark on CaRB, Re-OIE2016, TAC KBP-OIE, Wikidata-OIE, or asks about evaluating this task. Reports F1.

Evaluates how categorical and continuous label ambiguity impacts the performance of unimodal emotion recognition models (text, audio, facial) on the IEMOCAP dataset. It tests whether filtering data by annotator agreement or VAD score dispersion yields cleaner evaluation signals. The protocol highlights the disconnect between rigid single-label benchmarks and the inherent ambiguity of affective data. Use when the user wants to benchmark on IEMOCAP, or asks about evaluating this task. Reports w...

Tests an LLM's ability to extract, compute, and reason over heterogeneous Building Information Modeling (BIM) data (IFC files) using adaptive code execution or static baselines. It probes robustness to data heterogeneity, documentation retrieval, and tool augmentation. Use when the user wants to benchmark on ifc-bench v2, or asks about evaluating this task. Reports aggregate accuracy.

Evaluates large language models' ability to follow explicit, verifiable instructions embedded in prompts, such as length constraints, keyword inclusion, formatting rules, and language requirements. It measures both strict and loose compliance across individual instructions and entire prompts to assess structural and syntactic robustness. Use when the user wants to benchmark on IFEval, or asks about evaluating this task. Reports Inst-level strict-accuracy.

This benchmark evaluates an information retrieval system's ability to follow complex, domain-specific instructions when retrieving relevant passages. It probes whether models can interpret nuanced constraints (e.g., patient demographics, legal case details, financial goals) rather than just matching keyword semantics. Use when the user wants to benchmark on IfIR, or asks about evaluating this task. Reports InstFol@20.

This benchmark evaluates bidirectional machine translation quality between Igbo and English. It probes a model's ability to accurately translate news and contemporary media content across two directions (Igbo-to-English and English-to-Igbo) using human-validated parallel sentences. Use when the user wants to benchmark on Igbo-English MT Benchmark, or asks about evaluating this task. Reports BLEU.

Probes the reliability of text-to-infographic generation models by decomposing visual fidelity into atomic yes/no checks. It evaluates whether generated images accurately encode data, follow structural constraints, and maintain consistency across multiple verification questions. Use when the user wants to benchmark on IGenBench, or asks about evaluating this task. Reports Q-ACC.

Evaluates lifelong learning algorithms on incremental label refinement, requiring models to predict both coarse (superclass) and fine-grained (subclass) labels over time without forgetting prior knowledge, while operating under incomplete information constraints. Use when the user wants to benchmark on IIRC-CIFAR, IIRC-ImageNet, or asks about evaluating this task. Reports pw-JS.

Compute Ikala-allen/relation_extraction via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of Ikala-allen/relation_extraction.

Evaluates vision-language models on cross-depiction assembly instruction alignment, testing their ability to match, verify, locate, and predict steps from diagrams and videos. It also probes mechanistic properties like representational alignment and modality reliance to diagnose the 'depiction gap'. Use when the user wants to benchmark on IKEA-Bench, or asks about evaluating this task. Reports Accuracy.

Evaluates instance-level image retrieval capability, measuring a model's ability to correctly rank specific object instances within a massive, domain-diverse image corpus. It probes robustness to background clutter, scale variations, and the effectiveness of global versus local descriptors for re-ranking. Use when the user wants to benchmark on ILIAS, or asks about evaluating this task. Reports mAP@1k.

Evaluates multimodal models' ability to detect visual illusions (pareidolia) in images, comparing performance across raw, illusory, and low-pass filtered versions. It also measures zero-shot and fine-tuned OCR capabilities on text-containing illusion images to assess perceptual robustness and text recognition under distortion. Use when the user wants to benchmark on IllusionMNIST, IllusionFashionMNIST, IllusionAnimals, IllusionChar, or asks about evaluating this task. Reports Accuracy.

Evaluates the predictive accuracy and learning efficiency of various Inductive Logic Programming (ILP) systems across synthetic grid-world tasks, scalability tests, and standard logical reasoning benchmarks. The protocol measures how well each system generalizes from positive and negative examples to learn correct logic programs under varying domain sizes and example counts. Use when the user wants to benchmark on Robot, Robot2, Member, Benchmark ILP Problems, or asks about evaluating this ta...

Probes a model's ability to predict gene expression levels from promoter sequences across different cellular contexts, testing its capacity to capture long-range regulatory dependencies and quantitative biological signals. Use when the user wants to benchmark on ILSEP, or asks about evaluating this task. Reports Pearson r.

Evaluates large-scale visual recognition capabilities across three core tasks: image classification, single-object localization, and object detection. It probes a model's ability to categorize, localize, and detect objects across 1,000 diverse categories using a dataset of approximately 1 million images. Use when the user wants to benchmark on ILSVRC, or asks about evaluating this task. Reports classification error.

Evaluates industrial image anomaly detection algorithms across seven manufacturing datasets under unsupervised, few-shot, continual, and fully supervised settings. It probes both image-level classification and pixel-level localization capabilities, while also measuring computational efficiency like inference speed and GPU memory. Use when the user wants to benchmark on MVTec AD, MVTec LOCO-AD, MPDD, BTAD, MTD, VisA, DAGM, or asks about evaluating this task. Reports Image AUC.

Evaluates an agent's ability to perform in-context compositional reasoning from image prompts by generalizing learned primitive relations to unseen source-target pairs and complex composite tasks. Use when the user wants to benchmark on 3D Shapes, BitMoji Faces, CLEVR Objects, or asks about evaluating this task. Reports MSE.

Evaluates a model's ability to generate coherent natural language descriptions for images and align specific image regions with corresponding text segments. It measures both retrieval quality and generation fidelity against human-written references. Use when the user wants to benchmark on Flickr8K, Flickr30K, MSCOCO, or asks about evaluating this task. Reports BLEU.

Evaluates vision-language models on image captioning and image-text retrieval tasks to measure zero-shot and fine-tuned generalization on long-tail visual concepts and out-of-domain data. Use when the user wants to benchmark on nocaps, COCO Captions, Flickr30K, LocNar Flickr30K, or asks about evaluating this task. Reports CIDEr.

Evaluates multimodal conversational models on their ability to generate or retrieve engaging, style-conditioned responses grounded in images and dialogue history. It probes retrieval accuracy, generation quality, and human-perceived engagement in multi-turn image-grounded conversations. Use when the user wants to benchmark on IMAGE-CHAT, or asks about evaluating this task. Reports R@1.

Evaluates the ability of vision models to learn transferable visual representations and perform accurate image classification across varying data scales and domain shifts. It probes how well patch-based self-attention architectures generalize from large-scale pre-training to standard and low-data downstream recognition tasks. Use when the user wants to benchmark on ImageNet (ILSVRC-2012), or asks about evaluating this task. Reports accuracy.

Evaluates the ability of generative models with discrete latents to restore clean images from noisy inputs using a zero-shot, patch-based variational optimization framework. Use when the user wants to benchmark on Standard denoising benchmarks (e.g., House image), or asks about evaluating this task. Reports PSNR.

Evaluates a model's ability to predict human preferences for text-to-image generation by ranking pairs of images generated from the same text prompt. It measures alignment with human judgment on coherence, fidelity, and aesthetic quality. Use when the user wants to benchmark on ImageReward Test Set, or asks about evaluating this task. Reports Preference Accuracy.

Evaluates the visual fidelity and text-image alignment of generated images. It measures realism and distribution matching using FID, and semantic alignment using CLIP scores. Use when the user wants to benchmark on COCO-2014, or asks about evaluating this task. Reports FID (CLIP features).

Evaluates a model's ability to retrieve relevant images given a text query and vice versa. It probes cross-modal alignment and ranking capabilities under both standard test-set and large-scale candidate-pool settings. Use when the user wants to benchmark on Flickr30k, COCO, or asks about evaluating this task. Reports Recall@K (R@K).

Evaluates the capability of generative models to produce symbolic music (ABC notation) that aligns with a given input image. It probes both the intrinsic musical quality of the generated output and the semantic/emotional consistency between the source image and the resulting composition. Use when the user wants to benchmark on Image-to-Music test set [[30]], or asks about evaluating this task. Reports Music Quality Level.

Evaluates multimodal models' ability to culturally adapt images (transcreation) while preserving semantics, layout, and naturalness. Probes cross-cultural visual alignment and context-aware editing capabilities for global audiences. Use when the user wants to benchmark on Image Transcreation Dataset, or asks about evaluating this task. Reports culture-concept.

Evaluates vision-language models' ability to extract structural code (HTML, LaTeX, LilyPond) from images. It uses a round-trip validation pipeline where generated code is rendered back to an image and compared to the original using automated similarity metrics. Use when the user wants to benchmark on Image2Struct, or asks about evaluating this task. Reports EMS.

Evaluates text-to-image generation models on photorealism, image-text alignment, and compositional reasoning using standard dataset metrics and human preference studies. Use when the user wants to benchmark on MS-COCO, DrawBench, or asks about evaluating this task. Reports FID-30K.

Evaluates class-conditional image generation fidelity and diversity on ImageNet 256x256. It measures how closely the distribution of generated images matches real images and how well the model covers all classes. Use when the user wants to benchmark on ImageNet, or asks about evaluating this task. Reports FID.

Evaluates image classification accuracy on ultra-constrained microcontrollers (MCUs) with strict SRAM and Flash limits, measuring the trade-off between model quantization, memory footprint, and performance. Use when the user wants to benchmark on ImageNet, or asks about evaluating this task. Reports Top-1 accuracy.

Evaluates the trade-off between inference latency, energy consumption, and classification accuracy when dynamically routing image inputs between a lightweight mobile model and a powerful cloud model using a learned neural multiplexer. Use when the user wants to benchmark on ImageNet ILSVRC 2012, or asks about evaluating this task. Reports accuracy.

Evaluates out-of-distribution (OOD) detection capabilities by measuring how well models assign low confidence to images of objects that do not exist in their training distribution. It probes whether models can reliably distinguish in-distribution classes from novel anomalies without relying on spurious cues. Use when the user wants to benchmark on ImageNet-O, or asks about evaluating this task. Reports AUPR.

Evaluates the top-1 classification accuracy of a model on the ImageNet dataset. It probes the model's ability to correctly classify images into one of 1000 categories under various training conditions, specifically testing the impact of large minibatch sizes and learning rate scaling strategies on optimization and generalization. Use when the user wants to benchmark on ImageNet, or asks about evaluating this task. Reports top-1 error (%).

Evaluates the transferability of adversarially robust ImageNet pretraining to downstream classification tasks. It probes whether robustness induces more generalizable and discriminative feature representations compared to standard training, measured under both fixed-feature and full-network fine-tuning settings. Use when the user wants to benchmark on Birdsnap, Caltech-101, Caltech-256, CIFAR-10, CIFAR-100, Describable Textures (DTD), FGVC Aircraft, Food-101, Oxford 102 Flowers, Oxford-IIIT P...