Claude Skills by qhjqhj00

This benchmark evaluates a traceable cross-source retrieval-augmented generation framework for Chinese Tibetan medicine QA. It probes the model's ability to route queries across heterogeneous knowledge bases, fuse cross-source evidence, and generate faithful answers with correct citations. Use when the user wants to benchmark on Chinese Tibetan-medicine QA dataset, or asks about evaluating this task. Reports CrossEv@5.

This protocol evaluates models on Chinese toxicity detection across two tasks: binary sentence-level classification and fine-grained toxic span extraction. It measures classification accuracy and precision/recall, while also assessing the model's ability to extract contiguous, human-readable toxic spans and the faithfulness of those explanations via confidence masking. Use when the user wants to benchmark on COLD, ToxiCN, CNTP, or asks about evaluating this task. Reports F1, Overlap F1.

Evaluates a video representation's sensitivity to temporal direction by distinguishing between temporally opposite actions (e.g., opening vs. closing a door). Also tests general action recognition capability via linear probing on standard benchmarks. Use when the user wants to benchmark on Something-Something v2, EPIC-Kitchens, Charades, Kinetics-400, UCF-101, HMDB-51, or asks about evaluating this task. Reports Chiral Accuracy.

Evaluates the quality of open-ended, long-form responses from Vision-Language Models (VLMs) using pairwise preference ranking across five fine-grained criteria: overall preference, relevance, reasoning, hallucinations, and details. Use when the user wants to benchmark on CHIRP, or asks about evaluating this task. Reports pairwise_preference.

Compute the chisquare metric — provided by scipy.stats. Use when the user has predictions and ground-truth and needs to compute chisquare, or asks how to score with chisquare.

Evaluates machine translation quality between Cherokee and English, focusing on low-resource, morphologically complex translation. It probes both in-domain and out-of-domain generalization, as well as the reliability of automatic metrics versus human judgment for polysynthetic languages. Use when the user wants to benchmark on ChrEn, or asks about evaluating this task. Reports BLEU.

Compute the CHRFScore metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute CHRFScore, or asks how to score with CHRFScore.

Evaluates deep learning models for simultaneous multi-organ segmentation in 2D plant images. It probes the model's ability to accurately delineate root systems and aerial parts across different plant species and light conditions, while preserving structural fidelity for downstream phenotypic analysis. Use when the user wants to benchmark on Arabidopsis thaliana 2D phenotyping dataset, Tomato 2D phenotyping dataset, or asks about evaluating this task. Reports Dice coefficient.

Evaluates time series forecasting models on in-domain and zero-shot benchmarks across diverse domains and frequencies. It probes a model's ability to generalize to unseen temporal patterns using both probabilistic and point forecast metrics. Use when the user wants to benchmark on Benchmark I, Benchmark II, or asks about evaluating this task. Reports WQL.

Evaluates large language models' ability to understand and explain culturally nuanced Chinese internet humor. It measures how well models can generate human-preferred, two-sentence explanations for jokes derived from the Chinese platform Ruo Zhi Ba. Use when the user wants to benchmark on Chumor 1.0, or asks about evaluating this task. Reports winning rate.

Evaluates the ability of vision-language models and OCR systems to accurately transcribe historical documents, including both printed and handwritten text across diverse languages and scripts. It probes robustness to long-term document degradation, variable layouts, and long-context inputs in zero-shot and fine-tuned settings. Use when the user wants to benchmark on Churro-DS, or asks about evaluating this task. Reports normalized Levenshtein similarity.

Evaluates the classification accuracy and adversarial robustness of a Graph Neural Network-based Network Intrusion Detection System (NIDS) on distinguishing benign traffic from various attack types in network flow data. Use when the user wants to benchmark on CIC-IDS2017, or asks about evaluating this task. Reports weighted F1-score.

Evaluates the capability of machine learning models (traditional classifiers and CNNs on barcode-encoded features) to classify malware samples into benign or specific malware families. It probes how well structural patterns in 2D barcodes (QR and Aztec codes) capture executable features for downstream classification tasks. Use when the user wants to benchmark on CIC-MalMem-2022, or asks about evaluating this task. Reports accuracy.

Evaluates the adversarial robustness of tree ensemble models (RF, XGB, LGBM, EBM) on enterprise network intrusion detection using the CICIDS2017 dataset. It measures how well models maintain detection performance on benign and malicious traffic when subjected to constrained adversarial perturbations of time-series traffic features. Use when the user wants to benchmark on CICIDS2017, or asks about evaluating this task. Reports F1S.

Evaluates network intrusion detection capability by classifying network traffic flows as benign or malicious (or specific attack types) using graph-structured representations of network connections. It probes the model's ability to learn from adaptive graph construction and contrastive learning under resource-constrained conditions. Use when the user wants to benchmark on CICIDS2017, or asks about evaluating this task. Reports accuracy.

Evaluates incremental federated learning models for intrusion detection in IoT networks under evolving threat distributions. Probes the model's ability to adapt to concept drift over time while mitigating catastrophic forgetting in a federated setting. Use when the user wants to benchmark on CICIoMT2024, or asks about evaluating this task. Reports Accuracy (Acc).

Binary classification of real versus AI-generated synthetic images. It probes a model's ability to detect subtle background imperfections and artifacts introduced by latent diffusion models rather than semantic object content. Use when the user wants to benchmark on CIFAKE, or asks about evaluating this task. Reports accuracy.

Evaluates the robustness of adversarially trained neural networks against Projected Gradient Descent (PGD) attacks on CIFAR-10 and CIFAR-100. It measures both clean (natural) classification accuracy and robust accuracy under varying attack strengths (PGD-20 and PGD-100). Use when the user wants to benchmark on CIFAR-10, CIFAR-100, or asks about evaluating this task. Reports PGD-20 accuracy.

Evaluates the generalization and uncertainty quantification of Bayesian Neural Networks trained with novel Jensen-Shannon divergence loss functions compared to standard KL divergence, specifically under noisy and class-biased data conditions. Use when the user wants to benchmark on CIFAR-10, Breast Histopathology Dataset, or asks about evaluating this task. Reports validation accuracy.

This evaluation protocol probes a model's ability to perform standard supervised image classification and adaptive few-shot episodic learning. It measures how well the model generalizes to unseen classes under limited supervision by averaging accuracy over multiple sampled episodes. Use when the user wants to benchmark on CIFAR-100, Mini-ImageNet, or asks about evaluating this task. Reports accuracy.

Probes the accuracy of automatically generated 3D human pose and translation annotations for rock climbing motions. It evaluates how well a LiDAR-IMU fusion and blending optimization pipeline reconstructs off-ground climbing poses compared to manual ground truth. Use when the user wants to benchmark on CIMI4D, or asks about evaluating this task. Reports PMPJPE.

Evaluates multimodal large language models and video generation models on fine-grained cinematographic understanding (shot scale, angle, composition, camera movement, lighting, color, focal length) and camera movement generation from video clips. Use when the user wants to benchmark on CineTechBench, or asks about evaluating this task. Reports accuracy.

Evaluates image classification performance under significant domain shift between synthetic (CIFAR-10) and real-world/downsampled (ImageNet) sources. Probes model robustness to distributional bias and class-level statistical divergence across training and test domains. Use when the user wants to benchmark on CINIC-10, or asks about evaluating this task. Reports Test Error.

This benchmark evaluates whether large language models can generate secure cryptographic Python code and avoid common implementation flaws under varying security guidance. It probes the model's ability to follow secure prompting instructions, correctly implement cryptographic primitives, and avoid known anti-patterns like weak hashing or fixed IVs. Use when the user wants to benchmark on CIPHER, or asks about evaluating this task. Reports vulnerability_rates.

Evaluates a multimodal agent's ability to perform fine-grained visual classification and cultural reasoning on antique Chinese porcelain. It probes seven specific connoisseurship attributes (dynasty, reign period, kiln site, glaze color, decorative motif, vessel shape, and overall naming) through both multiple-choice and free-form generation tasks. Use when the user wants to benchmark on CiQi-Bench, or asks about evaluating this task. Reports accuracy.

Evaluates large language models' ability to interpret analog circuit diagrams and netlists, and perform multi-level reasoning to calculate correct numerical values for circuit parameters. Use when the user wants to benchmark on CIRCUIT, or asks about evaluating this task. Reports accuracy.

Evaluates composed image retrieval capabilities on culturally specific Thangka imagery. It tests the model's ability to align fine-grained sketch+text queries with target images across varying levels of textual semantic granularity, highlighting the domain gap between generic pre-training and specialized cultural retrieval. Use when the user wants to benchmark on CIRThan, or asks about evaluating this task. Reports Recall@K (R@K).

Evaluates the ability of citation recommendation systems to retrieve and rank relevant academic papers given a query context. It probes ranking quality, recall of relevant candidates, and normalized discounted cumulative gain across multiple academic datasets. Use when the user wants to benchmark on ACL-200, FullTextPeerRead, Refseer, arXiv, ArSyTa, or asks about evaluating this task. Reports MRR.

Evaluates the capability of citation recommendation models to identify relevant academic references given local citation contexts. It probes robustness across varying contextual features, including context length, reference position, academic field, publication year, citation count, and part-of-speech tags. Use when the user wants to benchmark on S2ORC/S2AG Diagnostic Datasets, or asks about evaluating this task. Reports MRR.

This benchmark evaluates a model's ability to generate concise, single-sentence summaries of scientific papers using citation sentences as ground truth. It probes extreme summarization capabilities and domain adaptation across academic disciplines. Use when the user wants to benchmark on CiteSum, or asks about evaluating this task. Reports ROUGE-1, ROUGE-2, ROUGE-L.

Evaluates real-time dynamic pedestrian intrusion detection from moving camera views, jointly performing area-of-interest segmentation and pedestrian detection to classify whether a pedestrian has intruded into a dynamic zone. It measures classification accuracy, segmentation quality, and detection precision while tracking computational efficiency. Use when the user wants to benchmark on Cityintrusion, Cityperson, Cityscape, or asks about evaluating this task. Reports PID_Acc.

Evaluates semantic scene understanding models on complex urban street scenes by measuring pixel-level classification accuracy and instance-level segmentation quality. It probes the model's ability to handle high-resolution imagery, diverse weather/lighting conditions, and fine-grained class distinctions in autonomous driving contexts. Use when the user wants to benchmark on Cityscapes, or asks about evaluating this task. Reports IoU.

Evaluates a RoBERTa-based classifier's ability to detect toxic or harmful content in online comments. It probes the model's sensitivity to explicit lexical cues versus implicit, context-dependent toxicity, highlighting failure modes that aggregate accuracy metrics miss. Use when the user wants to benchmark on Civil Comments, or asks about evaluating this task. Reports Accuracy.

This benchmark evaluates multimodal complaint analysis by measuring a model's ability to jointly process multi-turn textual dialogues and accompanying images to classify fine-grained aspects and severity levels of customer grievances. It probes cross-modal alignment, multi-label classification, and robustness to class imbalance and subjective tone variations. Use when the user wants to benchmark on CIViL, or asks about evaluating this task. Reports macro F1-score.

Compute ckb/unigram via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of ckb/unigram.

Evaluates a model's ability to classify cognitive load levels from raw, multimodal physiological signals (EEG, ECG, EDA) collected during driving scenarios. It probes the model's capacity to learn temporal and cross-modal patterns without hand-crafted features. Use when the user wants to benchmark on CL-Drive, or asks about evaluating this task. Reports accuracy.

Evaluates continual learning strategies for 3D engineering regression tasks, measuring their ability to learn from sequential data streams while mitigating catastrophic forgetting and maintaining predictive accuracy across parametric and point cloud modalities. Use when the user wants to benchmark on SplitSHIPD-Par, SplitSHIPD-PC, SplitSHAPENET, SplitRAADL, SplitDRIVAERNET, SplitDRIVAERNET++-Par, SplitDRIVAERNET++-PC, or asks about evaluating this task. Reports MPE.

Assesses mathematical reasoning and symbolic computation capabilities of LLMs across multiple languages. It uses dynamic, variable-driven templates to generate verifiable ground truths for each instance. The evaluation probes model resilience to linguistic variations and template-specific weaknesses. Use when the user wants to benchmark on CL-GSMSym, or asks about evaluating this task. Reports accuracy.

Evaluates large language models' ability to follow complex, variable-driven instructions across multiple languages. It measures strict compliance with prompt constraints to reveal cross-lingual robustness disparities. The benchmark highlights how functional tasks expose performance gaps that static benchmarks often miss. Use when the user wants to benchmark on CL-IFEval, or asks about evaluating this task. Reports Strict Prompt Accuracy.

Evaluates the robustness of classical neural networks and quantum neural networks against adversarial attacks by measuring performance degradation on a malware classification task after injecting random noise into input features. Use when the user wants to benchmark on ClaMP_Integrated, or asks about evaluating this task. Reports accuracy.

Evaluates a model's ability to classify symbolic music into genres, emotions, or composer styles, and to perform cross-modal semantic search between music scores (ABC/MIDI) and textual descriptions. It also probes multilingual retrieval capabilities by testing performance across machine-translated text queries. Use when the user wants to benchmark on WikiMT, VGMIDI, Pianist8, MidiCaps, or asks about evaluating this task. Reports Accuracy, MRR.

This benchmark evaluates query-conditioned target sound extraction (TSE), testing a model's ability to isolate a target audio source from a mixture using language captions or reference audio queries. It probes multi-modal query processing and positive/negative query valence across diverse acoustic environments and musical instruments. Use when the user wants to benchmark on AudioCaps, AudioSet, ESC-50, FSDKaggle2018, MUSIC21, or asks about evaluating this task. Reports SDRi, SISDRi.

Evaluates neural scene reconstruction and semantic segmentation capabilities on aerial UAV imagery. Probes the model's ability to generate high-fidelity 3D reconstructions, depth maps, and class-aware segmentation masks from multi-view inputs under varying scene complexities and viewpoint distributions. Use when the user wants to benchmark on ClaraVid, UAVid, or asks about evaluating this task. Reports reconstruction results.

Evaluates large language models on European Portuguese across cultural alignment, safety safeguards, chain-of-thought reasoning, natural language understanding, and common NLU tasks. It probes how well models handle culture-specific implicit knowledge, refuse harmful requests, and perform multiple-choice or generative QA in Portuguese. Use when the user wants to benchmark on Tuguesice-PT, DoNotAnswer-PT, MuSR, AA-Omniscience-Public, GPQA Diamond, MMLU, MMLU Pro, CoPA, MRPC, RTE, or asks about...

Evaluates the effectiveness of various language steering methods in large language models across 32 languages. It measures how well interventions force the model to output in a target language while preserving the semantic relevance of the response. Use when the user wants to benchmark on CLaS-Bench, or asks about evaluating this task. Reports steering score.

This benchmark probes multimodal large language models' ability to detect cross-modal contradictions between images and text. It evaluates whether models can identify inconsistencies when either modality contains errors or hallucinations, rather than assuming one modality is ground truth. The task reveals systematic modality biases and category-specific reasoning weaknesses. Use when the user wants to benchmark on CLASH, or asks about evaluating this task. Reports accuracy.

Predicts the outcome (win or lose) of U.S. class action lawsuits based on plaintiff complaint texts. It probes a model's ability to extract legally relevant allegations from long-form, unverified legal documents and make binary judgment predictions. Use when the user wants to benchmark on ClassActionPrediction, or asks about evaluating this task. Reports accuracy.

Evaluates large language models' ability to generate complete Python classes with interdependent methods, rather than standalone functions. It probes long-context code reasoning, dependency modeling, and the effectiveness of holistic versus incremental generation strategies. Use when the user wants to benchmark on ClassEval, or asks about evaluating this task. Reports pass@1.

Compute the ClasswiseWrapper metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute ClasswiseWrapper, or asks how to score with ClasswiseWrapper.

Evaluates the computational efficiency and 3D reconstruction accuracy of a GPU-accelerated binary feature descriptor (CLATCH) compared to traditional and deep learning-based descriptors within a Structure-from-Motion pipeline. Use when the user wants to benchmark on Photogrammetry Image Sets (8 scenes), or asks about evaluating this task. Reports SfM Scene RMSE (pixels).