Claude Skills by qhjqhj00

Tests an agent's capacity to handle complex constraint satisfaction by managing and resolving conflicting schedules derived purely from raw natural language descriptions. It simulates personal assistant scenarios where the model must maintain a high-density representation of events to detect conflicts accurately. Use when the user wants to benchmark on Calendar Scheduling, or asks about evaluating this task. Reports Exact Match (EM).

Evaluates the calibration quality of 3D scene understanding models by measuring the discrepancy between predicted confidence and actual accuracy. It probes reliability under both in-domain conditions and out-of-domain stressors like adverse weather, sensor failures, and domain shifts. Use when the user wants to benchmark on nuScenes, SemanticKITTI, Waymo Open, SemanticPOSS, SemanticSTF, ScribbleKITTI, Synth4D, S3DIS, or asks about evaluating this task. Reports ECE.

Measures the semantic novelty of LLM-generated text by quantifying its similarity to the closest segment in the model's pretraining corpus. It probes whether models merely reproduce memorized training data or generalize to produce compositionally distinct outputs. Use when the user has predictions and gold and needs to compute calibrated similarity.

Evaluates the sample complexity and verification cost required to estimate calibration error in AI models under rare-error regimes. It probes whether passive querying or active querying can reliably detect miscalibration and how estimation error scales with sample size and model smoothness. Use when the user has predictions and gold and needs to compute estimation error.

Compute the CalibrationError metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute CalibrationError, or asks how to score with CalibrationError.

Evaluates multi-objective ad ranking models on click-through rate (CTR) and conversion rate (CVR) prediction tasks, focusing on ranking quality, score calibration, and counterfactual utility estimation under position and selection bias. Use when the user wants to benchmark on Criteo, Avazu, or asks about evaluating this task. Reports AUC.

Evaluates the scalability and logical correctness of OWL2 EL reasoners when processing large ontologies with complex owl:hasValue restrictions. It measures whether systems can successfully materialize subclass hierarchies and infer individual and literal assertions, and how long they take under memory and time constraints. Use when the user wants to benchmark on CaLiGraph, or asks about evaluating this task. Reports Inferrable Assertions.

Compute the calinski_harabasz_score metric — provided by scikit-learn. Use when the user has predictions and ground-truth and needs to compute calinski_harabasz_score, or asks how to score with calinski_harabasz_score.

Compute the CalinskiHarabaszScore metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute CalinskiHarabaszScore, or asks how to score with CalinskiHarabaszScore.

Evaluates the ability of a curiosity-driven reinforcement learning auditor to autonomously generate prompts that elicit harmful, toxic, or target-specific outputs from black-box LLMs without parameter access. It measures how efficiently the auditor explores the prompt space to uncover rare or sensitive model behaviors. Use when the user wants to benchmark on Inverse Suffix Generation Task, Toxic Completion Task, or asks about evaluating this task. Reports Auditing Objective.

Evaluates deep learning models for high-energy physics calorimetry tasks, specifically particle shower generation and particle reconstruction (identification and energy regression) using simulated detector data. Use when the user wants to benchmark on LCD Calorimeter Dataset (GEN & REC), or asks about evaluating this task. Reports accuracy.

Evaluates a robot policy's ability to chain multiple sub-goals sequentially using only visual observations and goal images. It probes long-horizon planning, goal-conditioned control, and the capacity to generalize to unseen goal configurations without explicit reward signals. Use when the user wants to benchmark on CALVIN, or asks about evaluating this task. Reports Success rate.

Evaluates camera-only 4D occupancy forecasting for autonomous driving by predicting current and future voxel states in a 3D grid. It specifically probes the model's ability to distinguish between general movable objects (GMO) and general static objects (GSO) across multiple future time steps. Use when the user wants to benchmark on nuScenes, nuScenes-Occupancy, Lyft-Level5, or asks about evaluating this task. Reports ~IoU_f.

Evaluates a multimodal framework's ability to classify thoracic diseases at the study level by jointly modeling multi-view chest X-rays, clinical indications, and vital signs. It probes the model's capacity to integrate heterogeneous clinical data for accurate multi-label diagnosis across head, body, and tail disease categories. Use when the user wants to benchmark on MIMIC-CXR, CXR-LT 2023, CXR-LT 2024, or asks about evaluating this task. Reports mAP.

Evaluates a multimodal language model's ability to process long-duration ECG signals for clinical forecasting, diagnostic classification, report generation, and statistical grounding. It probes temporal reasoning, multi-lead interpretation, and instruction-following in medical AI settings. Use when the user wants to benchmark on Icentia11k, PTB-XL, CSN, CODE-15%, CPSC-2018, HEEDB, Penn, MIMIC-IV-ECG, ECGBench / ECG-QA, ECG Grounding Benchmark, or asks about evaluating this task. Reports F1 sc...

Evaluates domain generalization capability for metastatic breast cancer detection in histopathology images. The protocol trains models on data from known medical centers and tests them on completely unseen centers with different staining protocols and scanners to measure adaptability. Use when the user wants to benchmark on Camelyon17 WILDS, or asks about evaluating this task. Reports accuracy (%).

Evaluates speech emotion recognition (SER) models across multiple languages and emotional states. It probes a model's ability to map raw audio inputs to discrete emotional categories without relying on speaker or language metadata. Use when the user wants to benchmark on CAMEO, or asks about evaluating this task. Reports macro-averaged F1 score.

Evaluates a generative model's ability to simulate physical camera effects (bokeh, focal length, shutter speed, color temperature) while preserving scene consistency and adhering to text prompts. Use when the user wants to benchmark on Custom Camera Control Dataset, or asks about evaluating this task. Reports CorrCoef.

Evaluates the efficiency-effectiveness trade-off of Structure-from-Motion (SfM) strategies for novel view synthesis. It probes how different feature extractors, matchers, and mappers impact rendering quality and computational runtime across diverse indoor and outdoor scenes. Use when the user wants to benchmark on Mip-NeRF 360, Tanks and Temples, Zip-NeRF, or asks about evaluating this task. Reports PSNR.

Evaluates AI-powered platforms for processing camera trap images, measuring their ability to detect animals and classify species against ground truth labels. Use when the user wants to benchmark on Colombian rainforest camera trap images, or asks about evaluating this task. Reports F1 score.

Evaluates fine-grained camera motion recognition in VideoLLMs using multiple-choice questions and multi-label classification. Probes whether models can distinguish geometric camera movements from object motion and static shots. Use when the user wants to benchmark on CameraMotionVQA, CameraMotionDataset, or asks about evaluating this task. Reports answer accuracy.

Evaluates large language models' ability to perform structured reasoning over temporally segmented in-vehicle CAN traffic logs. It probes capabilities in temporal analysis, multi-condition inference, and behavioral interpretation for automotive cybersecurity forensics. Use when the user wants to benchmark on CAN-QA, or asks about evaluating this task. Reports accuracy.

Evaluates deep learning models for high-resolution (100 m) wildfire forecasting using multi-modal satellite and environmental data. It probes the model's ability to predict fire occurrence at the patch level across different temporal splits and land cover types, particularly under severe class imbalance and varying fire danger conditions. Use when the user wants to benchmark on CanadaFireSat, or asks about evaluating this task. Reports F1 score.

Evaluates Vision-Language Models' ability to analyze multi-scale candlestick charts (daily and weekly) and predict 30-day forward stock returns. It probes their capacity for visual technical analysis, trend recognition, and regression-based financial forecasting without relying on textual market data. Use when the user wants to benchmark on Multi-Scale Candlestick Stock Return Benchmark, or asks about evaluating this task. Reports IC (Information Coefficient).

Evaluates context-aware neural machine translation models on their ability to correctly translate context-dependent discourse phenomena (e.g., anaphoric pronouns, deixis, ellipsis) across sentence boundaries in concatenated input windows. Use when the user wants to benchmark on En→Ru movie subtitles (Voita et al., 2019), En→De TED talk subtitles (IWSLT17), Voita contrastive set, ContraPro, or asks about evaluating this task. Reports Contrastive accuracy.

Evaluates large language models and specialized MT systems on culture-aware machine translation across 12 language pairs. It probes the models' ability to preserve cultural nuances and adapt to explicit semantic versus communicative translation constraints. Use when the user wants to benchmark on CanMT, or asks about evaluating this task. Reports translation performance.

Evaluates robotic navigation policies under diverse simulated environments, testing robustness to both precise and misleading natural language instructions. Use when the user wants to benchmark on CANVAS, or asks about evaluating this task. Reports success rate (%).

Evaluates an LLM's ability to selectively suppress specific domain knowledge (e.g., privacy or sensitive topics) while preserving general capabilities and language fluency. It measures both forgetting effectiveness and utility retention across generative and discriminative tasks using prompt-based steering rather than parameter editing. Use when the user wants to benchmark on RWKU (Forget QA), WMDP, MMLU, or asks about evaluating this task. Reports ASG (Average Similarity Gap).

Evaluates LLMs on standard benchmarks modified with randomized answers to measure performance tracking and detect data contamination via a Bayes accuracy ceiling. The protocol compares model accuracy against a predefined theoretical maximum (Bayes accuracy) to identify overfitting or memorization. It also assesses robustness to reverse-engineering attacks and cross-lingual contamination. Use when the user wants to benchmark on GSM8K, ARC-Challenge, GPQA, MathQA, MMLU, HLE-MC, MMLU-ProX, BoolQ...

This evaluation protocol assesses the effectiveness of context-aware position encoding methods in sequential recommendation systems. It measures how well models rank a target item given a user's historical interaction sequence, testing the model's ability to capture temporal and semantic dependencies in user behavior. Use when the user wants to benchmark on AmazonElectronics, KuaiVideo, AmazonBooks, or asks about evaluating this task. Reports AUC.

Evaluates the quality of dense image captions by measuring how well they enable downstream multimodal models to answer visual questions accurately. It probes fine-grained visual perception, structured description capability, and the utility of captions for non-visual reasoning. Use when the user wants to benchmark on InfoVQA, DocVQA, ChartQA, Real World QA, Math Vista, SEED2 Plus, MME, MMB, MMStar, MMVet, AI2D, GQA, MMMU, WeMath, or asks about evaluating this task. Reports accuracy.

This benchmark evaluates text-to-speech models on generating high-fidelity, intelligible speech conditioned on free-form natural language style captions. It probes the model's ability to control intrinsic speaker traits, expressive styles, accents, emotions, and integrate non-verbal sound events across diverse real-world scenarios. Use when the user wants to benchmark on CapTTS, EmoCapTTS, AccCapTTS, CapTTS-SE, AgentTTS, or asks about evaluating this task. Reports binary_correctness.

Evaluates the ability of protein sequence and structure models to predict the subcellular localization compartments of human proteins. It probes multi-label classification performance under severe class imbalance, testing whether models can leverage 3D structural motifs or sequence embeddings to identify fine-grained organelle targeting patterns. Use when the user wants to benchmark on CAPSUL, or asks about evaluating this task. Reports F1-score.

This benchmark evaluates whether model-generated image captions retain sufficient visual information to answer domain-specific multiple-choice questions without access to the original image. It measures caption utility for downstream reasoning by testing if a text-only QA model can reliably select correct answers or explicitly acknowledge missing information when prompted only with the caption. Use when the user wants to benchmark on CaptionQA, or asks about evaluating this task. Reports Capt...

This evaluation framework probes systematic capability drift and forgetting in large language models after post-training. It measures degradation across latent competence (knowledge, reasoning), default behavioral preferences (refusal, verbosity, formatting), and protocol compliance (instruction following, tool use, citation) in legal and medical domains. Use when the user wants to benchmark on CapTrack Evaluation Suite, or asks about evaluating this task. Reports average forgetting.

Evaluates the robustness of classifiers against universal and individual adversarial perturbations under domain-specific linear constraints. Probes the trade-off between attack success rate and computational efficiency across finance, network security, medical IoT, and cyber-physical systems. Use when the user wants to benchmark on LCLD, IDS, IoMT, SWaT, WADI, or asks about evaluating this task. Reports ASR.

Evaluates the quality and validity of citations in generated answers for complex question answering. It probes whether models can correctly classify attributions as supportive, insufficient, contradictory, or irrelevant, and assesses their ability to handle varying reasoning complexities. Use when the user wants to benchmark on CAQA, ACLE-Manual, or asks about evaluating this task. Reports micro-F1.

Evaluates LLM agents' ability to resolve uncertainty and adhere to safety policies in automotive environments. It probes limit-awareness, consistency across multiple attempts, and robust multi-turn tool use under incomplete or ambiguous user requests. Use when the user wants to benchmark on CAR-bench, or asks about evaluating this task. Reports Passˆ3.

Evaluates continual semi-supervised learning on activity recognition by measuring how well a model adapts to time-varying unlabeled data streams across sequential sessions without predefined class boundaries. Use when the user wants to benchmark on Continual Activity Recognition (CAR), or asks about evaluating this task. Reports F1-score (class average).

Evaluates the robustness and energy efficiency of a neuromorphic spiking neural network for real-time anomaly detection on lunar rover sensor telemetry. It specifically probes the model's ability to maintain classification accuracy under gradient-based and temporal adversarial attacks while measuring hardware-level power consumption. Use when the user wants to benchmark on Cislunar Anomaly and Risk Dataset (CARD), or asks about evaluating this task. Reports Adversarial Success Rate (ASR).

Evaluates multivariate time series forecasting models on capturing temporal and cross-channel dependencies across multiple real-world benchmarks. It probes the model's ability to predict future values over varying horizons using fixed historical lookback windows. Use when the user wants to benchmark on ETTm1, ETTm2, ETTh1, ETTh2, Weather, Electricity, Traffic, or asks about evaluating this task. Reports MSE.

Evaluates binary classification of ECG heartbeat signals into normal versus abnormal categories. Probes the effectiveness of feature extraction and optimization pipelines for medical signal processing. Use when the user wants to benchmark on MIT-BIH Arrhythmia database (subset), or asks about evaluating this task. Reports accuracy.

Evaluates automated segmentation and diagnostic classification capabilities on cardiac magnetic resonance imaging (CMR) sequences. Probes the model's ability to accurately delineate cardiac structures across multiple anatomical views and classify cardiovascular diseases with clinical-grade metrics. Use when the user wants to benchmark on BAAI Cardiac CMR Cohort, or asks about evaluating this task. Reports DSC, AUC.

Evaluates information extraction systems on clinical literature for fine-grained extraction of experimental findings, including entities, attributes, and complex n-ary relations with discontinuous spans and variable arity. Use when the user wants to benchmark on CARE, or asks about evaluating this task. Reports relaxed overlap F1.

Evaluates whether LLMs using retrieval-augmented generation can faithfully apply clinical guidelines (specifically Written Exposure Therapy) to answer questions. It probes context fidelity, reasoning complexity, and question type to measure if models actually ground their inferences in retrieved evidence rather than relying on parametric knowledge or guessing. Use when the user wants to benchmark on CARE-RAG WET Guidelines, or asks about evaluating this task. Reports Inference Score.

Evaluates multimodal fusion of Electronic Health Records (EHR) and Chest X-Rays (CXR) for clinical decision support, specifically testing robustness to missing modalities, temporal imbalance, and subgroup fairness across phenotyping, mortality, and length-of-stay prediction. Use when the user wants to benchmark on CareBench, or asks about evaluating this task. Reports AUROC, AUPRC, F1, Accuracy, Cohen's Kappa.

This benchmark evaluates large language models' ability to perform critical appraisal and methodological reasoning on biomedical scientific articles. It probes whether models can correctly identify study design flaws, statistical limitations, and biases by answering multiple-choice questions derived from authentic French medical exams. The evaluation specifically measures both exact correctness and partial reasoning accuracy under varying context conditions. Use when the user wants to benchma...

Evaluates the trustworthiness of Medical Large Vision-Language Models across five dimensions: trustfulness (factuality and uncertainty), fairness (demographic disparities), safety (jailbreaking, toxicity, overcautiousness), privacy, and robustness. It probes the models' ability to generate accurate medical information, recognize their own uncertainty, avoid demographic bias, resist adversarial prompts, and handle sensitive data without leakage. Use when the user wants to benchmark on CARES, I...

Evaluates large language models' causal reasoning capabilities across three task categories: causal graph reasoning (adjacency matrix, d-separation, causal direction), knowledge discovery from tabular data, and decision-making under interventions and counterfactuals. Use when the user wants to benchmark on CARL-GT, or asks about evaluating this task. Reports F1 score.

Evaluates the end-to-end inference latency of an autonomous driving pipeline that runs parallel reinforcement learning and object detection models in a simulated urban environment. It measures how efficiently the middleware handles communication and computation overhead during real-time sensor processing and action fusion. Use when the user wants to benchmark on CARLA simulator, or asks about evaluating this task. Reports inference_latency.