Claude Skills by qhjqhj00

Evaluates embodied agents' navigation capabilities in photorealistic 3D Gaussian Splatting environments versus traditional mesh-based simulators. It probes cross-domain generalization, visual robustness, and human-aware collision avoidance in dynamic scenes. Use when the user wants to benchmark on InteriorGS + Real-world GS, Habitat-Matterport 3D (HM3D), AnimatableGaussians, or asks about evaluating this task. Reports SR, SPL.

Probes the ability of a runtime-based predictor to accurately estimate GPU training iteration execution times and cost-normalized throughput across different DNN architectures and GPU generations without requiring full training runs. Use when the user wants to benchmark on ImageNet, WMT'16, LSUN, or asks about evaluating this task. Reports average prediction error.

Compute hack/test_metric via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of hack/test_metric.

Evaluates language models' proficiency in Korean cultural knowledge and context. It probes capabilities across vocabulary (loan words, standard nomenclature, rare words), history, general knowledge, and reading comprehension, specifically highlighting the limitations of English-trained or non-Korean-tailored models. Use when the user wants to benchmark on HAE-RAE Bench, or asks about evaluating this task. Reports accuracy.

Compute hage2000/code_eval_stdio via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of hage2000/code_eval_stdio.

Compute hage2000/my_metric via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of hage2000/my_metric.

Evaluates Large Vision-Language Models (LVLMs) for hallucinations by measuring their ability to generate faithful image descriptions (generative evaluation) and detect hallucinations in provided captions (discriminative evaluation). It specifically probes fine-grained hallucination categories: object, relation, attribute, and event hallucinations, while also analyzing the impact of output length and Chain-of-Thought prompting. Use when the user wants to benchmark on COCO 2014, or asks about e...

Evaluates zero-shot text-to-speech synthesis capability for generating minute-long audio from text and a short reference prompt. It measures linguistic accuracy, speaker similarity, audio quality, and temporal dynamics against ground truth speech. Use when the user wants to benchmark on MinutesSpeech, LibriSpeech, or asks about evaluating this task. Reports WER.

This benchmark probes the hallucination detection capabilities of Large Audio-Language Models (LALMs) across speech, environmental sound, and music domains. It systematically induces hallucinations using adversarial prompts and mixed-audio inputs to evaluate response correctness, affirmative bias, and refusal behavior beyond standard accuracy. Use when the user wants to benchmark on HalluAudio, or asks about evaluating this task. Reports Accuracy.

Evaluates the ability of various detection methods to identify hallucinations in financial question-answering systems augmented with knowledge graphs. It probes robustness to noisy or contradictory KG triplets by comparing performance with and without structured evidence. Use when the user wants to benchmark on HalluBench, or asks about evaluating this task. Reports F1.

Probes a model's ability to avoid generating factually incorrect statements about visual content. It measures alignment between model outputs and ground-truth visual facts using binary detection and scoring metrics. Use when the user wants to benchmark on POPE, AMBER-d, HallusionBench, or asks about evaluating this task. Reports Accuracy (Acc).

Evaluates the ability of Large Vision-Language Models to generate factually aligned outputs by measuring object hallucination rates in captions and yes/no answers, as well as logical reasoning and attribute consistency across diverse visual prompts. Use when the user wants to benchmark on POPE, CHAIR, MMHal-Bench, or asks about evaluating this task. Reports POPE Average Accuracy.

Evaluates whether reinforcement finetuned language models appropriately refuse to answer unanswerable or ambiguous questions, and measures their accuracy on standard solvable math benchmarks to ensure performance is not degraded by the refusal training. Use when the user wants to benchmark on UWMP, SelfAware, Synthetic Unanswerable Math (SUM), GSM8K, Minerva, MATH-500, OlympiadBench, AMC23, or asks about evaluating this task. Reports refusal_rate.

This evaluation framework quantifies two distinct types of hallucinations in large language models: factuality (truthfulness of generated information) and faithfulness (consistency with input context or instructions). It probes model performance across 15 diverse knowledge-intensive tasks, including closed-book QA, summarization, reading comprehension, and fact-checking, using zero-shot and few-shot in-context prompts. Use when the user wants to benchmark on FEVER, FaithDial, NQ-open, TriviaQ...

Evaluates LVLMs' ability to resist prompt-induced hallucinations by disentangling perception failures from instruction-induced presuppositions. It probes whether models rely on visual evidence or textual priors when answering questions that imply the presence of non-existent objects. Use when the user wants to benchmark on HalluScope, or asks about evaluating this task. Reports AdP.

Evaluates an agent's ability to learn humanlike abstractions and affordances for rapid problem solving in a structured visual game. It probes three levels of generalization: perceptual recognition, conceptual abstraction of semantics, and algorithmic strategy formation under limited training exposure. Use when the user wants to benchmark on HALMA, or asks about evaluating this task. Reports goal_reaching ($ ho_g$).

Evaluates language models' ability to capture authorial style and stylometric patterns in scholarly text, independent of topical content. It probes whether models can distinguish documents by the same author across different topics and languages using triplet classification and document retrieval tasks. Use when the user wants to benchmark on HALvest-Contrastive, or asks about evaluating this task. Reports Accuracy.

Compute the hamming_loss metric — provided by scikit-learn. Use when the user has predictions and ground-truth and needs to compute hamming_loss, or asks how to score with hamming_loss.

Compute the HammingDistance metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute HammingDistance, or asks how to score with HammingDistance.

Evaluates the ability of a model to personalize a 3D hand avatar from a single RGB image and render it under novel poses and lighting conditions. It probes physically-based rendering accuracy, material/albedo recovery, and relighting generalization. Use when the user wants to benchmark on InterHand2.6M, HARP relit, or asks about evaluating this task. Reports PSNR.

Evaluates the accuracy of 3D hand pose estimation methods by measuring joint localization error on isolated frame pairs. It probes how well different directional distance metrics handle orientation information and varying temporal offsets between frames. Use when the user wants to benchmark on Synthetic dataset, Realistic dataset, or asks about evaluating this task. Reports average joint error.

Evaluates sequential dexterous manipulation by requiring a robot to first grasp a target object and then perform a specific downstream task (e.g., pushing, pressing, twisting, pulling, or picking a second object) while maintaining the grasp. It probes the policy's ability to allocate finger resources and maintain stable contacts to satisfy competing subtask constraints. Use when the user wants to benchmark on HANDFUL-Bench, or asks about evaluating this task. Reports terminal success rate ($p...

Evaluates a robot's ability to understand ambiguous human instructions and infer human subgoals in physically and socially complex household environments. It probes pragmatic reasoning, goal recognition, and collaborative task completion under partial observability. Use when the user wants to benchmark on HandMeThat, or asks about evaluating this task. Reports success rate.

Evaluates video foundation models' ability to understand fine-grained spatiotemporal dynamics in hand-object interactions. It probes spatial reasoning, motion tracking, and part-level geometric grounding through multiple-choice questions and video object segmentation tasks. Use when the user wants to benchmark on HanDyVQA, or asks about evaluating this task. Reports top-1 accuracy.

Probes whether neural NLI models rely on superficial syntactic heuristics (e.g., lexical overlap, subsequence matching) rather than genuine logical reasoning by presenting structurally similar counterexamples where heuristics lead to incorrect predictions. Use when the user wants to benchmark on HANS, or asks about evaluating this task. Reports accuracy.

Evaluates Chinese entity linking models on few-shot and zero-shot scenarios, specifically probing their ability to link mentions to tail and emerging Wikidata entities without relying on head entity popularity or dataset-specific fine-tuning. Use when the user wants to benchmark on Hansel, TAC-KBP2015, or asks about evaluating this task. Reports R@1.

Evaluates the ability of classical, deep learning, and generative models to accurately classify human activities from sensor data. Probes temporal pattern recognition, sensor fusion handling, and generalization across varying data complexities and sensor modalities. Use when the user wants to benchmark on UCI-HAR, Opportunity, PAMAP2, WISDM, Berkeley MHAD, or asks about evaluating this task. Reports Accuracy.

This benchmark evaluates continual learning algorithms on sensor-based human activity recognition (HAR) datasets. It measures how well models balance plasticity (learning new activities) and stability (retaining old activities) while incrementally processing tasks, specifically probing robustness to class imbalance, sensor noise, and cross-user data leakage. Use when the user wants to benchmark on House A (HA), CASAS (WS, Milan, Twor, Aruba), PAMAP2, DSADS, HAPT, or asks about evaluating this...

Probes the ability of multimodal and unimodal models to recognize human activities from wearable sensor, pose, and video data. It evaluates classification performance, data efficiency across low-data regimes (1-100% training fractions), and zero-shot transfer capability to unseen real-world datasets. Use when the user wants to benchmark on MM-Fit, MHEALTH, MyoGym, MotionSense, or asks about evaluating this task. Reports Macro F1-Score.

Evaluates deep learning architectures (CNNs, LSTMs, DNNs) for frame-by-frame human activity recognition using wearable sensor time-series data. It probes the models' capacity to capture temporal dependencies and generalize across diverse domains (kitchen gestures, lifestyle/exercise, and medical gait analysis) while handling severe class imbalance. Use when the user wants to benchmark on Opportunity, PAMAP2, Daphnet Gait, or asks about evaluating this task. Reports mean f1-score.

Evaluates the ability of lightweight convolutional neural networks to accurately classify human activities from wearable sensor time-series data. It probes the trade-off between model compression (parameter count and FLOPs) and classification performance on highly imbalanced, multi-class activity recognition tasks. Use when the user wants to benchmark on UCI-HAR, OPPORTUNITY, PAMAP2, UNIMIB-SHAR, WISDM, or asks about evaluating this task. Reports weighted F1 score.

Evaluates whether LLMs are vulnerable to draft-based co-authoring jailbreaks that exploit collaborative writing contexts to elicit harmful completions. It probes the model's ability to detect concealed malicious intent in incomplete drafts and assesses the trade-off between safety refusal and writing utility. Use when the user wants to benchmark on HarDBench, or asks about evaluating this task. Reports Harmfulness Score (HS).

Evaluates multi-modal human activity recognition capabilities by classifying 300 action categories from synchronized RGB frames and asynchronous event streams under challenging real-world conditions such as low light, occlusion, and dynamic backgrounds. Use when the user wants to benchmark on HARDVS 2.0, or asks about evaluating this task. Reports accuracy.

Evaluates the clinical quality and diagnostic alignment of machine-generated histopathology reports by measuring semantic alignment of extracted pathological entities and their interrelations against ground truth reports. It probes a model's ability to accurately capture domain-specific terminology, diagnostic conclusions, and their contextual connections. Use when the user has predictions and gold and needs to compute HARE Score.

Evaluates the robustness of LLM safety defenses against multi-turn human and automated jailbreak attacks. It probes whether current refusal mechanisms and machine unlearning methods can withstand adversarial red teaming aimed at recovering harmful or dual-use knowledge. Use when the user wants to benchmark on HarmBench, WMDP-Bio, or asks about evaluating this task. Reports ASR.

Evaluates multimodal large language models' ability to align video and music across four dimensions: rhythmic synchronization, thematic coherence, emotional congruence, and cultural relevance. It tests both open-ended descriptive reasoning and multiple-choice selection to measure temporal and semantic alignment capabilities. Use when the user wants to benchmark on HarmonySet, or asks about evaluating this task. Reports HarmonySet-OE Score.

Evaluates out-of-distribution (OOD) generalization in sensor-based human activity recognition (HAR) across four realistic domain-shift scenarios: cross-person, cross-position, cross-device, and cross-time. It benchmarks how well 16 OOD methods with CNN or Transformer backbones maintain performance when trained on one domain and tested on unseen domains. Use when the user wants to benchmark on DSADS, or asks about evaluating this task. Reports accuracy.

This benchmark evaluates a model's ability to recommend relevant hashtags for real-world social media images using only visual input. It probes contextual image understanding and multi-label classification by measuring how well predicted hashtags align with actual user-generated tags. Use when the user wants to benchmark on HARRISON, or asks about evaluating this task. Reports Precision@1.

Compute harshhpareek/bertscore via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of harshhpareek/bertscore.

Evaluates deep learning models for eye disease screening (AMD, DR, glaucoma) on 2D fundus and 3D OCT images. It measures both overall diagnostic performance and demographic fairness across race, gender, and ethnicity to assess equitable model behavior. Use when the user wants to benchmark on Harvard-EF30k, or asks about evaluating this task. Reports AUC.

Evaluates the ability of computer vision models to classify hand shadow puppet silhouettes into one of 15 distinct categories. It probes feature extraction robustness, particularly for rotationally asymmetric and visually similar silhouettes under varying lighting and motion dynamics. Use when the user wants to benchmark on HaSPeR, or asks about evaluating this task. Reports top-1 accuracy.

Evaluates binary and multi-label hate speech detection models on Brazilian Portuguese text. It specifically probes a model's sensitivity to targeted minority groups and its ranking quality under severe class imbalance. Use when the user wants to benchmark on ToxiGen-PT, Portuguese Superset Benchmark, HateBR, OLID-BR, TuPy-E, ToLD-BR, or asks about evaluating this task. Reports Macro-Recall.

Evaluates a model's ability to distinguish between hate speech, offensive language, and neutral text in social media posts. It probes the classifier's sensitivity to contextual nuances, reclaimed slurs, and demographic-specific biases in labeling. Use when the user wants to benchmark on Hate Speech and Offensive Language Dataset, or asks about evaluating this task. Reports F1 score.

Evaluates deep learning models' ability to predict continuous, interval-scaled hate speech scores from raw text comments. It benchmarks against existing APIs and transformer baselines using cross-validated error and correlation metrics. Use when the user wants to benchmark on Custom hate speech corpus (YouTube, Reddit, Twitter), or asks about evaluating this task. Reports RMSE.

Evaluates multimodal models' ability to detect hateful or offensive memes across multiple domains and under low-resource, out-of-distribution conditions. It probes robustness to distribution shifts, adversarial image perturbations, and the effectiveness of retrieval-augmented inference versus standard fine-tuning or in-context learning. Use when the user wants to benchmark on HatefulMemes, HarMeme, MAMI, Harm-P, MultiOFF, PrideMM, or asks about evaluating this task. Reports AUC.

Evaluates multimodal models' ability to detect hateful or harmful memes by analyzing the alignment between image and text content. It probes robustness against visual and textual confounders that appear benign individually but become harmful when combined. Use when the user wants to benchmark on HatefulMemes, HarMeme, or asks about evaluating this task. Reports AUC.

Evaluates embodied agents' decision-making and planning capabilities in dynamically changing disaster environments (fire, flood, wind). It probes the ability to reason about evolving object states, environmental propagation dynamics, and spatial-temporal trade-offs to successfully rescue valuable items. Use when the user wants to benchmark on HAZARD, or asks about evaluating this task. Reports rescued value rate (Value).

Evaluates the ability to distinguish AI-generated responses from human expert answers and assesses perceived helpfulness across multiple domains. It probes linguistic realism, factual reliability, and stylistic alignment with human communication. Use when the user wants to benchmark on HC3, or asks about evaluating this task. Reports detection accuracy.

Evaluates atmospheric retrieval models on exoplanet transmission spectra to constrain chemical abundances, temperature-pressure profiles, and cloud properties. It probes the model's ability to disentangle spectral features across multi-wavelength observations and quantify detection significance of trace gases. Use when the user wants to benchmark on JWST NIRCam transmission spectra, HST WFC3 transmission spectra, HST STIS transmission spectra, or asks about evaluating this task. Reports reduc...

Evaluates end-to-end autonomous driving planning models in real-world closed-loop scenarios, measuring success rate, trajectory stability, and safety compliance during urban driving. Use when the user wants to benchmark on Real-world driving dataset, or asks about evaluating this task. Reports closed-loop success rate.