Claude Skills by qhjqhj00

Evaluates vision-language models' ability to solve multi-hop visual reasoning tasks by measuring how accurately their final predicted answers match the ground truth. It specifically probes the model's capacity for structured reasoning and answer extraction in complex domains like geometry, science, and visual question answering. Use when the user wants to benchmark on MAVIS-Geometry, A-OKVQA, GeoQA170K, CLEVR-Math, ScienceQA, or asks about evaluating this task. Reports accuracy.

This benchmark probes a model's ability to detect framing bias in multimodal news content (text-image pairs) and generate grounded, correct rationales for its decisions. It evaluates both closed-ended classification accuracy and open-ended reasoning quality using an LLM-as-judge protocol. Use when the user wants to benchmark on ViLBias, or asks about evaluating this task. Reports Accuracy.

Evaluates video-language continual learning by testing a model's ability to retain episodic memories across streaming, long-duration videos without catastrophic forgetting. It probes cross-modal inference and temporal localization across three non-classification tasks: moment queries, natural language queries, and visual queries. Use when the user wants to benchmark on ViLCo-Bench, or asks about evaluating this task. Reports Average Recall@k (IoU=m).

This evaluation probes a model's ability to generalise to novel robotic manipulation tasks by testing robustness to instruction variations and increased task difficulty. It specifically measures compositional generalisation capabilities across four systematicity levels, ranging from object pose sensitivity to entirely novel objects and tasks. Use when the user wants to benchmark on VIMABench, or asks about evaluating this task. Reports compositional generalisation capabilities.

Evaluates vision-language models on generating Vietnamese clinical reports from paired PET/CT images and answering medical questions about them. Probes the model's ability to align 3D medical imaging features with low-resource language text and produce clinically accurate descriptions. Use when the user wants to benchmark on ViMed-PET, or asks about evaluating this task. Reports BLEU-4.

This benchmark evaluates automatic speech recognition (ASR) models on Vietnamese medical audio containing embedded English terminology. It specifically probes the model's ability to accurately transcribe both the matrix language and code-switched segments, measuring overall transcription quality alongside specialized metrics for code-switched and non-code-switched spans. Use when the user wants to benchmark on ViMedCSS, or asks about evaluating this task. Reports WER.

Evaluates Vietnamese machine reading comprehension models on span extraction from passages, including handling unanswerable questions. It measures how well systems can locate exact answer spans or correctly identify when no answer exists in the context. Use when the user wants to benchmark on UIT-ViQuAD 2.0 (ViMRC), or asks about evaluating this task. Reports F1-score.

Evaluates a model's ability to predict the helpfulness of product reviews by jointly processing textual descriptions and visual content. It probes multimodal alignment and ranking capabilities in low-resource language settings, specifically Vietnamese. Use when the user wants to benchmark on ViMRHP, or asks about evaluating this task. Reports NDCG@K.

Evaluates video language models on multilingual, culturally-diverse video understanding across 14 languages and 15 domains. It probes the models' ability to answer multiple-choice and open-ended questions about short, medium, and long videos, with a specific focus on low-resource languages and cultural reasoning. Use when the user wants to benchmark on ViMUL-Bench, or asks about evaluating this task. Reports accuracy.

Probes a model's ability to generate executable, visually faithful code (HTML, SVG, LaTeX, SMILES) from input images across diverse domains. It evaluates both syntactic correctness via execution rate and perceptual alignment with target images using coarse-to-fine visual similarity metrics. Use when the user wants to benchmark on ChartMimic, Design2Code, UniSVG, Image2Struct, Cosyn-400k, or asks about evaluating this task. Reports UniSVG Final Score.

Evaluates a model's ability to answer clinical questions about chest X-rays and localize lesions via bounding boxes. It probes visual question answering, spatial grounding, and multi-task learning in a medical imaging context. Use when the user wants to benchmark on VinDr-CXR-VQA, or asks about evaluating this task. Reports F1 score.

Compute Viona/fuzzy_reordering via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of Viona/fuzzy_reordering.

Compute Viona/infolm via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of Viona/infolm.

Compute Viona/kendall_tau via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of Viona/kendall_tau.

Compute Vipitis/shadermatch via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of Vipitis/shadermatch.

Evaluates multimodal mathematical reasoning and high-resolution visual perception capabilities of vision-language models. It probes the model's ability to decompose complex problems into structured reasoning chunks, interleave visual tool calls, and produce accurate final answers across geometric, mathematical, and fine-grained visual benchmarks. Use when the user wants to benchmark on GeoQA, MathVista-Math, MMStar-Math, VisualProbe, V*, HR-Bench, or asks about evaluating this task. Reports a...

Evaluates computational tools and genomic features for predicting prokaryotic virus-host interactions. It probes the ability of models to correctly link viral sequences to their host taxa using either pairwise link prediction or taxonomic classification formulations. Use when the user wants to benchmark on RefSeq-VHDB, MetaHiC-VHDB, or asks about evaluating this task. Reports Top-1 accuracy.

Evaluates industrial anomaly detection and segmentation capabilities by measuring how well self-supervised pre-training methods transfer to identifying surface defects. It probes a model's ability to localize fine-grained anomalies in highly imbalanced, high-resolution industrial imagery under both one-class and few-shot supervised regimes. Use when the user wants to benchmark on VisA, MVTec-AD, or asks about evaluating this task. Reports AU-PR.

Evaluates the robustness of traffic sign recognition models against adversarial attacks (PGD) and distribution shifts (ImageNet-C corruptions, color quantization). It specifically probes multi-task learning models for spurious correlations across visual attributes (color, shape, symbol, text) by measuring error propagation and task-dependent vulnerability. Use when the user wants to benchmark on VISAT, or asks about evaluating this task. Reports epsilon (model error).

Evaluates the robustness of multimodal large language models (MLLMs) against vision-centric jailbreak attacks that inject realistic, image-driven contextual dialogues to elicit harmful responses. It probes safety alignment under adversarial multimodal prompts designed to bypass safety filters through semantic alignment and toxicity obfuscation. Use when the user wants to benchmark on MM-SafetyBench, SafeBench-Tiny, HarmBench, or asks about evaluating this task. Reports ASR.

Evaluates an AI agent's ability to maintain conversational context, resolve co-references, and ground follow-up questions in visual content. The task requires ranking a set of candidate answers based on an image and dialog history. Use when the user wants to benchmark on VisDial v0.9, or asks about evaluating this task. Reports MRR.

Evaluates the classification performance of LLM-generated neural network architectures by training each for a single epoch on seven computer vision benchmarks. It measures Top-1 accuracy to assess architectural quality, while also tracking generation efficiency via hash validation speed and duplicate rejection rates. Use when the user wants to benchmark on MNIST, CelebA-Gender, CIFAR-10, CIFAR-100, ImageNette, SVHN, Places365, or asks about evaluating this task. Reports Top-1 accuracy after 1...

Probes the ability of vision-language models to distinguish in-distribution from out-of-distribution samples under semantic, covariate, and real-world distribution shifts. It evaluates both zero-shot and few-shot prompt learning approaches across multiple benchmarks to assess robustness and ranking consistency. Use when the user wants to benchmark on ImageNet-X, ImageNet-FS-X, Wilds-FS-X, or asks about evaluating this task. Reports AUROC.

Evaluates Large Vision-Language Models on their ability to detect, localize, and ground objects in images across diverse and challenging scenarios, including in-domain dense detection, out-of-domain real-world settings, and generalization to unseen categories or scenes. Use when the user wants to benchmark on MSCOCO Val2017, ODINW-13, or asks about evaluating this task. Reports mAP.

Evaluates multimodal coding agents' ability to generate, deploy, and verify visual website development tasks across three hierarchical levels: static webpage generation, interactive frontend development, and full-stack website construction. It probes long-horizon planning, cross-modal reasoning, multi-page coordination, and visual/functional fidelity. Use when the user wants to benchmark on Vision2Web, or asks about evaluating this task. Reports Visual Score (VS), Functional Score (FS).

Evaluates vision-language models' ability to follow complex, real-world instructions on images. It probes open-ended generation, context-sensitive reasoning, and instruction-conditioned captioning by measuring how well model outputs align with human preferences and high-quality references. Use when the user wants to benchmark on VisIT-Bench, or asks about evaluating this task. Reports Elo rating, Win rate vs. reference.

Evaluates the intuitive number sense of Multimodal Large Language Models (MLLMs) by testing their ability to estimate and reason about seven visual numerical attributes (angle, scale, length, quantity, depth, area, volume) across four estimation tasks (range estimation, value estimation, value comparison, multiplicative estimation). Use when the user wants to benchmark on VisNumBench, or asks about evaluating this task. Reports Accuracy (%).

This benchmark probes a model's ability to accurately perceive basic geometric information—such as shape, angle, length, area, and intersections—in scientific figures and diagrams. It isolates visual perception from higher-level reasoning or domain knowledge by using direct, low-reasoning questions on synthetic and real-world images. Use when the user wants to benchmark on VisOnlyQA, or asks about evaluating this task. Reports accuracy.

Evaluates whether text-to-image models correctly render spatial relationships between objects mentioned in a prompt. It disentangles object generation accuracy from spatial correctness to reveal model biases like object priority and merging. Use when the user has predictions and gold and needs to compute VISOR.

Evaluates the ability of coding agents to generate executable visualization code across multiple programming languages and chart families, including iterative self-debugging capabilities. It probes both initial code generation fidelity and the model's capacity to recover from execution errors using feedback logs. Use when the user wants to benchmark on VisPlotBench, or asks about evaluating this task. Reports Execution Pass Rate.

Evaluates a model's ability to estimate the number of HTTP/3 responses in encrypted QUIC traffic using only observable packet characteristics. It probes the capability to extract meaningful temporal and structural patterns from encrypted flows without plaintext inspection. Use when the user wants to benchmark on VisQUIC, or asks about evaluating this task. Reports CAP±k.

This benchmark evaluates the visual reasoning capabilities of vision-language models across a perceptual-to-reasoning continuum. It isolates three levels of difficulty: basic perceptual grounding under transformations, single-attribute reasoning (color, count, orientation), and multi-attribute compositional reasoning. The setup tests whether models rely on genuine visual abstraction or fall back to linguistic priors when faced with naturalistic perturbations and rule-based inference. Use when...

This benchmark evaluates a model's ability to generate concise, structured summaries of scientific conference talks from video inputs. It specifically probes informativeness, alignment with visual/audio content, and factual consistency against the corresponding paper abstracts. Use when the user wants to benchmark on VISTA, or asks about evaluating this task. Reports ROUGE-1 F1.

Evaluates cross-modal vision-text alignment in Multimodal Large Language Models (MLLMs) across high-level semantic VQA, general multimodal understanding, and fine-grained visual perception/retrieval tasks. Use when the user wants to benchmark on VQAv2, OK-VQA, GQA, TextVQA, RealWorldQA, DocVQA, MMBench, SEED, AI2D, MMMU, MMStar, MME, RefCOCO, RefCOCO+, RefCOCOg, or asks about evaluating this task. Reports performance.

Evaluates conversational factuality and hallucination detection in LLMs by decomposing dialogue turns into atomic claims, verifying them against reference texts and dialogue history, and categorizing unverifiable content. It measures how well models track factual consistency across sequential turns. Use when the user wants to benchmark on FaithDial, or asks about evaluating this task. Reports claim-level accuracy.

Evaluates language models' zero-shot visual and textual commonsense reasoning without relying on ground-truth images. It probes the model's ability to infer object properties (color, shape, size) and answer general knowledge questions by internally generating and fusing multiple image variations from text prompts. Use when the user wants to benchmark on ImageNetVC, Object Commonsense (Memory Color, Color Terms, ViComTe, Size), Commonsense Reasoning (PIQA, SIQA, HellaSwag, WinoGrande, ARC, Ope...

Evaluates multi-modal large language models' ability to perform chain-of-thought reasoning with dynamic visual focusing on specific image regions. It probes localized visual understanding, intermediate bounding box prediction, and multi-turn reasoning across document, chart, general VQA, relation reasoning, and fine-grained domains. Use when the user wants to benchmark on Visual CoT Benchmark, or asks about evaluating this task. Reports accuracy.

Evaluates how vision-language models resolve conflicts between visual input and language priors by reasoning about altered visual attributes (color and size). It probes whether models rely on visual evidence or textual priors when they contradict. Use when the user wants to benchmark on Visual-Counterfact, or asks about evaluating this task. Reports MAC.

Evaluates fine-grained scene graph generation by predicting subject-predicate-object triplets from images. It measures recall and F1 scores across head, body, and tail predicate classes to assess performance on long-tailed distributions and missing annotations. Use when the user wants to benchmark on Visual Genome, or asks about evaluating this task. Reports mR@K, F@K.

Evaluates a model's ability to extract entity spans and link them to key-value pairs from complex, real-world document images. It probes joint vision-language understanding, handling poor image quality, occlusion, and multi-lingual text without relying on external OCR pipelines. Use when the user wants to benchmark on FUNSD, XFUND, CORD, SIBR, or asks about evaluating this task. Reports F1-score.

Evaluates multimodal large language models' ability to comprehend and reason about visual prompts (points, bounding boxes, free-form shapes) for fine-grained object classification, region captioning, OCR, and complex visual reasoning. Use when the user wants to benchmark on LVIS, PACO, COCO-Text, RefCOCOg, MDVP-Bench, LLaVA-Bench, Ferret-Bench, or asks about evaluating this task. Reports Accuracy.

Evaluates the perceptual visual quality of interpolated frames generated by optical flow methods against human judgments. It measures how well traditional objective metrics like RMSE correlate with crowdsourced subjective quality ratings across multiple video sequences. Use when the user wants to benchmark on Middlebury, or asks about evaluating this task. Reports SROCC.

Evaluates vision-language models on complex visual reasoning tasks including arithmetic counting, structural perception, and spatial transformations. It specifically probes the model's ability to generalize under domain shifts and adapt to distribution changes with limited data. Use when the user wants to benchmark on CLEVR-Math, Super-CLEVR, Geo170K/Math360K/Geometry3K, TRANCE, or asks about evaluating this task. Reports accuracy-rate (Acc).

Probes a model's ability to identify and classify interactions between pairs of objects in an image (subject-predicate-object triples). It focuses on capturing relational semantics beyond isolated object detection. Use when the user wants to benchmark on Visual Relationship Dataset, Visual Genome, or asks about evaluating this task. Reports recall@50.

Evaluates the stability and generalization of PPO policies trained with mode-dependent layers (BatchNorm, dropout) across visual reinforcement learning environments. It probes whether a deterministic rectification phase prevents reward collapse and aligns training-evaluation dynamics compared to standard training modes. Use when the user wants to benchmark on Procgen, Histopathology Patch-Localization, Natural Image Patch-Localization, or asks about evaluating this task. Reports normalized re...

Evaluates a model's ability to assign a semantic class label to every pixel in an image, capturing fine-grained scene understanding. It measures pixel-level classification accuracy and boundary alignment across diverse outdoor and indoor environments. Use when the user wants to benchmark on Pascal Context, Sift Flow, COCO Stuff, or asks about evaluating this task. Reports GPA.

This benchmark evaluates visual language models' ability to understand and reason about spatial relationships between objects in images. It specifically probes orientation-dependent relations, frame-of-reference shifts (intrinsic vs. relative), and zero-shot generalization to unseen object concepts. Use when the user wants to benchmark on VSR, or asks about evaluating this task. Reports accuracy.

This evaluation probes how Vision-Language Models ground their responses in visual input versus relying on language priors or user bias. It measures perceptual awareness, visual dependency, and alignment conflicts by comparing model behavior across original, blank, noisy, and semantically conflicting images. Use when the user wants to benchmark on GQA, VQAv2, A-OKVQA, POPE, or asks about evaluating this task. Reports VNS.

Probes multimodal visual reasoning capabilities over complex, LaTeX-rendered table images. It specifically tests multi-step inference, structural layout understanding, and the ability to extract and reason over tabular data from visual inputs rather than raw text. Use when the user wants to benchmark on Visual-TableQA, or asks about evaluating this task. Reports Relaxed Accuracy.

Evaluates multimodal large language models' ability to perform precise spatial reasoning and visual text grounding in document images. It tests whether models can generate accurate bounding boxes that support their textual answers, both from scratch (OCR-free) and when provided with OCR text (OCR-based), while also measuring their instruction-following capability. Use when the user wants to benchmark on ChartQA, DocVQA, InfographicsVQA, TRINS, or asks about evaluating this task. Reports IoU.