Claude Skills by qhjqhj00

Evaluates tabular machine learning models in data lake environments by leveraging auxiliary tables to improve prediction on a target table. It probes two integration paradigms: table unionability (vertical concatenation to increase training samples) and table joinability (horizontal enrichment to add features). Use when the user wants to benchmark on LakeMLB, or asks about evaluating this task. Reports predictive performance.

Evaluates the morphosyntactic well-formedness and grammaticality of generated natural language text. It measures how closely a generated sentence or corpus adheres to language-specific dependency rules extracted from treebanks. Use when the user has predictions and gold and needs to compute L'AMBRE.

Evaluates open-vocabulary multi-object tracking by requiring models to follow multiple targets in video sequences guided by natural language descriptions. It probes the model's ability to jointly perform text-grounded detection and long-term identity association across diverse, real-world scenarios. Use when the user wants to benchmark on LaMOT, or asks about evaluating this task. Reports HOTA.

This benchmark evaluates a model's ability to generate long-form, personalized question-answering responses by aligning outputs with fine-grained, user-specific information needs extracted from community Q&A narratives. It probes aspect-based response quality rather than binary correctness, measuring how well generated answers address individual criteria tailored to a specific user's profile. Use when the user wants to benchmark on LaMP-QA, or asks about evaluating this task. Reports aspect-b...

Evaluates a multimodal machine learning workflow's ability to predict 3D subsurface geological, hydrogeological, and geophysical features from sparse, heterogeneous field data. It tests the model's generalization capability using transductive learning and mutual information maximization across five cross-validation splits. Use when the user wants to benchmark on Lana'i 3D Subsurface Grid, or asks about evaluating this task. Reports R-squared.

Evaluates land cover segmentation models on Sentinel-2 imagery using sparse annotations to predict fuel maps. It tests the model's ability to generalize across European regions affected by wildfires and compare against dense ground truth datasets (LUCAS, Urban Atlas). Use when the user wants to benchmark on Sentinel-2 Land Cover Dataset, or asks about evaluating this task. Reports F1 score.

Evaluates semantic segmentation models on high-resolution aerial imagery for mapping four land cover classes: buildings, woodlands, water, and roads. It probes the model's ability to accurately segment fine-grained, small, and narrow objects in rural environments from RGB imagery. Use when the user wants to benchmark on LandCover.ai, or asks about evaluating this task. Reports mIoU.

Evaluates a transformer variant's ability to retrieve relevant past context blocks and maintain language modeling performance over extended sequence lengths. It probes long-range dependency retention and random-access memory retrieval capabilities compared to standard and recurrent transformer baselines. Use when the user wants to benchmark on PG-19, arXiv math papers, RedPajama (subset), or asks about evaluating this task. Reports perplexity.

Evaluates the internal reasoning dynamics of LLMs on multi-choice tasks by tracking intermediate thought states and measuring convergence behavior. It quantifies consistency, uncertainty, and perplexity across different model scales, reasoning tasks, and decoding methods to visualize how reasoning trajectories evolve toward correct or incorrect answers. Use when the user wants to benchmark on AQuA, MMLU, StrategyQA, CommonSenseQA, or asks about evaluating this task. Reports reasoning accuracy.

Evaluates models' ability to predict lane-level traffic speed and flow by modeling spatio-temporal dependencies on graph-structured lane networks. It tests performance across both regular and irregular lane configurations, emphasizing both predictive accuracy and training efficiency. Use when the user wants to benchmark on PeMS, PeMSF, HuaNan, or asks about evaluating this task. Reports MAE.

Compute langdonholmes/cohen_weighted_kappa via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of langdonholmes/cohen_weighted_kappa.

Evaluates language identification models on noisy web crawl data to measure their ability to accurately filter in-language sentences for low-resource languages. It probes domain mismatch and class imbalance effects on real-world LangID deployment. Use when the user wants to benchmark on Web Crawl & Held-out Eval Set, or asks about evaluating this task. Reports precision.

This metric probes an LLM's internal representation quality and cross-lingual alignment by measuring how closely the embedding space of a target language clusters around an English baseline. It quantifies multilingual capability and pre-training data imbalance by computing similarity scores across specific transformer layers. Use when the user has predictions and gold and needs to compute Language Ranker.

Evaluates multilingual language transfer by measuring how well models adapt to German and Bulgarian while preserving source language (English) capabilities. Probes catastrophic forgetting and cross-lingual generalization across reasoning, math, reading comprehension, and commonsense tasks. Use when the user wants to benchmark on Multilingual Language Transfer Benchmarks (EN/DE/BG), or asks about evaluating this task. Reports normalized accuracy.

Evaluates large language models on their proficiency in Lao, a low-resource Southeast Asian language. It probes factual knowledge, K12 curriculum alignment, culturally grounded reasoning, bilingual translation fidelity, and open-ended generation quality through multiple-choice, translation, and pairwise arena tasks. Use when the user wants to benchmark on LaoBench, or asks about evaluating this task. Reports Accuracy.

Evaluates an LLM's ability to perform legal argument reasoning by predicting the correct continuation of a court's argument chain. Given case facts and preceding arguments, the model must select the most plausible next argument from multiple options, testing its understanding of legal logic and precedent application. Use when the user wants to benchmark on LAR-ECHR, or asks about evaluating this task. Reports accuracy.

Evaluates how well vision models and latent action representations capture semantic action categories and map visual features to low-level robotic control trajectories. It probes both high-level action understanding and physical grounding for generalizable vision-to-action alignment across diverse robotic and human motion datasets. Use when the user wants to benchmark on VLABench, CALVIN, RoboCOIN, AgiBotWorld-Beta, or asks about evaluating this task. Reports Top-1 Accuracy.

This protocol evaluates the cross-lingual safety alignment of LLMs by measuring how frequently they comply with jailbreak prompts across multiple languages and resource levels. It simultaneously verifies that safety alignment does not degrade general capabilities such as multilingual knowledge, reasoning, and instruction following. Use when the user wants to benchmark on MultiJail, HarmBench (translated), M-MMLU, MT-Bench, MGSM, or asks about evaluating this task. Reports Attack Success Rate ...

Evaluates deep learning models on video-based laparoscopic surgical training tasks. It probes the model's ability to recognize task-specific procedural errors and predict structured global skill ratings from synchronized stereo video streams. Use when the user wants to benchmark on LASANA, or asks about evaluating this task. Reports error_recognition.

Probes the ability to extract aspect-based sentiment quadruples (target, aspect, opinion, sentiment) from text in low-resource agglutinative languages. It evaluates exact-match performance across entity detection, relation linking, and full quadruple composition. Use when the user wants to benchmark on LASQ, or asks about evaluating this task. Reports F1.

Probes the resource-latency trade-off between FPGA programmable logic (hls4ml) and AMD AI Engines for dense neural network layers. It quantifies the minimum PL hardware resources required to match AIE inference latency, identifying architectural crossover points for different layer shapes and reuse factors. Use when the user has predictions and gold and needs to compute LARE (Latency-Adjusted Resource Equivalence).

Evaluates the real-time performance and observability accuracy of an eBPF-based tracing library by measuring request throughput and tail latency under inference workloads. It verifies the framework's ability to disambiguate request boundaries from streaming system calls without application instrumentation. Use when the user has predictions and gold and needs to compute latency statistics.

Measures the inference latency of binarized, 8-bit, and 32-bit convolutional layers on edge devices to evaluate the efficiency and speedup of the Larq Compute Engine framework compared to standard implementations. Use when the user has predictions and gold and needs to compute latency.

Evaluates a language model's reasoning, coding, and general knowledge capabilities using a suite of standard academic benchmarks. It specifically probes how test-time compute scaling (via recurrent depth) impacts performance across mathematical, coding, and commonsense reasoning tasks. Use when the user wants to benchmark on GSM8K, MATH (Minerva), MathQA, MBPP, HumanEval, ARC-E, ARC-C, HellaSwag, MMLU, OBQA, PiQA, SciQ, WinoGrande, or asks about evaluating this task. Reports flexible extract ...

Evaluates a model's ability to detect unanswerable Text-to-SQL queries by analyzing intermediate hidden activations, aiming to prevent hallucinated SQL generation and unsafe execution. It probes whether the system can reliably distinguish between answerable and unanswerable prompts across diverse domains and linguistic ambiguities. Use when the user wants to benchmark on TriageSQL, AMBROSIA, SQuAD 2.0, MD-Enterprise, or asks about evaluating this task. Reports F1.

Evaluates Latvian-specific encoder models on lightweight diagnostic tasks, morphosyntactic parsing, and semantic representation quality to benchmark low-resource language modeling capabilities. Use when the user wants to benchmark on EuroEval Latvian diagnostics, COPA (Latvian), Universal Dependencies Latvian treebank (UD v2.16), Latvian WSD dataset, or asks about evaluating this task. Reports MCC.

This benchmark evaluates the capability of models to detect and temporally localize content-driven audio-visual forgeries in long videos. It probes multimodal boundary matching and temporal manipulation detection by requiring models to identify fake segments and predict their precise start and end timestamps. Use when the user wants to benchmark on LAV-DF, or asks about evaluating this task. Reports AP@0.5.

Evaluates a model's ability to decompose raster graphic designs into a sequence of re-editable layers. It measures visual reconstruction quality and the number of edits required to match a ground-truth layer structure, accounting for the ill-posed nature of layer ordering. Use when the user wants to benchmark on Crello, or asks about evaluating this task. Reports RGB L1, Alpha IoU.

Evaluates optical flow estimation on non-Lambertian surfaces (transparent, reflective, diffuse) and multi-layer scenes. It probes a model's ability to predict flow through transparent occluders and handle complex material properties without relying on test-time optimizations. Use when the user wants to benchmark on LayeredFlow, or asks about evaluating this task. Reports EPE.

Evaluates a model's ability to generate high-fidelity images conditioned on spatial layouts and text descriptions, measuring both perceptual quality and precise object-level spatial alignment. Use when the user wants to benchmark on COCO-3K, HiCo-7K, or asks about evaluating this task. Reports FID.

Evaluates layout-guided image generation models on their ability to follow spatial control instructions (number, position, size, shape) across in-distribution and out-of-distribution layouts. Probes generalization to arbitrary object configurations and fine-grained spatial reasoning. Use when the user wants to benchmark on CLEVR, LayoutBench, or asks about evaluating this task. Reports AP (AP50).

Evaluates the inference latency, throughput, and SLA compliance of a dynamic batching system under varying request arrival rates and diverse DNN workloads. Use when the user wants to benchmark on ResNet, GNMT, Transformer, VGGNet, MobileNet, LAS, BERT, or asks about evaluating this task. Reports SLA violation rate.

Evaluates the closed-loop driving performance and generalization of end-to-end autonomous driving policies in simulation and on real-world datasets. It probes the model's ability to navigate long-horizon routes, handle diverse weather and lighting conditions, and transfer synthetic pre-training to real-world driving scenarios without violating traffic rules. Use when the user wants to benchmark on CARLA Town13, Bench2Drive, Longest6 v2, NAVSIM v1, NAVSIM v2, WOD-E2E, or asks about evaluating ...

Evaluates a model's ability to verify whether a candidate (Task, Dataset, Metric) triple is actually used or mentioned in a specific AI research paper. The task is framed as a natural language inference problem where the model must distinguish between valid triples and randomly sampled invalid ones. Use when the user wants to benchmark on AI Research Paper Collection, or asks about evaluating this task. Reports micro-F1.

Evaluates whether pre-trained Recognizing Textual Entailment (RTE) models can generalize to unseen task-dataset-metric (TDM) extraction pairs in a zero-shot setting. It probes whether models learn genuine semantic entailment or merely memorize training distribution patterns. Use when the user wants to benchmark on LEADERBOARDS, or asks about evaluating this task. Reports macro F1.

Evaluates federated learning algorithms under realistic constraints including device-level data skew, heterogeneous data distributions, and communication bottlenecks. It measures model accuracy after federated training across multiple simulated devices. Use when the user wants to benchmark on Shakespeare, Sent140, FEMNIST, CelebA, Synthetic, Reddit, or asks about evaluating this task. Reports AccuracyTop1.

Evaluates vision and vision-language models on plant disease diagnosis, including fine-grained image classification, few-shot adaptation, and zero-shot visual question answering. It probes the models' ability to recognize subtle visual symptoms, reason over taxonomic pathogen information, and generalize across agricultural domains. Use when the user wants to benchmark on LeafNet, or asks about evaluating this task. Reports Accuracy.

Evaluates an LLM-based framework's ability to dynamically generate research leaderboards by measuring topic relevance, content quality (coverage, recency, structure), and generation speed compared to manual curation. Use when the user has predictions and gold and needs to compute Leaderboard Content Quality.

Evaluates a language model's ability to generate correct, step-by-step formal proof tactics for mathematical statements within the Lean 4 proof assistant. Use when the user wants to benchmark on miniF2F, or asks about evaluating this task. Reports solve_rate.

Evaluates the quality of answers generated by RAG systems across specialized domains. It probes the model's ability to retrieve relevant information, synthesize comprehensive responses, and maintain diversity and practical utility. Additionally, it measures retrieval efficiency and the impact of structural knowledge on generation. Use when the user wants to benchmark on UltraDomain, or asks about evaluating this task. Reports Comprehensiveness.

Evaluates a unified framework for multi-task domain adaptation few-shot learning across image classification, object detection, and video classification. It probes the model's ability to adapt to new domains and scale label budgets incrementally from 1-shot to full dataset size. Use when the user wants to benchmark on DomainNet, Office-Home, Office31, Pool and Car, xView, UCF101, or asks about evaluating this task. Reports accuracy.

Evaluates end-to-end learned image signal processing (ISP) pipelines that map mobile RAW sensor data to high-fidelity RGB images. It probes the trade-off between image reconstruction fidelity, subjective visual quality, and real-time inference efficiency on mobile hardware. Use when the user wants to benchmark on Fujifilm UltraISP dataset, or asks about evaluating this task. Reports PSNR.

Probes a model's ability to retrieve relevant Chinese criminal case documents from a large corpus based on legal queries. It specifically tests alignment with multi-dimensional legal relevance criteria, including case characterization, penalty matching, and procedural similarity. Use when the user wants to benchmark on LeCaRDv2, or asks about evaluating this task. Reports Recall@K.

Evaluates code generation and algorithmic reasoning capabilities on competitive programming problems. It specifically probes temporal robustness by testing on problems released after a strict cutoff date to detect data contamination, and measures performance across difficulty levels and algorithmic topics. Use when the user wants to benchmark on LeetCodeDataset, or asks about evaluating this task. Reports pass@1.

Evaluates the ability of state-space models (Mamba/SSD-Mamba) and transformers to perform statutory classification and case law retrieval on long-context legal documents. It probes how well models capture fine-grained semantic distinctions and maintain global coherence over thousands of tokens while balancing accuracy with computational throughput. Use when the user wants to benchmark on SCOTUS, ILDC, ECtHR, EUR-Lex, or asks about evaluating this task. Reports Accuracy.

Evaluates domain-adapted BERT models on legal text classification and named entity recognition to measure the impact of further pre-training and hyperparameter tuning strategies. Use when the user wants to benchmark on EURLEX57K, ECHR-CASES, CONTRACTS-NER, or asks about evaluating this task. Reports accuracy, F1.

This benchmark probes a model's ability to understand and predict precise legal terminology and procedural concepts in Turkish court documents. It evaluates both masked language modeling capabilities on legal cloze sentences and structural segmentation accuracy for parsing document sections. Use when the user wants to benchmark on Legal Cloze Test benchmark, v12 Court Decision Segmentation Dataset, or asks about evaluating this task. Reports Top-1 Accuracy.

Evaluates a fine-tuned open-source language model on its ability to perform keyword extraction, summarization, and sentiment analysis on the Indian Constitution. The protocol tests whether domain-specific fine-tuning improves the model's capacity to grasp nuanced legal semantics and structural elements. Use when the user wants to benchmark on Indian Constitution, or asks about evaluating this task. Reports precision, recall, and F1 score.

This benchmark evaluates information retrieval systems on Swiss legal rulings and legislation. It tests the ability to rank relevant legal documents against long, multilingual queries and corpora. Use when the user wants to benchmark on Legal Information Retrieval, or asks about evaluating this task. Reports NDCG.

This benchmark evaluates language models on identifying legal violations and associating victims in unstructured legal text. It probes two core capabilities: named entity recognition for specific causes of action and natural language inference for linking victims to legal claims across different legal domains. Use when the user wants to benchmark on LegalLens NER Dataset, LegalLens NLI Dataset, or asks about evaluating this task. Reports Macro F1.