Claude Skills by qhjqhj00

Evaluates how well coordinate-based MLPs with different input feature mappings (none, basic, positional encoding, Gaussian random Fourier features) can learn high-frequency functions across various low-dimensional regression tasks in computer vision and graphics. Use when the user wants to benchmark on Natural images, Text images, 3D shape, Shepp-Logan phantoms, ATLAS dataset, NeRF ATLAS scene, or asks about evaluating this task. Reports PSNR, IoU.

Compute the fowlkes_mallows_score metric — provided by scikit-learn. Use when the user has predictions and ground-truth and needs to compute fowlkes_mallows_score, or asks how to score with fowlkes_mallows_score.

Compute the FowlkesMallowsIndex metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute FowlkesMallowsIndex, or asks how to score with FowlkesMallowsIndex.

Evaluates offline-to-online finetuning of model-based reinforcement learning world models on continuous control and visuomotor tasks. It probes the model's ability to adapt to seen and unseen task variations with limited online interactions while mitigating extrapolation errors via uncertainty regularization. Use when the user wants to benchmark on D4RL, xArm, Quadruped Locomotion, Real xArm, or asks about evaluating this task. Reports Success rate (%).

Evaluates the end-to-end inference and training latency, power consumption, and energy efficiency of a tensorized neural network hardware accelerator on an FPGA platform, comparing against baseline and prior FPGA accelerators. Use when the user has predictions and gold and needs to compute Latency (ms), Energy Efficiency (GOPS/W).

Evaluates the end-to-end system performance of an FPGA-accelerated machine learning inference service, specifically measuring inference latency and throughput under varying network conditions and concurrent workloads. Use when the user has predictions and gold and needs to compute round-trip inference latency.

Evaluates trade-offs between inference latency and hardware resource utilization when deploying variational autoencoders on FPGAs using different synthesis frameworks (SNL vs. hls4ml) and quantization levels. Use when the user has predictions and gold and needs to compute Latency.

Evaluates the downstream transfer capability of fractal-based pre-trained visual representations on fine-grained image classification and medical image segmentation tasks. It measures how effectively synthetic Iterated Function System (IFS) pre-training captures transferable features compared to training from scratch or using ImageNet/FractalDB pre-training. Use when the user wants to benchmark on CUB-2011, Stanford Cars, Stanford Dogs, FGVC Aircraft, CIFAR-100, GlaS, or asks about evaluating...

Evaluates the effectiveness of pre-training convolutional neural networks on automatically generated fractal image datasets (FractalDB) compared to natural image pre-training and self-supervised learning, measuring downstream classification accuracy on standard benchmarks. Use when the user wants to benchmark on CIFAR-10, CIFAR-100, ImageNet-100, Places-30, ImageNet-1k, Places-365, Pascal VOC 2012, Omniglot, or asks about evaluating this task. Reports classification accuracy.

Evaluates the sky localization precision and detection sensitivity of gravitational-wave detector networks for multi-messenger follow-up of compact binary mergers. It quantifies how well a network can identify and pinpoint sources within a specific distance and localization area threshold. Use when the user has predictions and gold and needs to compute fractional follow-up metric.

Evaluates bone fracture detection and localization in pelvic X-ray images using point-based annotations. It measures image-level classification accuracy and pixel-wise localization precision under clinically relevant false positive rates. Use when the user wants to benchmark on PXR Trauma Registry Dataset, or asks about evaluating this task. Reports AU-ROC, FROC.

Evaluates a model's ability to detect abnormal human activities by predicting multi-timescale future and past pose trajectories. The framework measures prediction errors across different temporal granularities and combines them to identify anomalous frames. Use when the user wants to benchmark on HR-ShanghaiTech, HR-Avenue, Corridor, or asks about evaluating this task. Reports Frame-AUC.

Evaluates the computational efficiency and hardware utilization of five deep learning frameworks (Caffe, Neon, TensorFlow, Theano, Torch) across standard neural network architectures on CPU and GPU hardware. Use when the user wants to benchmark on MNIST, ImageNet, IMDB, or asks about evaluating this task. Reports forward pass time (ms).

Evaluates the training throughput and execution efficiency of deep learning frameworks by measuring how quickly they process standard model architectures on a single GPU. It compares PyTorch against TensorFlow, MXNet, CNTK, Chainer, and PaddlePaddle to assess device utilization and runtime optimization. Use when the user has predictions and gold and needs to compute Throughput.

Evaluates the cross-lingual adaptation and zero-shot/few-shot transfer capabilities of decoder-only LLMs across reading comprehension, topic classification, machine translation, mathematical reasoning, and summarization tasks in Southeast Asian, African, and Indic languages. Use when the user wants to benchmark on BeleBele, Sib-200, Flores-200, GSM8K-NTL, IndicGenBench, or asks about evaluating this task. Reports Accuracy, ChrF++.

This benchmark evaluates the robustness of fraud detection models to label noise in training data. It measures how effectively various noise-removal techniques preserve predictive performance when tested on clean, unseen data. The protocol specifically probes a model's ability to mitigate artificially injected label corruption across multiple real-world fraud datasets. Use when the user wants to benchmark on Fraud Dataset Benchmark (FDB), or asks about evaluating this task. Reports ROC-AUC.

Evaluates a graph neural network classifier's ability to determine whether credit card transactions require customer contact, aiming to optimize fraud detection workflows and reduce false positives. Use when the user wants to benchmark on Custom credit card transaction dataset, or asks about evaluating this task. Reports accuracy.

This evaluation probes a machine learning model's ability to accurately detect fraudulent financial transactions in highly imbalanced tabular data, while also measuring the system-level overhead and economic viability of integrating blockchain-based audit trails. It tests both detection accuracy on real-world and synthetic datasets and the practical throughput/latency constraints of on-chain verification workflows. Use when the user wants to benchmark on Kaggle Credit Card Fraud, Enterprise P...

This benchmark evaluates large language models' robustness against multi-round fraud and phishing inducements. It probes whether models can successfully identify and defend against deceptive prompts across five fraud categories under both standard helpful-assistant and role-play settings, while also measuring cross-lingual performance gaps. Use when the user wants to benchmark on Fraud-R1, or asks about evaluating this task. Reports Defense Success Rate (DSR).

Evaluates a model's ability to detect fraudulent reviewer groups by analyzing spatio-temporal co-review patterns. It probes the model's capacity to distinguish genuine groups from coordinated fraudster groups using graph representation learning and temporal modeling. Use when the user wants to benchmark on Yelp, Amazon, or asks about evaluating this task. Reports F1-value.

Evaluates the focal ratio degradation (FRD) of multi-mode optical fibres under automated testing conditions to verify compliance with astronomical instrumentation specifications. It compares automated optical bench measurements against manual ring tests to ensure measurement consistency and accuracy. Use when the user has predictions and gold and needs to compute FRD (Focal Ratio Degradation).

Measures the distance between the feature distributions of real and generated images using a pretrained Inception network. It evaluates both the fidelity and diversity of generated samples by comparing their mean and covariance in the feature space. Use when the user has predictions and gold and needs to compute FID.

Evaluates the quality and diversity of synthesized human motion by measuring the distributional distance between ground truth and synthetic motion sequences in a learned latent space. Use when the user has predictions and gold and needs to compute Fréchet Motion Distance (FMD).

Evaluates few-shot document-level relation extraction by testing a model's ability to identify relations between entity pairs across documents using limited support examples. It specifically probes domain adaptation capabilities, handling of class imbalance, and robustness to NOTA (none-of-the-above) distributions in realistic document-level settings. Use when the user wants to benchmark on FREDo, or asks about evaluating this task. Reports macro F1.

Evaluates test-time self-supervised adaptation for feed-forward 3D reconstruction models. It probes the model's ability to refine camera pose estimation and 3D geometry reconstruction on unseen scenes by enforcing cross-view feature consistency without ground-truth labels. Use when the user wants to benchmark on ETH3D, ScanNet++, 7-Scenes, HiROOM, or asks about evaluating this task. Reports AUC@3, F1-score.

Evaluates bilingual French-English language understanding, cultural knowledge, and generation capabilities of LLMs across classification and open-ended tasks. Probes the model's ability to perform few-shot reasoning, factual recall, and text generation in both languages. Use when the user wants to benchmark on FrenchBench, English Benchmarks, or asks about evaluating this task. Reports accuracy.

Evaluates a sequential recommendation model's ability to predict the next item in a user's interaction history by jointly modeling intra-session and inter-session behavioral dynamics. It probes the model's recommendation accuracy, robustness to noisy cross-domain data, and stability under sparse interaction conditions. Use when the user wants to benchmark on Amazon Beauty, Sports & Outdoors, Toys & Games, or asks about evaluating this task. Reports HR@K, NDCG@K.

This benchmark evaluates models' ability to recover latent demand during stockout periods in perishable retail. It probes whether algorithms can disentangle true consumption patterns from supply-induced censoring using hourly temporal data and contextual covariates. Success is measured by prediction accuracy, bias mitigation, and the decoupling of recovered demand from stockout ratios. Use when the user wants to benchmark on FreshRetailNet-50K, or asks about evaluating this task. Reports WAPE.

Evaluates the ability of LLMs to generate comprehensive, well-organized, and verifiable Wikipedia-like articles from a given topic. It probes outline planning, factual coverage, structural coherence, and source grounding. Use when the user wants to benchmark on FreshWiki, or asks about evaluating this task. Reports ROUGE-1.

Evaluates the quality of lightweight, non-semantic speech embeddings by training simple downstream classifiers on averaged embedding features to perform audio classification tasks. It also measures inference latency on a mobile device to assess real-time suitability for on-device deployment. Use when the user wants to benchmark on NOSS benchmark, ESC-50 (human sounds subset), Mask speech dataset, or asks about evaluating this task. Reports test accuracy.

Compute Fritz02/execution_accuracy via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of Fritz02/execution_accuracy.

Evaluates an end-to-end agentic framework for small-molecule drug design across eight benchmarks spanning the full discovery pipeline, from target identification and knowledge retrieval to virtual screening, interaction profiling, de novo design, and retrosynthetic planning. Use when the user wants to benchmark on Humanity’s Last Exam (HLE), UniProt, Open Targets Platform, ADMETLab 3.0, DAVIS, PLIP, CrossDocked, USPTO-50k, PaRoute, or asks about evaluating this task. Reports score.

Probes a model's ability to generate and iteratively refine front-end code through multi-turn conversational instructions, handling both textual and visual feedback. It specifically measures functional correctness, user experience design quality, and the model's tendency to overwrite prior implementations in long-context interactions. Use when the user wants to benchmark on FronTalk, or asks about evaluating this task. Reports pass rate (PR), usability (UX).

Evaluates advanced mathematical reasoning and experimental problem-solving. It tests whether models can iteratively write and execute Python code to verify hypotheses, refine strategies, and derive correct solutions to expert-level, unsolved math problems. Use when the user wants to benchmark on FrontierMath, or asks about evaluating this task. Reports accuracy.

Few-shot molecular property prediction and regression on a large-scale benchmark with thousands of tasks. Probes generalization across diverse protein targets and varying support set sizes. Use when the user wants to benchmark on FS-Mol, or asks about evaluating this task. Reports ΔAUPRC.

Compute fschlatt/ner_eval via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of fschlatt/ner_eval.

Probes a model's ability to learn new classes incrementally in a few-shot setting while retaining knowledge of previously learned classes, measuring resistance to catastrophic forgetting. Use when the user wants to benchmark on miniImageNet, CIFAR-100, CUB-200, or asks about evaluating this task. Reports Average accuracy.

This benchmark evaluates few-shot generalization capability by measuring classification accuracy in an episodic setting where models must recognize novel classes using only a few labeled support examples. It probes the model's ability to adapt quickly to new categories and its robustness to test-time data augmentation. Use when the user wants to benchmark on miniImagenet, tieredImagenet, CUB, Animals, or asks about evaluating this task. Reports accuracy.

Evaluates the performance and data efficiency of a Vision-Language-Action (VLA) model trained on a synthetically distilled coreset compared to models trained on full datasets. It probes long-horizon manipulation, multi-task skill acquisition, and generalization across spatial, object, goal, and temporal dimensions. Use when the user wants to benchmark on CALVIN, Meta-World, LIBERO, or asks about evaluating this task. Reports Success Rate (SR %), Average Task Completion Length (Avg. Len).

Evaluates automatic speech recognition (ASR) systems on spontaneous, formal parliamentary speech in Danish. It tests both in-domain recognition accuracy and cross-domain transferability between the new FT Speech corpus and the established SBRead corpus. Use when the user wants to benchmark on FT Speech, SBRead, or asks about evaluating this task. Reports WER.

Evaluates whether post-hoc ensembling strategies (e.g., greedy selection, top-N, model averaging) improve classification accuracy and uncertainty calibration over single fine-tuned language models. It probes the robustness of combining multiple finetuned classifiers across varying training data sizes (10% vs 100%). Use when the user wants to benchmark on DBpedia, News, SetFit, SST-2, Tweet, IMDB, or asks about evaluating this task. Reports classification error.

Evaluates the ability of tabular and time-series regression models to predict ship fuel consumption using operational, environmental, and temporal features. It probes how well models leverage in-context learning, weather covariates, and sequential patterns across different vessel types. Use when the user wants to benchmark on FuelCast, or asks about evaluating this task. Reports MAE.

Evaluates multimodal late-fusion models for predicting scenic visibility (clear, cloudy, perfect, obscured) across short- to medium-term forecasting horizons (+0d to +3d). It probes the model's ability to integrate visual webcam features with meteorological forecasts to handle class imbalance and temporal dynamics in environmental perception. Use when the user wants to benchmark on FujiView, or asks about evaluating this task. Reports accuracy (ACC).

Evaluates real-time interactive behaviors in full-duplex spoken dialogue models. It specifically probes turn-taking, pause handling, backchanneling, and interruption management capabilities without relying on human studies. Use when the user wants to benchmark on Full-Duplex-Bench, or asks about evaluating this task. Reports descriptive metrics.

Evaluates a large audio language model's capabilities across spoken question answering, audio understanding, speech recognition, function calling, and instruction following. It probes the model's ability to process speech inputs, generate text/speech outputs, and adhere to complex voice instructions while maintaining speech quality and safety. Use when the user wants to benchmark on VoiceBench, OpenAudioBench, UltraEval-Audio, MMAU, MMAU-Pro, MMSU, Librispeech, Common Voice, Speech-ACEBench, ...

Evaluates an LLM's ability to correctly identify, retrieve, and invoke external APIs or functions based on a user query. It probes zero-shot and multi-turn function-calling capabilities, including handling live vs. non-live APIs, detecting irrelevant queries, and mitigating hallucinations. Use when the user wants to benchmark on BFCL-v3, API-Bank, or asks about evaluating this task. Reports AST.

Evaluates end-to-end form understanding on noisy scanned documents, covering text detection, optical character recognition, word grouping, semantic entity labeling, and entity linking. Use when the user wants to benchmark on FUNSD, or asks about evaluating this task. Reports F1-score.

Evaluates a robot policy's ability to ground heterogeneous sensor modalities (vision, touch, sound) into language instructions for zero-shot task execution in partially observable environments. It probes multimodal prompting, compositional reasoning, and the necessity of auxiliary contrastive and language grounding losses. Use when the user wants to benchmark on WidowX Multimodal Teleoperation Dataset, or asks about evaluating this task. Reports task success.

Evaluates audio source separation models on mixed reverberant or dry recordings containing 1–4 active sources. It measures reconstruction fidelity and source-counting accuracy across varying mixture complexities. Use when the user wants to benchmark on FUSS, or asks about evaluating this task. Reports SI-SNR.

This benchmark evaluates multilingual Retrieval-Augmented Generation (RAG) systems across three tasks: monolingual knowledge extraction, cross-lingual knowledge transfer, and multilingual knowledge selection. It probes a model's ability to retrieve and generate answers in eight languages, assess cross-lingual transfer capabilities, and measure selection bias when presented with conflicting answers across languages. Use when the user wants to benchmark on Futurepedia, or asks about evaluating ...