All authors
Claude Skills by qhjqhj00
github.com/qhjqhj007,572 skills29 installs10,436 views
- Ml4cfd Competition EvalEvaluates machine learning surrogates for 2D airfoil aerodynamics on prediction accuracy, computational speed-up, physical consistency, and out-of-distribution generalization. The benchmark compares learned models against a standard CFD solver (OpenFOAM) across in-distribution and novel geometric configurations. Use when the user wants to benchmark on ML4CFD Competition Dataset, or asks about evaluating this task. Reports Global Score.Votes: 0GitHub stars: 3
- Mlaad Cross Dataset EvalEvaluates the cross-dataset generalization capability of voice anti-spoofing models. It probes whether models trained on one synthetic audio dataset can accurately detect deepfake or spoofed speech when tested on entirely different datasets, including those with only spoof samples or different languages. Use when the user wants to benchmark on ASVspoof19, ASVspoof21-DF, ASVspoof21-LA, FakeOrReal, InTheWild, MLAAD v1, Voc.v, WaveFake, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mldr EvalEvaluates retrieval over long multilingual documents (up to 8,192 tokens), testing a model's ability to capture information from extended contexts across multiple languages. Use when the user wants to benchmark on MLDR, or asks about evaluating this task. Reports nDCG@10.Votes: 0GitHub stars: 3
- Mle Bench Lite EvalEvaluates an agent's ability to iteratively refine and improve runnable solutions for competition-style ML tasks over a long horizon. It probes sustained experiment improvement and competitive performance rather than just initial submission validity. Use when the user wants to benchmark on MLE-Bench Lite, or asks about evaluating this task. Reports Any Medal%.Votes: 0GitHub stars: 3
- Mleb EvalThis benchmark evaluates the capability of embedding models to perform legal information retrieval across diverse jurisdictions, document types, and legal tasks. It probes how well models understand judicial reasoning, regulatory interpretation, and multinational contract analysis compared to general-purpose IR models. Use when the user wants to benchmark on MLEB, or asks about evaluating this task. Reports NDCG@10.Votes: 0GitHub stars: 3
- Mlgym EvalEvaluates LLM agents on open-ended AI research tasks across 13 diverse benchmarks. It measures the agent's ability to navigate codebases, run experiments, and improve model performance, assessing capabilities from reproducing existing research to achieving state-of-the-art results. Use when the user wants to benchmark on MLGym Benchmarks, or asks about evaluating this task. Reports AutoML-inspired optimization metric.Votes: 0GitHub stars: 3
- Mlissard EvalThis benchmark evaluates a model's ability to perform simple sequential reasoning tasks (e.g., counting, copying, list intersection) while extrapolating to longer input sequences. It specifically probes length generalization and the impact of multilingual in-context examples on reasoning robustness across different languages. Use when the user wants to benchmark on MLissard, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mllm Cl EvalEvaluates continual learning capabilities in multimodal large language models across two settings: Domain Continual Learning (DCL) for acquiring IID domain-specific knowledge (remote sensing, medical, autonomous driving, science, finance) and Ability Continual Learning (ACL) for non-IID skill acquisition (OCR, math & logic, visual perception, GUI agent). It measures the model's ability to learn new tasks sequentially while mitigating catastrophic forgetting of previously learned capabilities....Votes: 0GitHub stars: 3
- Mllm Hallucination EvalEvaluates the ability of multimodal large language models (MLLMs) to generate accurate image descriptions and answer questions without hallucinating non-existent objects or attributes. It probes object detection, attribute recognition, and spatial understanding under various prompts. Use when the user wants to benchmark on CHAIR (MSCOCO subset), POPE (COCO subset), MME (Hallucination subset), MMBench, or asks about evaluating this task. Reports CHAIR_s.Votes: 0GitHub stars: 3
- Mlp4rec EvalEvaluates a model's ability to predict the next item in a user's sequential interaction history. It probes the model's capacity to capture temporal dependencies, cross-channel correlations in item embeddings, and cross-feature interactions using only MLP-based architectures. Use when the user wants to benchmark on MovieLens-100k, Amazon Beauty, or asks about evaluating this task. Reports NDCG@10.Votes: 0GitHub stars: 3
- Mlperf Abfp EvalEvaluates the inference accuracy of deep neural networks when simulated with Adaptive Block Floating-Point (ABFP) number representation and analog-to-digital converter (ADC) noise. It probes how tile width, amplification gain, and bitwidth affect model quality, and compares the effectiveness of Quantization-Aware Training (QAT) versus Differential Noise Finetuning (DNF) in recovering baseline float32 performance. Use when the user wants to benchmark on MLPerf datacenter inference benchmark, o...Votes: 0GitHub stars: 3
- Mlperf Automotive EvalEvaluates real-time perception capabilities for automotive systems, specifically 2D object detection, 2D semantic segmentation, and 3D object detection. It measures how well inference engines meet strict latency constraints and accuracy tolerances required for safety-critical driving tasks. Use when the user wants to benchmark on nuScenes, MLCommons Cognata Dataset, or asks about evaluating this task. Reports tail_latency.Votes: 0GitHub stars: 3
- Mlperf Bert EvalEvaluates the distributed training efficiency of the BERT-Large model on variable-length masked language modeling tasks. It measures how quickly the model converges to a target accuracy and the sustained throughput achieved across multiple GPUs. Use when the user wants to benchmark on MLPerf BERT, or asks about evaluating this task. Reports Time to 72% MLM accuracy.Votes: 0GitHub stars: 3
- Mlperf Edge Inference EvalThis protocol evaluates the inference performance of DNN models under various quantization precisions (FP16/INT8, static/dynamic) across multiple inference frameworks on edge and server hardware. It measures standard accuracy and latency metrics using the MLPerf Edge Inference benchmark suite. Use when the user wants to benchmark on ImageNet ILSVRC2012, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mlperf Hardware EvalEvaluates the performance and energy efficiency of a proposed deep learning hardware architecture against state-of-the-art GPUs and specialized accelerators. It probes the architecture's ability to handle diverse DL workloads (CNNs, transformers, RNNs) across different batch sizes and its software maturity for general-purpose mapping. Use when the user wants to benchmark on MLPerf benchmark suite, or asks about evaluating this task. Reports Speedup, Rel. Efficiency.Votes: 0GitHub stars: 3
- Mlperf Hpc EvalEvaluates end-to-end performance of HPC systems for scientific machine learning, focusing on data staging, I/O efficiency, and model convergence under massive dataset constraints. It measures how well systems handle large-scale volumetric and high-resolution image workloads while meeting strict accuracy targets. Use when the user wants to benchmark on CosmoFlow, DeepCAM, or asks about evaluating this task. Reports MAE, IOU.Votes: 0GitHub stars: 3
- Mlperf Inference EvalEvaluates ML inference systems across diverse hardware and software stacks under realistic deployment scenarios. It measures both model quality against strict baselines and system performance (latency/throughput) to enable architecture-neutral comparisons of production-like workloads. Use when the user wants to benchmark on ImageNet, COCO, WMT16 EN-DE, or asks about evaluating this task. Reports Top-1 accuracy.Votes: 0GitHub stars: 3
- Mlperf Input Pipeline EvalThis evaluation protocol measures the throughput and latency of machine learning input data pipelines across standard computer vision and NLP benchmarks. It probes how efficiently a data processing framework can ingest, transform, and feed batches to a training loop compared to sequential baselines and competing systems. Use when the user wants to benchmark on ImageNet, COCO, WMT16, WMT17, or asks about evaluating this task. Reports epoch duration.Votes: 0GitHub stars: 3
- Mlperf Mobile EvalEvaluates mobile AI inference performance across computer vision and NLP tasks on resource-constrained hardware. It measures both model accuracy against strict thresholds and hardware/software stack efficiency under realistic deployment conditions. Use when the user wants to benchmark on ImageNet 2012 validation, COCO 2017 validation, ADE20K validation, SQuAD v1.1 Dev, or asks about evaluating this task. Reports Top-1 accuracy.Votes: 0GitHub stars: 3
- Mlperf Power EvalEvaluates the energy efficiency of machine learning systems across diverse hardware scales (data center, edge, tiny) and workloads (inference and training). It measures how effectively systems convert electrical energy into computational progress, tracking improvements in samples processed per joule over time and across system configurations. Use when the user wants to benchmark on MLPerf, or asks about evaluating this task. Reports samples per joule.Votes: 0GitHub stars: 3
- Mlperf Tiny EvalEvaluates the classification accuracy and hardware efficiency of CNN accelerators on resource-constrained TinyML workloads. It probes the trade-off between inference latency, energy consumption, and model accuracy under post-training approximate matrix decomposition. Use when the user wants to benchmark on MLPerfTiny, or asks about evaluating this task. Reports Top-1 Accuracy.Votes: 0GitHub stars: 3
- Mlperf Tiny Nas EvalEvaluates whether differentiable NAS methods can discover neural architectures that maximize classification accuracy while minimizing energy consumption and latency. The protocol enforces strict weight memory constraints on edge hardware and measures real-world deployment metrics on the NUCLEO-H743ZI2 MCU. Use when the user wants to benchmark on Image Classification (CIFAR-10), Visual Wake Word (MSCOCO 2014), KeyWord Spotting (Speech Commands v2), or asks about evaluating this task. Reports a...Votes: 0GitHub stars: 3
- Mlperf Tpu EvalEvaluates the performance and communication overhead of fault-tolerant 2-D allreduce algorithms compared to standard allreduce during data-parallel ML training on TPU-v3 mesh networks. It measures end-to-end training latency and relative efficiency under simulated chip failure conditions. Use when the user wants to benchmark on MLPerf-v0.7 ResNet-50, MLPerf-v0.7 BERT, or asks about evaluating this task. Reports Relative Efficiency.Votes: 0GitHub stars: 3
- Mlperf Training EvalEvaluates the training performance and scalability of machine learning implementations across diverse hardware and software stacks by measuring time to solution under standardized model and hyperparameter configurations. Use when the user wants to benchmark on MLPerf Training Suite, or asks about evaluating this task. Reports time to solution.Votes: 0GitHub stars: 3
- Mlpf Particle Flow EvalEvaluates a graph neural network's ability to reconstruct particle-flow objects (charged and neutral hadrons) from detector-level tracks and calorimeter clusters in high-pileup simulated events. It probes multi-task learning for particle classification and momentum/energy regression under realistic collider conditions. Use when the user wants to benchmark on DELPHES simulated QCD multijet and ttbar events, or asks about evaluating this task. Reports Efficiency.Votes: 0GitHub stars: 3
- Mlqa EvalEvaluates cross-lingual extractive question answering by measuring how well models can answer questions in one language using context in another, and how performance generalizes across different language pairs. Use when the user wants to benchmark on MLQA, or asks about evaluating this task. Reports F1 score.Votes: 0GitHub stars: 3
- Mlqa Xquad EvalThis benchmark evaluates a model's ability to perform cross-lingual extractive reading comprehension in a zero-shot setting. It probes how well a model trained on high-resource English (and Chinese) data can generalize to answer questions in low-resource languages by leveraging translated parallel corpora and multilingual attention mechanisms. Use when the user wants to benchmark on MLQA, XQuAD, or asks about evaluating this task. Reports F1.Votes: 0GitHub stars: 3
- Mls Asr EvalEvaluates automatic speech recognition (ASR) systems on multilingual read speech data. It measures word error rate across multiple languages and decoding strategies to assess model performance and data scaling effects. Use when the user wants to benchmark on MLS, or asks about evaluating this task. Reports WER.Votes: 0GitHub stars: 3
- Mlsum EvalEvaluates abstractive text summarization models across multiple languages (French, German, Spanish, Russian, Turkish) to measure generation quality and investigate cross-lingual performance gaps and model biases. Use when the user wants to benchmark on MLSUM, or asks about evaluating this task. Reports ROUGE-L.Votes: 0GitHub stars: 3
- Mlvu EvalEvaluates long-horizon video understanding and reasoning capabilities of multimodal models on extended video sequences. Use when the user wants to benchmark on MLVU, or asks about evaluating this task. Reports M-Avg.Votes: 0GitHub stars: 3
- Mm Alignbench EvalEvaluates how well multi-modal large language models align with human preferences when answering open-ended questions about diverse images. It probes the model's ability to follow complex instructions, handle real-world scenarios, and produce responses that match human expectations better than baseline models. Use when the user wants to benchmark on MM-AlignBench, or asks about evaluating this task. Reports Win Rate.Votes: 0GitHub stars: 3
- Mm Bright EvalThis benchmark evaluates reasoning-intensive retrieval capabilities across text-only and multimodal settings. It probes models' ability to align visual and textual information, navigate technical domain queries, and rank relevant documents or images based on complex, multi-modal prompts. Use when the user wants to benchmark on MM-BRIGHT, or asks about evaluating this task. Reports nDCG@10.Votes: 0GitHub stars: 3
- Mm Food 100k EvalEvaluates the predictive capability of vision-language models on food-related tasks, specifically testing their ability to estimate nutritional content (kilocalories) and identify categorical food attributes (dish name, ingredients, cooking method) from images. The protocol isolates the value of structured, human-verified data by comparing base foundation models against their supervised fine-tuned counterparts on a frozen test split. Use when the user wants to benchmark on MM-Food-100K, or as...Votes: 0GitHub stars: 3
- Mm Graph EvalEvaluates multimodal graph learning models on node classification, link prediction, and knowledge graph completion tasks. It probes the ability of models to integrate high-resolution visual and textual node features with graph structure to make accurate predictions across diverse real-world domains. Use when the user wants to benchmark on Amazon-Sports, Amazon-Cloth, Goodreads-LP, Goodreads-NC, Ele-fashion, MM-CoDEx-s, MM-CoDEx-m, or asks about evaluating this task. Reports MRR, accuracy.Votes: 0GitHub stars: 3
- Mm Inference EvalThis evaluation probes the effectiveness and efficiency of dynamic sparse attention methods for long-context vision-language models (VLMs). It tests the model's ability to perform long-video understanding, retrieve specific visual or mixed-modality information from extremely long contexts (Needle in a Haystack), and maintain accuracy while reducing computational cost and latency. Use when the user wants to benchmark on Video Understanding Benchmarks, V-NIAH, MM-NIAH, or asks about evaluating ...Votes: 0GitHub stars: 3
- Mm Iq EvalProbes human-like abstraction and visual reasoning capabilities in multimodal models across eight fine-grained paradigms, including logical operations, geometry, and spatial relationships. It measures how well models generalize to novel abstract patterns without relying on memorized visual features. Use when the user wants to benchmark on MM-IQ, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mm Judgebench EvalEvaluates the cross-lingual generalization and robustness of Large Vision-Language Models (LVLMs) acting as automated judges. It probes their ability to correctly rank paired multimodal responses across 25 languages while measuring susceptibility to positional and length biases. Use when the user wants to benchmark on MM-JudgeBench, or asks about evaluating this task. Reports average accuracy.Votes: 0GitHub stars: 3
- Mm Neuroonco EvalThis benchmark evaluates the multimodal diagnostic reasoning capabilities of large vision-language models on brain tumor MRI scans. It probes whether models can integrate subtle visual cues with structured anatomical knowledge to produce accurate diagnoses, while also measuring their ability to recognize uncertainty through explicit rejection options. Use when the user wants to benchmark on MM-NeuroOnco-Bench, or asks about evaluating this task. Reports Accuracy.Votes: 0GitHub stars: 3
- Mm Safetybench++ EvalEvaluates contextual safety in multi-modal large language models by measuring how well they refuse harmful queries while correctly answering safe ones, with a focus on whether their safety reasoning aligns with the given context. It probes the model's ability to avoid over-defensive refusals on benign inputs while maintaining high response quality. Use when the user wants to benchmark on MM-SafetyBench++, or asks about evaluating this task. Reports Contextual Correctness Rate (CCR).Votes: 0GitHub stars: 3
- Mm Safetybench EvalEvaluates the safety and detoxification capabilities of multimodal large language models by measuring the fraction of harmful responses across various toxicity categories, while also assessing continuous toxicity severity and general multimodal reasoning capability. Use when the user wants to benchmark on MM-SafetyBench, or asks about evaluating this task. Reports Harmful Rate (HaR).Votes: 0GitHub stars: 3
- Mm Scale EvalEvaluates vision-language models' ability to perform fine-grained moral reasoning and safety alignment on multimodal scenarios. It probes how well models rank, calibrate, and separate safe from unsafe situations when provided with text, image, or combined modalities. Use when the user wants to benchmark on MM-Scale, or asks about evaluating this task. Reports NDCG@5.Votes: 0GitHub stars: 3
- Mm Uavbench EvalEvaluates Multimodal Large Language Models on low-altitude UAV scenarios, probing their capabilities in visual perception, multi-view spatial reasoning, and egocentric/exocentric planning across diverse real-world aerial imagery tasks. Use when the user wants to benchmark on MM-UAVBench, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mm Upd EvalEvaluates Large Multimodal Models' ability to detect unsolvable visual-question problems and appropriately refuse to answer. It specifically probes three failure modes: absent answers, incompatible answer sets, and image-question mismatches. Use when the user wants to benchmark on MM-UPD Bench, or asks about evaluating this task. Reports Dual accuracy.Votes: 0GitHub stars: 3
- Mm Upt EvalEvaluates the multi-modal mathematical reasoning capabilities of MLLMs on diverse visual math problems including geometry, charts, and tables. It tests the model's ability to solve multiple-choice and fill-in-the-blank questions using both human-created and synthetically generated unlabeled data. Use when the user wants to benchmark on MathVision, MathVerse, MathVista, We-Math, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mm Vet V2 EvalThis benchmark evaluates large multimodal models on integrated vision-language capabilities, with a specific focus on sequential image-text understanding, spatial reasoning, knowledge retrieval, and long-form generation. It probes how well models can process interleaved visual and textual inputs to answer complex, real-world questions. Use when the user wants to benchmark on MM-Vet v2, or asks about evaluating this task. Reports MM-Vet-v2 score.Votes: 0GitHub stars: 3
- Mmad Bbox EvalEvaluates the fine-grained localization capability of vision-language models on industrial anomaly detection. It measures how accurately models can predict bounding boxes around defects compared to ground-truth annotations and human experts. Use when the user wants to benchmark on MMAD-BBox, or asks about evaluating this task. Reports BBox-Mask IoU.Votes: 0GitHub stars: 3
- Mmad EvalEvaluates Multimodal Large Language Models (MLLMs) on industrial anomaly detection tasks, probing their ability to perform fine-grained visual reasoning, multi-image comparison, and defect-related classification, localization, and description. It specifically tests whether models can leverage template normal images and domain knowledge to identify and analyze anomalies in industrial products. Use when the user wants to benchmark on MMAD, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mmar EvalEvaluates deep audio reasoning capabilities by testing both final answer correctness and the logical quality of intermediate reasoning steps. It covers single-domain (sound, music, speech) and mixed-domain audio tasks to measure how well models avoid spurious correlations and follow verifiable reasoning paths. Use when the user wants to benchmark on MMAR, or asks about evaluating this task. Reports Avg.Votes: 0GitHub stars: 3
- Mmarco Passage Ranking EvalEvaluates multilingual passage retrieval models on translated versions of the MS MARCO dataset and the Mr. TyDi dataset. It probes the ability of dense retrieval and reranking models to handle cross-lingual and zero-shot retrieval scenarios, as well as the impact of translation quality on retrieval effectiveness. Use when the user wants to benchmark on mMARCO, Mr. TyDi, or asks about evaluating this task. Reports MRR@10.Votes: 0GitHub stars: 3
- Mmaudiosep Separation EvalEvaluates a generative model's ability to separate target sounds from mixture audio using video and text queries, while preserving video-to-audio generation capabilities. It probes multimodal conditioning, cross-domain knowledge transfer, and the perceptual quality of generated separated audio against discriminative baselines. Use when the user wants to benchmark on VGGSound-Clean, MUSIC, VGGSound, or asks about evaluating this task. Reports FAD.Votes: 0GitHub stars: 3