All authors
Claude Skills by qhjqhj00
github.com/qhjqhj007,572 skills29 installs10,436 views
- Mmwp EvalEvaluates large language models' ability to perform mathematical, commonsense, and natural language inference reasoning in low-, medium-, and high-resource languages. It specifically probes cross-lingual transfer capabilities using a zero-shot chain-of-thought setting without requiring parallel multilingual instruction data. Use when the user wants to benchmark on MMWP, MGSM, MSVAMP, X-CSQA, XNLI, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mmxu Test EvalEvaluates multi-modal vision-language models on their ability to perform visual question answering across two temporal X-ray images to detect regional disease progression. It probes temporal reasoning, subtle change detection, and bias mitigation in medical imaging diagnostics. Use when the user wants to benchmark on MMXU-test, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mnist Cl Scenarios EvalEvaluates a model's ability to learn sequentially from multiple tasks without catastrophic forgetting across three distinct continual learning splits of MNIST: incremental task, domain, and class learning. It measures how well baseline and continual learning methods retain knowledge from previous tasks while adapting to new ones. Use when the user wants to benchmark on MNIST, or asks about evaluating this task. Reports average accuracy (%).Votes: 0GitHub stars: 3
- Mnli Anli EvalEvaluates the out-of-domain generalization and robustness of NLI models trained under different data collection protocols. It measures how well models perform on held-out, genre-diverse, and adversarial benchmarks compared to in-domain validation performance. Use when the user wants to benchmark on MNLI-mismatched, ANLI, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mnmt EvalEvaluates multilingual neural machine translation performance across many-to-one, one-to-many, and many-to-many translation scenarios, testing how dynamic parameter differentiation impacts translation quality across diverse language pairs and resource levels. Use when the user wants to benchmark on OPUS, WMT, IWSLT'17, or asks about evaluating this task. Reports BLEU.Votes: 0GitHub stars: 3
- Mnn Inference Benchmark EvalEvaluates the inference latency and computational efficiency of mobile deep learning engines across different hardware platforms, compute backends, and neural network architectures. Use when the user wants to benchmark on MobileNet-v1, SqueezeNet-v1.1, ResNet-18, Inception-v3, or asks about evaluating this task. Reports inference time (ms).Votes: 0GitHub stars: 3
- Mnv 17 EvalEvaluates the ability of speech recognition models to jointly transcribe Mandarin speech and identify nonverbal vocalizations (NVs) like laughs or sighs. It also isolates the strict accuracy of NV event detection and measures whether adding NV recognition degrades core lexical transcription performance. Use when the user wants to benchmark on MNV-17, or asks about evaluating this task. Reports CER.Votes: 0GitHub stars: 3
- Mo Int 20 EvalEvaluates the ability of automated theorem provers and LLMs to formally prove complex algebraic inequalities at the International Mathematical Olympiad level using a deductive search engine in Lean. Use when the user wants to benchmark on MO-INT-20, or asks about evaluating this task. Reports number of solved problems.Votes: 0GitHub stars: 3
- Mobclip Downstream EvalEvaluates geospatial representation models on 11 regression tasks spanning social, economic, and natural domains across multiple spatial scales (point, grid, county, city). It probes the model's ability to capture complex human-centric and environmental patterns using fused multimodal embeddings rather than relying solely on geographic coordinates. Use when the user wants to benchmark on MobCLIP Downstream Benchmark (11 tasks), or asks about evaluating this task. Reports R^2.Votes: 0GitHub stars: 3
- Mobibench EvalEvaluates mobile GUI agents' ability to complete tasks on mobile interfaces by measuring task success rate across diverse, multi-path trajectories. It also enables modular component analysis (screen parsing, history generation, inference style, reflection) to identify performance bottlenecks and optimal configurations for different foundation models. Use when the user wants to benchmark on MobiBench, or asks about evaluating this task. Reports Task Success Rate (TSR).Votes: 0GitHub stars: 3
- Mobiface EvalThis benchmark evaluates the capability of visual tracking algorithms to maintain robust face localization in unconstrained, mobile-captured video sequences. It specifically probes resilience to challenging real-world conditions such as rapid camera motion, out-of-plane rotations, scale changes, and partial occlusions. Use when the user wants to benchmark on iBUG MobiFace, or asks about evaluating this task. Reports AUC.Votes: 0GitHub stars: 3
- Mobiface Lfw Megaface EvalEvaluates the accuracy and robustness of lightweight face recognition models on unconstrained face verification tasks, specifically measuring pair-wise verification accuracy and identification rates under extreme distractor conditions. Use when the user wants to benchmark on Labeled Faces in the Wild (LFW), MegaFace, or asks about evaluating this task. Reports Accuracy, TAR@FAR=10^-6.Votes: 0GitHub stars: 3
- Mobile Agent V2 EvalProbes an agent's ability to navigate and execute multi-step UI operations on real mobile devices (Android/HarmonyOS) based on natural language instructions. It evaluates end-to-end task completion, step-level correctness, decision-making precision, and the capacity to detect and correct operational errors via reflection. Use when the user wants to benchmark on Mobile-Agent-v2 Evaluation Set, or asks about evaluating this task. Reports Success Rate (SR).Votes: 0GitHub stars: 3
- Mobile Bench EvalEvaluates LLM-based mobile agents on real-world task execution across single-app and multi-app scenarios. It probes the agent's ability to plan, navigate UIs, call APIs, and collaborate across applications to complete user-defined goals. Use when the user wants to benchmark on Mobile-Bench, or asks about evaluating this task. Reports PassRate.Votes: 0GitHub stars: 3
- Mobile Construction EvalTests a robot's ability to simultaneously navigate and construct a target structure in 1D/2D/3D grid worlds under partial observability and environmental uncertainty. It evaluates the bi-directional coupling between localization and manipulation planning. Use when the user wants to benchmark on Mobile Construction Benchmark, or asks about evaluating this task. Reports IoU.Votes: 0GitHub stars: 3
- Mobile Dl Inference EvalEvaluates the effectiveness of parallel deep learning inference strategies across heterogeneous mobile processors (CPU, GPU, DSP) under varying workloads and dynamic system conditions. It probes how operator support, scheduling granularity, and competing processes impact inference latency, resource utilization, and system responsiveness. Use when the user wants to benchmark on Standard DL Models (YOLOv2, VGG-16, PoseNet, FST, RetinaFace, ResNet-18, ResNet-50), or asks about evaluating this ta...Votes: 0GitHub stars: 3
- Mobile Dl Training PerformanceThis evaluation probes the hardware efficiency and resource constraints of training deep learning models on mobile SoCs. It measures how different model architectures and batch sizes impact GPU/CPU utilization, power/energy draw, and memory footprint during training. Use when the user has predictions and gold and needs to compute GPU utilization.Votes: 0GitHub stars: 3
- Mobile EvalEvaluates an autonomous mobile device agent's ability to execute multi-step UI operations using visual perception. It probes task planning, self-reflection, and cross-application interaction under varying instruction complexity. Use when the user wants to benchmark on Mobile-Eval, or asks about evaluating this task. Reports Success (Su).Votes: 0GitHub stars: 3
- Mobile Gesture Recognition EvalThis evaluation probes a model's ability to recognize hand gestures from mobile device inertial sensor data (accelerometer and gyroscope). It measures classification robustness across varying signal speeds, amplitudes, and noise levels by testing on three distinct datasets with different gesture sets and sensor configurations. Use when the user wants to benchmark on MGD, BUAA Mobile Gesture Database, SmartWatch Gesture Database, or asks about evaluating this task. Reports classification accur...Votes: 0GitHub stars: 3
- Mobile Gpu Inference EvalThis benchmark evaluates the inference performance of a mobile GPU-accelerated neural network engine. It measures execution latency and peak memory consumption across different hardware platforms, precision formats (FP32 vs FP16), and batch sizes. It probes the engine's ability to leverage GPU parallelism and half-precision arithmetic for efficient edge deployment. Use when the user wants to benchmark on MNIST, Cifar-10, Style Transfer Dataset, or asks about evaluating this task. Reports exec...Votes: 0GitHub stars: 3
- Mobile Gui Agent EvalEvaluates the capability of multimodal large language models to act as autonomous mobile GUI agents. It probes their ability to plan tasks, predict correct interaction types, accurately ground UI elements, and successfully complete complex, multi-step workflows in both static and dynamic Android environments. Use when the user wants to benchmark on AndroidControl, AndroidLab, Android Agent Arena (A3), or asks about evaluating this task. Reports Success Rate (SR).Votes: 0GitHub stars: 3
- Mobile Inference Benchmark EvalEvaluates the practical feasibility of running deep learning models on mobile hardware by measuring end-to-end latency and energy consumption during CNN inference. It compares on-device execution against cloud-based execution to identify hardware and network bottlenecks. Use when the user wants to benchmark on Mobile Benchmark Image Set, or asks about evaluating this task. Reports end-to-end latency.Votes: 0GitHub stars: 3
- Mobile Mmlu EvalEvaluates language models' understanding of mobile-specific domains and tasks under on-device constraints. It probes the models' ability to answer multiple-choice questions across 80 real-world mobile domains, emphasizing practical usability, privacy, and personalization in daily mobile interactions. Use when the user wants to benchmark on Mobile-MMLU, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mobile O EvalEvaluates a compact on-device unified vision-language-diffusion model's capabilities in multimodal understanding, text-to-image generation, and image editing. It probes the model's ability to align cross-modal representations and generate high-fidelity images while maintaining real-time inference speeds on edge hardware. Use when the user wants to benchmark on GenEval, MMMU, MM-Vet, SEED, TextVQA, ChartQA, POPE, GQA, ImageEdit, or asks about evaluating this task. Reports GenEval overall score.Votes: 0GitHub stars: 3
- Mobile R1 Benchmark EvalEvaluates a vision-language model's ability to navigate and complete multi-step tasks on mobile GUIs. It measures step-level action correctness, full trajectory success, and robustness to intermediate errors in a simulated Android environment. Use when the user wants to benchmark on Chinese Mobile Agent Benchmark, or asks about evaluating this task. Reports Accuracy (Acc.).Votes: 0GitHub stars: 3
- Mobile Skin Classification EvalEvaluates a CNN model's ability to classify smartphone-captured images of skin lesions into one of seven dermatological conditions. It probes the model's robustness to class imbalance and the effectiveness of data preprocessing strategies like oversampling and augmentation. Use when the user wants to benchmark on Smartphone Skin Disease Dataset, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Mobile Traffic Forecasting EvalEvaluates the ability of deep learning models to forecast multi-service mobile network traffic volumes at the antenna level over short time horizons (up to 1 hour). It probes spatiotemporal sequence modeling by requiring models to capture both spatial correlations across antennas and temporal dependencies in traffic patterns. Use when the user wants to benchmark on Real-world mobile traffic dataset (36 services, 800 antennas), or asks about evaluating this task. Reports MAE.Votes: 0GitHub stars: 3
- Mobile Vlm Deployment EvalEvaluates the runtime efficiency, hardware utilization, and thermal/energy impact of deploying vision-language models on mobile devices. It measures latency breakdowns, CPU/GPU/NPU usage, power consumption, and output characteristics across different inference frameworks. Use when the user wants to benchmark on Custom Mobile VLM Inference Test Set, or asks about evaluating this task. Reports Latency.Votes: 0GitHub stars: 3
- Mobileagentbench EvalThis benchmark evaluates the performance of LLM-based mobile agents on Android GUI navigation tasks. It measures end-to-end task completion, action efficiency, response latency, computational cost, and the agent's ability to correctly determine task completion without stopping too early or too late. Use when the user wants to benchmark on MobileAgentBench, or asks about evaluating this task. Reports Success Rate (SR).Votes: 0GitHub stars: 3
- Mobileaibench EvalEvaluates the task performance, latency, and hardware utilization of quantized LLMs and LMMs on real mobile devices across standard NLP, multi-modal, and trust & safety benchmarks. Use when the user wants to benchmark on Databricks, HotpotQA, sql-create-context, CNN, XSum, VQA-v2, GQA, VisWiz, TextVQA, SQA, AlpacaEval, MT-Bench, MMLU, GSM8K, TruthQA, BBQ, SC-101, Adv-Inst, DNA, Priv-Lk, or asks about evaluating this task. Reports Accuracy.Votes: 0GitHub stars: 3
- Mobilecaps Covidx EvalEvaluates a lightweight hybrid deep learning model for classifying chest X-ray images into COVID-19, Normal, and Pneumonia categories, and optionally predicting disease severity scores. Use when the user wants to benchmark on COVIDx, or asks about evaluating this task. Reports F1 Score.Votes: 0GitHub stars: 3
- Mobilei2v I2v EvalEvaluates the visual quality and generation speed of image-to-video diffusion models optimized for mobile deployment. It probes the model's ability to generate temporally coherent 17-frame videos from a single reference image while maintaining high resolution and low latency on mobile hardware. Use when the user wants to benchmark on Unspecified (FVD benchmarks), or asks about evaluating this task. Reports FVDhum.Votes: 0GitHub stars: 3
- Mobilekernelbench EvalEvaluates LLMs' capability to generate syntactically valid, functionally correct, and hardware-efficient C/C++ kernels for mobile inference engines. It probes framework-specific API usage, compilation robustness, functional verification against ONNX baselines, and runtime speedup optimization. Use when the user wants to benchmark on MobileKernelBench, or asks about evaluating this task. Reports Compilation success rate (CSR).Votes: 0GitHub stars: 3
- Mobileworld EvalEvaluates autonomous mobile agents on long-horizon, cross-application workflows, requiring them to handle ambiguous instructions via agent-user interaction and integrate external tools via MCP. It probes planning, GUI grounding, clarification strategies, and tool orchestration in real-world mobile environments. Use when the user wants to benchmark on MobileWorld, or asks about evaluating this task. Reports Success Rate (SR).Votes: 0GitHub stars: 3
- Mobility Forecasting EvalThis benchmark evaluates the accuracy of deep learning models for multivariate time series forecasting of aggregated human mobility across six urban regions. It specifically probes how well models can predict short-term (30-minute) passenger counts while preserving differential privacy through input or gradient perturbation mechanisms. Use when the user wants to benchmark on Paris Mobility Dataset, or asks about evaluating this task. Reports RMSE.Votes: 0GitHub stars: 3
- Mobility Timeseries EvalEvaluates the accuracy of time series forecasting models on urban mobility data across different prediction horizons. It probes how well traditional, deep learning, and foundation models capture short-term, medium-term, and long-term temporal dependencies in bike-sharing flows. Use when the user wants to benchmark on BikeNYC, BikeVIE, or asks about evaluating this task. Reports RMSE.Votes: 0GitHub stars: 3
- Mobilitybench EvalEvaluates LLM-based route-planning agents on real-world mobility queries, probing their ability to handle multi-waypoint itineraries, preference-constrained routing, and multimodal travel. It measures how well agents understand instructions, decompose tasks, select tools, and produce valid, constraint-satisfying routes. Use when the user wants to benchmark on MobilityBench, or asks about evaluating this task. Reports Final Pass Rate (FPR).Votes: 0GitHub stars: 3
- Mocentric Bench EvalEvaluates whether video multi-modal LLMs genuinely utilize motion cues for pixel-level visual grounding. It specifically probes their ability to distinguish true motion from static fake motion (Motion Existence) and to differentiate forward from reversed motion sequences (Motion Order). Use when the user wants to benchmark on MoCentric-Bench, or asks about evaluating this task. Reports mIoU.Votes: 0GitHub stars: 3
- Modad EvalEvaluates a model's ability to mitigate spurious correlations (bias) in image classification by measuring performance on both overall test sets and specifically on bias-conflicting samples where the spurious attribute contradicts the true label. Use when the user wants to benchmark on Corrupted CIFAR-10, BAR, BFFHQ, Waterbirds, or asks about evaluating this task. Reports Average Accuracy.Votes: 0GitHub stars: 3
- Modality Bias EvalThis evaluation probes a model's ability to automatically detect and classify sample-specific modality bias in multimodal misinformation content. It measures how well automated quantification methods align with human judgment regarding whether a sample relies on image-only, text-only, or balanced modalities. Use when the user wants to benchmark on Fakeddit, MMFakeBench, or asks about evaluating this task. Reports Accuracy.Votes: 0GitHub stars: 3
- Modeconv Anomaly Detection EvalEvaluates a graph neural network's ability to detect structural anomalies by reconstructing multivariate time-series sensor data. It measures how well the model captures physical material properties and eigenmode shifts compared to spectral GNN baselines, while also benchmarking computational efficiency. Use when the user wants to benchmark on Luxembourg dataset, Simulated Smart Bridge dataset, or asks about evaluating this task. Reports reconstruction error.Votes: 0GitHub stars: 3
- Model Recovery EvalThis benchmark evaluates the accuracy and hardware efficiency of neural flow-based architectures for recovering underlying dynamics from time-series data. It probes the model's ability to estimate parameters of nonlinear dynamical systems while measuring computational resource constraints like runtime, power, and memory footprint on edge hardware. Use when the user wants to benchmark on Chaotic Lorenz, F8 Cruiser, Lotka Volterra, Pathogenic Attack System, Automated Insulin Delivery (OhioT1D),...Votes: 0GitHub stars: 3
- Model Selection Energy EvalThis evaluation protocol assesses the trade-off between model size (parameter count) and task utility across multiple AI benchmarks. It aims to identify energy-efficient models that maintain high performance, enabling estimation of global AI inference energy savings through strategic model selection. Use when the user wants to benchmark on OpenLLM Leaderboard, LMSys Chatbot Arena, NPHardEval, BigCode Leaderboard, mtebLeaderboard, WMT English-German, Open Object Detection Leaderboard, ImageNet...Votes: 0GitHub stars: 3
- Model Serving Latency EvalEvaluates the inference latency and end-to-end turn-around time of five machine learning model-serving frameworks across four distinct real-world inference scenarios. It probes how framework specialization (DL-specific vs. general-purpose) and input payload size affect serving performance and stability. Use when the user wants to benchmark on Malware detection, Cryptocoin price forecasting, Image classification, Sentiment analysis, or asks about evaluating this task. Reports average_latency.Votes: 0GitHub stars: 3
- Model Written EvaluationsTests whether language models exhibit specific emergent or undesirable behaviors (e.g., sycophancy, self-preservation, political bias) by measuring their preference for behavior-matching versus behavior-mismatching labels. It evaluates how well smaller or preference-trained models predict the behavioral tendencies of larger or RLHF-trained counterparts. Use when the user wants to benchmark on Model-Written Evaluations (133 behaviors), or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Modeltables EvalEvaluates the ability of retrieval systems to find relevant AI model tables from a heterogeneous corpus. It probes semantic understanding of structured data and cross-source table matching. Use when the user wants to benchmark on ModelTables, or asks about evaluating this task. Reports P@1.Votes: 0GitHub stars: 3
- Modified Median Absolute DeviationEvaluates the precision and accuracy of stellar flux recovery and positional measurements in simulated Roman Space Telescope images. It probes how well effective PSF models can recover input fluxes and coordinates across different spatial grids, filters, and magnitudes. Use when the user has predictions and gold and needs to compute modified median absolute deviation ($\hat{\sigma}$).Votes: 0GitHub stars: 3
- ModifiedpanopticqualityCompute the ModifiedPanopticQuality metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute ModifiedPanopticQuality, or asks how to score with ModifiedPanopticQuality.Votes: 0GitHub stars: 3
- Modnet Eval ProtocolEvaluates the predictive accuracy and uncertainty quantification of a descriptor-based neural network on materials property prediction tasks, with a focus on low-data regimes and bias-imbalance. Use when the user wants to benchmark on Naccarato et al. refractive index, Petretto et al. vibrational thermodynamics, or asks about evaluating this task. Reports mean absolute error.Votes: 0GitHub stars: 3
- Modscan EvalMeasures stereotypical bias in large vision-language models across gender, race, and occupational/persona attributes. It evaluates how model outputs deviate from real-world demographic baselines or equal distribution when presented with visual inputs paired with text prompts. Use when the user wants to benchmark on UTKFace, SD-v2.1 Generated (Persona Traits), or asks about evaluating this task. Reports stereotypical_bias.Votes: 0GitHub stars: 3