All authors
Claude Skills by qhjqhj00
github.com/qhjqhj007,572 skills29 installs10,436 views
- Featureid 3ds EvalEvaluates a derivative-free camera control policy's ability to align vision-language models in 3D multi-object scenes. It probes robustness to viewpoint changes and object occlusions using minimal demonstration data. Use when the user wants to benchmark on FeatureID-3DS, PartialView-3DS, or asks about evaluating this task. Reports prediction error.Votes: 0GitHub stars: 3
- Fed Ecg EvalEvaluates federated learning algorithms on ECG classification tasks under non-IID and long-tailed label distribution challenges across multiple medical institutions. It probes how well FL methods generalize across heterogeneous clinical data and handle class imbalance without centralizing all data. Use when the user wants to benchmark on Fed-ECG, or asks about evaluating this task. Reports Micro F1-Score (Mi-F1).Votes: 0GitHub stars: 3
- Fed Echo EvalEvaluates federated learning algorithms on echocardiogram video segmentation tasks under label incompleteness and high data heterogeneity across multiple medical institutions. It probes how FL methods handle missing annotations and conflicting labels from different clinical sites. Use when the user wants to benchmark on Fed-ECHO, or asks about evaluating this task. Reports DICE.Votes: 0GitHub stars: 3
- Fed Plora EvalEvaluates the performance of heterogeneous federated fine-tuning methods across multiple NLP domains (instruction following, NLU, finance) under both IID and non-IID data partitioning settings. It probes the ability of LoRA-based federated learning algorithms to maintain model accuracy while accommodating resource-constrained clients with varying rank allocations. Use when the user wants to benchmark on Natural Instructions, GLUE benchmark, FPB, FIQA, TFNS, or asks about evaluating this task....Votes: 0GitHub stars: 3
- Feda4fair EvalEvaluates the fairness of federated learning models across heterogeneous client distributions. It probes whether server-level aggregation masks persistent unfairness at the individual client level by measuring demographic disparity and equal opportunity difference on bias-heterogeneous tabular datasets. Use when the user wants to benchmark on attribute-silo, value-silo, attribute-device, value-device, or asks about evaluating this task. Reports DD.Votes: 0GitHub stars: 3
- Fedaiot EvalEvaluates federated learning algorithms on authentic IoT data modalities under realistic constraints like non-IID data partitioning, label noise, and quantized training. It probes how data heterogeneity, client sampling ratios, and hardware limitations affect model convergence and final performance across diverse sensing tasks. Use when the user wants to benchmark on WISDM-W, WISDM-P, UT-HAR, Widar, VisDrone, CASAS, AEP, EPIC-SOUNDS, or asks about evaluating this task. Reports Accuracy (%).Votes: 0GitHub stars: 3
- Federated Llm Peft EvalEvaluates the effectiveness and efficiency of federated fine-tuning large language models using parameter-efficient fine-tuning (PEFT) algorithms across code generation, general language, and mathematical reasoning tasks under different data heterogeneity and privacy constraints. Use when the user wants to benchmark on Fed-CodeAlpaca, Fed-Dolly, Fed-GSM8K-3, HumanEval, HELM, GSM8K-test, or asks about evaluating this task. Reports Evaluation Scores(%).Votes: 0GitHub stars: 3
- Fedgraphnn System EvalEvaluates the computational efficiency and security overhead of federated graph neural network training. Measures how system-level metrics like training time, FLOPs, and parameter counts scale across diverse graph datasets under non-IID data partitioning and secure aggregation protocols. Use when the user wants to benchmark on SIDER, BACE, Clintox, BBBP, Tox21, FreeSolv, ESOL, Lipo, hERG, QM9, Ciao, Epinions, CORA, Citeseer, DBLP, PubMed, or asks about evaluating this task. Reports Wall-clock...Votes: 0GitHub stars: 3
- Fedhpo B EvalEvaluates the effectiveness of federated hyperparameter optimization (FedHPO) methods across diverse FL tasks. It measures how well optimizers can find high-performing hyperparameter configurations under distributed, communication-constrained, and heterogeneous data settings. Use when the user wants to benchmark on FedHPO-B, or asks about evaluating this task. Reports mean rank.Votes: 0GitHub stars: 3
- Fedmosaic EvalEvaluates personalized federated learning (PFL) methods on heterogeneous multi-modal and text-only clients. It probes a model's ability to personalize to its own data distribution ('Self') while maintaining generalization to unseen or other clients' tasks ('Others') under both static and dynamic distribution shifts. Use when the user wants to benchmark on DRAKE, HFLB, Fed-Scope, Fed-Aya, Fed-LLM-Large, or asks about evaluating this task. Reports A_last.Votes: 0GitHub stars: 3
- Fednet Traffic Forecasting EvalEvaluates the ability of a federated learning model to perform multi-step time-series forecasting of node-level traffic in optical networks, measuring how prediction accuracy degrades with longer history windows and forecasting horizons. Use when the user wants to benchmark on Optical network traffic dataset, or asks about evaluating this task. Reports R² score.Votes: 0GitHub stars: 3
- Feedbackqa EvalThis benchmark evaluates retrieval-based question answering systems and their ability to incorporate post-deployment user feedback. It probes a model's capacity to retrieve relevant answer passages, generate human-like explanations for answer quality, and rerank candidate answers using interactive feedback signals. Use when the user wants to benchmark on FEEDBACKQA, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Feedsum EvalThis benchmark evaluates how well LLM-generated feedback aligns with human preferences for text summarization, and tests whether preference learning (DPO) using multi-dimensional feedback improves summary quality over supervised fine-tuning. Use when the user wants to benchmark on FeedSum, or asks about evaluating this task. Reports Spearman correlation.Votes: 0GitHub stars: 3
- Feel Emotion EvalEvaluates the generalization and transferability of emotion recognition models across heterogeneous physiological signal datasets. It probes how well different modeling paradigms (handcrafted features, raw signal deep learning, and contrastive pretraining) perform under subject-independent, cross-dataset, and low-data regimes. Use when the user wants to benchmark on WESAD, NURSE, EMOGNITION, UBFC_PHYS, PhyMER, EmoWear, MAUS, CLAS, CASE, Unobtrusive, CEAP-360VR, ScientISST MOVE, Dapper, ForDig...Votes: 0GitHub stars: 3
- Felipehonorato EerCompute Felipehonorato/eer via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of Felipehonorato/eer.Votes: 0GitHub stars: 3
- Fengwu Weather Forecast EvalEvaluates global medium-range weather forecasting accuracy over 10-day lead times across multiple atmospheric variables. It measures point-wise prediction error and anomaly correlation against observed climatology, with explicit latitude weighting to account for spherical grid distortion. Use when the user wants to benchmark on ERA5, or asks about evaluating this task. Reports ACC.Votes: 0GitHub stars: 3
- FeqaEvaluates the faithfulness of abstractive summaries by generating questions from summary sentences and verifying if the answers can be extracted from the source document. It probes a model's ability to avoid hallucination and maintain factual grounding relative to the source text. Use when the user has predictions and gold and needs to compute FEQA.Votes: 0GitHub stars: 3
- Fer Gimefive EvalEvaluates the ability of convolutional neural networks to classify facial expressions into six basic emotion categories (happiness, surprise, sadness, anger, disgust, fear) from static images and video frames. The benchmark tests both classification accuracy and the model's capacity to generalize across in-the-wild and posed facial expression datasets. Use when the user wants to benchmark on RAF-DB, FER2013, FER GiMeFive, or asks about evaluating this task. Reports Accuracy (%).Votes: 0GitHub stars: 3
- Fettadbench EvalThis benchmark evaluates federated time-series anomaly detection systems by measuring how well models maintain detection accuracy when trained across decentralized clients compared to centralized baselines. It probes the robustness of anomaly detection architectures under federated learning protocols and varying degrees of non-IID data partitioning. Use when the user wants to benchmark on Time-series anomaly detection datasets (specific names not provided in excerpt), or asks about evaluating...Votes: 0GitHub stars: 3
- Few Shot Action Recognition EvalEvaluates a model's ability to recognize human actions in video clips using only a few labeled examples per class. It probes the model's capacity to leverage motion dynamics and semantic cues for robust classification under data-scarce conditions. Use when the user wants to benchmark on Something-Something, Kinetics, UCF101, HMDB51, FineGym, or asks about evaluating this task. Reports average few-shot accuracy.Votes: 0GitHub stars: 3
- Few Shot Audio Classification EvalThis benchmark evaluates few-shot audio classification capability across diverse acoustic domains including environmental sounds, musical instruments, bird species, and speaker recognition. It measures how well a model generalizes to novel classes with only a few labeled examples per class using a prototypical network framework. Use when the user wants to benchmark on ESC-50, FSD2018, NSynth, BirdCLEF 2020, VoxCeleb1, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Few Shot Bot EvalEvaluates prompt-based large language models on 15 diverse dialogue tasks, including response generation, conversational parsing, and skill selection, using a few-shot learning setup without fine-tuning. The protocol tests the model's ability to dynamically select the most appropriate task prompt based on dialogue history and generate accurate responses or parses. Use when the user wants to benchmark on Persona Chat, Empathetic Dialogues (ED), Wizard of Wikipedia (WoW), Image Chat (IC), Wizar...Votes: 0GitHub stars: 3
- Few Shot Classification EvalEvaluates few-shot classification performance under standard and explicit data-shift conditions. It probes a model's ability to generalize from a small number of labeled support samples to query samples across different image domains and distribution shifts. Use when the user wants to benchmark on miniImageNet, Office-Home, Easy-Office-Home, Hard-Office-Home, or asks about evaluating this task. Reports classification accuracy.Votes: 0GitHub stars: 3
- Few Shot Entity Recognition EvalEvaluates few-shot entity recognition in document images by measuring how well a model identifies and classifies entity spans using minimal labeled examples. It probes the model's ability to jointly leverage textual semantics and spatial layout information under low-data regimes. Use when the user wants to benchmark on FUNSD, CORD-Lv1, or asks about evaluating this task. Reports word-level F-1 score.Votes: 0GitHub stars: 3
- Few Shot Histopathology EvalEvaluates the few-shot classification capability of deep learning models on histopathology images. It probes how well models generalize from extremely limited labeled examples (1, 5, or 10 per class) across disjoint medical imaging domains with varying resolutions and class distributions. Use when the user wants to benchmark on Komura & Ishikawa (2021), CRC-TP, NCT, LC25000, or asks about evaluating this task. Reports Accuracy(%).Votes: 0GitHub stars: 3
- Few Shot Image Classification EvalEvaluates few-shot image classification models on semantically coherent versus uniformly sampled tasks. It probes the model's ability to generalize from limited support examples to query images across varying class coarseness and scale (5-way vs 100-way). Use when the user wants to benchmark on tieredImageNet, Danish Fungi 2020, or asks about evaluating this task. Reports Top-1 accuracy.Votes: 0GitHub stars: 3
- Few Shot Meta Learning EvalEvaluates few-shot classification performance of meta-learning algorithms on standard image datasets. It specifically probes robustness to distribution shift or difficulty by measuring accuracy on dynamically identified 'hard' episodes versus average episodic performance. Use when the user wants to benchmark on CIFAR-FS, mini-ImageNet, tieredImageNet, or asks about evaluating this task. Reports episodic accuracy.Votes: 0GitHub stars: 3
- Few Shot Ner Doc EvalEvaluates few-shot named entity recognition on document images, measuring how well models identify entity spans with limited training examples. It also probes model robustness to geometric image manipulations like rotation, scaling, and shifting during inference. Use when the user wants to benchmark on FUNSD, CORD, or asks about evaluating this task. Reports word-level F-1 score.Votes: 0GitHub stars: 3
- Few Shot Nlg EvalEvaluates parameter-efficient fine-tuning methods for few-shot natural language generation from structured data (knowledge graphs and semantic representations) to text. It probes the model's ability to adapt to data-scarce regimes while preserving generation fluency and factual alignment with the source structure. Use when the user wants to benchmark on WebNLG 2020, E2E, DART, or asks about evaluating this task. Reports BLEU.Votes: 0GitHub stars: 3
- Few Shot Nlp EvalEvaluates the zero-shot and few-shot capabilities of large language models across a diverse suite of NLP, reasoning, and commonsense benchmarks. It measures how efficiently a model scales with compute and whether additional training objectives unlock emergent reasoning abilities. Use when the user wants to benchmark on GPT-3 suite, BigBench Emergent Suite, Commonsense QA benchmarks, Closed-book QA benchmarks, or asks about evaluating this task. Reports average score.Votes: 0GitHub stars: 3
- Few Shot No Labels EvalEvaluates few-shot image classification capability using a label-free, similarity-based approach. It probes how well self-supervised visual representations can classify novel classes with only a few key images per class, without any training or test labels. Use when the user wants to benchmark on miniImageNet, CIFAR-100FS, FC100, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Few Shot Segmentation EvalEvaluates a model's ability to perform semantic segmentation on 3D point clouds using only a few labeled examples per category. It probes how well learned point embeddings generalize to unseen shapes when supervision is extremely limited, testing both few-shot (few labeled shapes) and few-point (few labeled points per shape) scenarios. Use when the user wants to benchmark on ShapeNet segmentation dataset, or asks about evaluating this task. Reports mIOU.Votes: 0GitHub stars: 3
- Few Shot Ts Gen EvalEvaluates the ability of a generative model to produce high-fidelity time series data under extreme data scarcity (few-shot fine-tuning). It probes cross-domain generalization and robustness to varying sequence lengths and channel dimensions by comparing generated samples against real test data. Use when the user wants to benchmark on ECG200, ETTh2, ETTm1, ETTm2, ILI, Weather, Synthetic sine wave, or asks about evaluating this task. Reports contextFID (c-FID).Votes: 0GitHub stars: 3
- Few Shot Video Classification EvalEvaluates a model's ability to classify video actions from a small number of labeled examples per class. It probes few-shot learning capabilities by measuring accuracy across thousands of randomly sampled episodes on standard video benchmarks. Use when the user wants to benchmark on Kinetics, Something-Something V2, complete-Kinetics, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Fewclue EvalEvaluates Chinese NLP models on few-shot learning across nine tasks, including single-sentence classification, sentence-pair classification, and machine reading comprehension. It tests the ability of pre-trained language models and few-shot prompting/fine-tuning methods to generalize with limited labeled data. Use when the user wants to benchmark on FewCLUE, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Fewjoint EvalEvaluates few-shot joint language understanding by measuring a model's ability to simultaneously predict dialogue intent and extract slots from query sentences using only a small support set from unseen domains. Use when the user wants to benchmark on FewJoint, or asks about evaluating this task. Reports Sentence Accuracy.Votes: 0GitHub stars: 3
- Fewmmbench EvalEvaluates multimodal large language models on few-shot learning capabilities across nine diverse tasks. It probes the models' ability to leverage in-context demonstrations (0, 4, or 8 shots) and chain-of-thought reasoning under controlled retrieval settings, measuring performance relative to zero-shot baselines. Use when the user wants to benchmark on FewMMBench, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Fewshot Fmri Decoding EvalEvaluates few-shot learning methods for decoding brain activation maps from fMRI data. It probes a model's ability to classify cognitive tasks using very limited labeled examples (1 or 5 per class) by randomly sampling novel classes and instances. Use when the user wants to benchmark on IBC dataset, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Fewshot Ft Vs Icl EvalEvaluates the in-domain and out-of-domain generalization capabilities of large language models adapted via few-shot fine-tuning versus in-context learning across standard natural language inference and paraphrase detection benchmarks. Use when the user wants to benchmark on MNLI, RTE, QQP, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Fewstab EvalThis benchmark evaluates the robustness of few-shot image classifiers to spurious class-attribute correlations. It constructs evaluation tasks where support sets contain images with engineered spurious attributes, and query sets lack these attributes or contain attributes from other classes, then measures classification accuracy compared to standard random task sampling. Use when the user wants to benchmark on miniImageNet, tieredImageNet, CUB-200, or asks about evaluating this task. Reports ...Votes: 0GitHub stars: 3
- Feynman Sr EvalEvaluates the ability of genetic programming systems to discover exact symbolic mathematical expressions from numerical data points. It probes search efficiency, robustness to domain constraints (e.g., NaNs), and the impact of different fitness functions on expression discovery. Use when the user wants to benchmark on Feynman dataset, or asks about evaluating this task. Reports typically_solved.Votes: 0GitHub stars: 3
- Ffb EvalEvaluates in-processing group fairness methods by measuring the trade-off between model utility (acc) and various fairness metrics across multiple datasets and hyperparameter settings. Use when the user wants to benchmark on Adult, or asks about evaluating this task. Reports acc.Votes: 0GitHub stars: 3
- Ffdl Platform OverheadEvaluates the runtime performance overhead and scalability of a containerized deep learning platform (FfDL) compared to bare-metal and specialized hardware, using standard image classification benchmarks. It measures how much throughput degrades when running DL training jobs in a Kubernetes-based multi-tenant environment versus direct execution. Use when the user has predictions and gold and needs to compute performance_overhead_pct.Votes: 0GitHub stars: 3
- Fft Harmlessness EvalEvaluates large language models' harmlessness across three dimensions: factuality (resistance to misinformation and counterfactuals), fairness (prediction disparity across demographic groups), and toxicity (generation of harmful content when prompted with jailbreaks). Use when the user wants to benchmark on FFT, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Fg Cxr EvalEvaluates a model's ability to generate accurate, clinically correct chest X-ray reports while aligning its visual focus with radiologist gaze patterns. It probes both natural language generation quality and interpretable attention prediction to ensure diagnostic reasoning is visually grounded. Use when the user wants to benchmark on FG-CXR, or asks about evaluating this task. Reports C, F1_ex, fwIoU.Votes: 0GitHub stars: 3
- Fg Mae Transfer EvalEvaluates the transfer learning capability of a feature-guided masked autoencoder on remote sensing imagery. It probes the model's ability to adapt to downstream scene classification and semantic segmentation tasks across multispectral and SAR modalities using linear probing and fine-tuning protocols. Use when the user wants to benchmark on BigEarthNet-MM, BigEarthNet-SAR, EuroSAT, EuroSAT-SAR, DFC2020, or asks about evaluating this task. Reports mAP.Votes: 0GitHub stars: 3
- Fgbench EvalTests functional-group reasoning capability by asking models to predict how molecular properties change when specific functional groups are added, removed, or modified at given positions. Use when the user wants to benchmark on FGBench, or asks about evaluating this task. Reports Accuracy (Acc).Votes: 0GitHub stars: 3
- Fgn Weather Forecasting EvalEvaluates a functional generative network for medium-range probabilistic weather forecasting against operational ground truth (HRES-fc0) and a diffusion-based baseline (GenCast). Probes the model's ability to capture joint spatial structures and predict tropical cyclone tracks using deterministic and probabilistic scoring rules. Use when the user wants to benchmark on HRES-fc0, ERA5, or asks about evaluating this task. Reports probabilistic metrics.Votes: 0GitHub stars: 3
- Fgner EvalEvaluates fine-grained named entity recognition (FgNER) capabilities across multiple languages by measuring how accurately models identify and classify specific entity types within text sequences. The protocol assesses sequence labeling performance using standard span-based metrics. Use when the user wants to benchmark on Various NER datasets (cited in text), or asks about evaluating this task. Reports F1-score.Votes: 0GitHub stars: 3
- Fgnn Sbr EvalEvaluates session-based recommendation models on e-commerce clickstream data by predicting the next item in a session using graph neural networks and cross-session information. Use when the user wants to benchmark on Yoochoose1/64, Yoochoose1/4, Diginetica, or asks about evaluating this task. Reports R@20, MRR@20.Votes: 0GitHub stars: 3