All authors
Claude Skills by qhjqhj00
github.com/qhjqhj007,572 skills29 installs10,436 views
- Fairjob EvalEvaluates the fairness and predictive performance of job recommendation models on a real-world tabular dataset. It probes whether models exhibit disparate impact across gender groups when predicting senior job opportunities, while accounting for selection bias and privacy constraints inherent in online advertising systems. Use when the user wants to benchmark on FairJob, or asks about evaluating this task. Reports Demographic Parity (DP).Votes: 0GitHub stars: 3
- Fairmedqa EvalEvaluates large language models for demographic bias in medical question answering by measuring performance disparities across counterfactual clinical vignettes that systematically vary race, sex, and socioeconomic status while preserving clinical outcomes. Use when the user wants to benchmark on FairMedQA, or asks about evaluating this task. Reports accuracy disparity (AD).Votes: 0GitHub stars: 3
- Fairmt Bench EvalEvaluates the fairness and bias resistance of conversational LLMs in multi-turn dialogue settings. It probes whether models accumulate stereotypes or toxic content across turns, handle implicit bias in context, and maintain safety under various interaction patterns like jailbreaks or misinformation. Use when the user wants to benchmark on FairMT-10K, or asks about evaluating this task. Reports bias ratio.Votes: 0GitHub stars: 3
- Fairness Algorithms EvalThis evaluation probes the trade-off between predictive performance and group fairness across various machine learning pipelines. It systematically compares fairness-unaware baselines against preprocessing and in-training fairness interventions, measuring how well algorithms maintain accuracy while satisfying demographic parity and equalized odds constraints. Use when the user wants to benchmark on Titanic, German, Adult, S-D, S-P, I-D, or asks about evaluating this task. Reports Fair Efficie...Votes: 0GitHub stars: 3
- Fairness And Downstream EvalEvaluates demographic fairness and downstream NLU task performance of language models. It measures bias across gender, race, and age using established fairness benchmarks, and verifies that fairness interventions do not degrade accuracy on standard classification and regression tasks. Use when the user wants to benchmark on HolisticBias, WEAT/SEAT, CrowS-Pairs, GLUE, or asks about evaluating this task. Reports Fairscore.Votes: 0GitHub stars: 3
- Fairness Aware Automl EvalThis evaluation probes an AutoML framework's ability to jointly optimize predictive performance and fairness constraints during pipeline search. It measures how well a multi-criteria genetic algorithm balances accuracy metrics against demographic parity, equalised odds, and ABROCA while simultaneously selecting data and features. Use when the user wants to benchmark on adult, credit-card, portuguese-bank-marketing, or asks about evaluating this task. Reports DP.Votes: 0GitHub stars: 3
- Fairness Aware Gnn EvalEvaluates the trade-off between prediction accuracy and statistical fairness for graph neural networks on node classification tasks. It probes how different in-processing and preprocessing methods, backbone architectures, and early stopping conditions affect both standard performance metrics and fairness constraints across synthetic, social, and knowledge graph datasets. Use when the user wants to benchmark on Credit, Bail, Pokec-n, Pokec-z, Pokec-n-Large, Pokec-z-Large, DBpedia, YAGO, Wikida...Votes: 0GitHub stars: 3
- Fairness Aware Graph Learning EvalThis benchmark evaluates the trade-offs between predictive utility and fairness across various graph learning algorithms. It probes how well different methods balance accuracy with demographic parity and equal opportunity constraints on real-world graph-structured data. Use when the user wants to benchmark on 7 real-world datasets, or asks about evaluating this task. Reports Δ_SP.Votes: 0GitHub stars: 3
- Fairness Comparison EvalEvaluates the comparative performance of fairness-enhancing machine learning interventions across multiple datasets. It probes how different algorithmic strategies trade off predictive accuracy against a comprehensive set of fairness metrics under standardized preprocessing and fixed train-test splits. Use when the user wants to benchmark on Standardized benchmark datasets (unspecified in excerpt), or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Fairness Explanation EvalThis benchmark evaluates the faithfulness and utility of counterfactual explanations in recommendation systems. It measures how effectively generated explanations identify fairness-disparaging attributes by iteratively erasing them and observing the resulting impact on recommendation accuracy and item exposure inequality. Use when the user wants to benchmark on Yelp, Douban Movie, Last-FM, or asks about evaluating this task. Reports NDCG@K.Votes: 0GitHub stars: 3
- Fairness Label Noise EvalThis evaluation probes the ability of label noise correction methods to mitigate group-dependent label noise while preserving predictive performance and improving algorithmic fairness. It measures how well pre-processing techniques remove underlying discrimination from training data before classifier training. Use when the user wants to benchmark on OpenML (9 datasets), or asks about evaluating this task. Reports AUC.Votes: 0GitHub stars: 3
- Fairness Recourse Subgroup EvalEvaluates the fairness of algorithmic recourse across demographic subgroups by ranking them according to various counterfactual-based fairness definitions. It probes whether different recourse fairness metrics capture distinct aspects of bias, actionability constraints, and subgroup granularity in real-world decision systems. Use when the user wants to benchmark on Adult, or asks about evaluating this task. Reports unfairness_score.Votes: 0GitHub stars: 3
- Fairness Repair EvalEvaluates the ability of an AutoML-based fairness repair framework to mitigate bias in machine learning models while preserving predictive accuracy. It measures the trade-off between accuracy retention and bias reduction across multiple binary classification datasets and model architectures. Use when the user wants to benchmark on Adult Census (race), Bank Marketing (age), German Credit (sex), Titanic (sex), or asks about evaluating this task. Reports Accuracy difference.Votes: 0GitHub stars: 3
- Fairness Sequential EvalEvaluates sequential decision policies under simulated historical and measurement bias to measure how accounting for unrealized outcomes affects fairness disparities and cumulative utility. It probes whether uncertainty-aware exploration mitigates selection rate differences and false positive rate parity violations without sacrificing profit. Use when the user wants to benchmark on Synthetic Sequential Simulation, or asks about evaluating this task. Reports selection rate difference.Votes: 0GitHub stars: 3
- Fairpfn Causal Fairness EvalEvaluates a model's ability to remove the causal influence of protected attributes from predictions while maintaining predictive accuracy. It probes counterfactual fairness and causal effect removal on both synthetically generated causal graphs and real-world tabular datasets. Use when the user wants to benchmark on Synthetic Causal Case Studies, Law School Admissions, Adult Census Income, or asks about evaluating this task. Reports ATE.Votes: 0GitHub stars: 3
- FairpfnevalEvaluates a model's ability to mitigate the causal and counterfactual effects of protected attributes on predictions while maintaining predictive accuracy, without requiring explicit causal graph knowledge. Use when the user wants to benchmark on Synthetic Causal Case Studies, Law School Admissions, Adult Census Income, or asks about evaluating this task. Reports Total Causal Effect (TCE/TeE).Votes: 0GitHub stars: 3
- Fairprep EvalThis evaluation benchmarks pre-processing group fairness techniques on tabular datasets by measuring their impact on fairness metrics and downstream model performance. It probes whether bias mitigation methods improve group-level parity without significantly degrading predictive utility across varying decision thresholds. Use when the user wants to benchmark on Adult, COMPAS, German, or asks about evaluating this task. Reports disparate impact.Votes: 0GitHub stars: 3
- Fairx Gcig EvalEvaluates whether a model maintains predictive utility while achieving procedural fairness (explanation invariance across protected groups) and outcome fairness. It probes the alignment between equalized odds and group-level feature attribution consistency. Use when the user wants to benchmark on Adult, German Credit, COMPAS, Bank Marketing, or asks about evaluating this task. Reports GCIG.Votes: 0GitHub stars: 3
- Faithfulness EvalEvaluates the faithfulness and causal alignment of chain-of-thought reasoning in LLMs by measuring how much the final answer depends on the generated reasoning steps versus the original question. It uses causal mediation analysis to compute indirect and direct effects, and a simulator-based metric to quantify rationale faithfulness. Use when the user wants to benchmark on StrategyQA, GSM8K, Causal Understanding, Quarel, OpenBookQA, QASC, or asks about evaluating this task. Reports Controlled ...Votes: 0GitHub stars: 3
- Fake News OccurrenceAssesses whether multilingual LLM responses contain misinformation or fake content when queried on specific topics. The evaluation relies on an automated GPT-based judge to determine the presence of fake information in model generations. Use when the user has predictions and gold and needs to compute fake news occurrence.Votes: 0GitHub stars: 3
- Fake Video Detection EvalEvaluates the generalization of existing fake image detection models to synthetic videos generated by diffusion models. Probes whether static image-based forgery detectors can identify AI-generated video content when reduced to single frames. Use when the user wants to benchmark on VidProM, DVSC2023, or asks about evaluating this task. Reports Accuracy.Votes: 0GitHub stars: 3
- Fake Voice Detection EvalEvaluates the robustness and cross-domain generalization of fake voice detectors against 17 state-of-the-art generators (TTS, voice conversion, audio reconstruction) using a one-to-one protocol. It quantifies both generator quality and detector effectiveness through composite scores to expose method-specific vulnerabilities that aggregated benchmarks typically mask. Use when the user wants to benchmark on LibriSpeech (test-clean), ASVspoof-21LA, ASVspoof-21DF, ASVspoof-5, Fake or Real (FoR), ...Votes: 0GitHub stars: 3
- Fakebench EvalEvaluates large multimodal models on explainable fake image detection across closed-ended classification and open-ended reasoning tasks. It probes the models' ability to accurately classify image authenticity and generate evidence-based, interpretable justifications using visual and textual forensic cues. Use when the user wants to benchmark on FakeBench, or asks about evaluating this task. Reports Accuracy (ACC).Votes: 0GitHub stars: 3
- Fakeclue Loki EvalEvaluates large multimodal models on synthetic image detection and artifact explanation. It probes the model's ability to classify images as real or fake and generate natural language explanations for specific visual artifacts. Use when the user wants to benchmark on FakeClue, LOKI, or asks about evaluating this task. Reports Acc.Votes: 0GitHub stars: 3
- Falcon EvalEvaluates the efficiency and accuracy of homomorphically encrypted convolution operations and end-to-end private inference networks. It measures communication overhead, inference latency, and classification accuracy under simulated WAN/LAN bandwidths and varying polynomial degrees. Use when the user wants to benchmark on CIFAR-10, CIFAR-100, Tiny ImageNet, or asks about evaluating this task. Reports latency.Votes: 0GitHub stars: 3
- Falserject EvalEvaluates an LLM's tendency to over-refuse benign prompts that merely appear harmful. It probes the model's ability to distinguish safe from unsafe contexts in controversial queries and provide helpful, context-aware responses instead of unnecessary refusals. Use when the user wants to benchmark on FalseReject, or asks about evaluating this task. Reports over-refusal.Votes: 0GitHub stars: 3
- Famma EvalEvaluates multimodal large language models on financial domain reasoning, including calculation-heavy arithmetic problems and knowledge-intensive non-arithmetic questions. It probes cross-lingual capabilities and robustness to data contamination by testing on both textbook-derived and expert-crafted live questions. Use when the user wants to benchmark on FAMMA-Basic, FAMMA-LivePro, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Famteb EvalEvaluates the effectiveness of text embedding models across seven diverse tasks (classification, clustering, pair classification, reranking, retrieval, semantic textual similarity, and summary retrieval) specifically for the Persian language. Use when the user wants to benchmark on FaMTEB, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Fanar20 Benchmarks EvalEvaluates bilingual language understanding, reasoning, and instruction-following capabilities of generative AI models. It probes general knowledge, commonsense reasoning, and culturally aligned Arabic comprehension across multiple standard and custom benchmarks. Use when the user wants to benchmark on English Benchmarks (MMLU, HellaSwag, ARC-Challenge, PIQA, Winogrande), OALL v1, or asks about evaluating this task. Reports English Avg., Arabic Avg..Votes: 0GitHub stars: 3
- Fanaticpythoner Bertscore With Torch DtypeCompute FanaticPythoner/bertscore-with-torch_dtype via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of FanaticPythoner/bertscore-with-torch_dtype.Votes: 0GitHub stars: 3
- Fanns Benchmark EvalEvaluates the accuracy and efficiency of filtered approximate nearest neighbor search (FANNS) algorithms on high-dimensional transformer-based embeddings. It measures how well different indexing methods maintain recall under various real-world attribute filtering constraints while scaling to millions of vectors. Use when the user wants to benchmark on arxiv-for-fanns-medium, arxiv-for-fanns-large, or asks about evaluating this task. Reports recall@10.Votes: 0GitHub stars: 3
- Fanstore Io EvalEvaluates the I/O throughput and bandwidth of a distributed runtime file system (FanStore) across varying node counts and file sizes, comparing it against local SSDs, FUSE, and shared file systems like Lustre. Use when the user wants to benchmark on ImageNet-1k, SRGAN, FRNN, Custom Synthetic Benchmark, or asks about evaluating this task. Reports bandwidth (MB/s), throughput (files/s).Votes: 0GitHub stars: 3
- Far3det EvalEvaluates 3D object detection performance specifically in the far-field range (50-80m), highlighting the limitations of fixed-threshold metrics and sparse lidar data. It probes how well models detect distant objects using lidar, RGB, or fused modalities under adaptive distance-aware tolerance thresholds. Use when the user wants to benchmark on nuScenes, Far nuScenes, or asks about evaluating this task. Reports 3D mAP.Votes: 0GitHub stars: 3
- Fara 7b Agentic EvalThis evaluation probes the agentic capabilities of computer-use models by measuring their ability to complete multi-step web browsing and task-completion tasks on live websites. It assesses both functional success rates and operational efficiency, including token usage, cost, and interaction length. Use when the user wants to benchmark on WebVoyager, Online-Mind2Web, DeepShop, WebTailBench, ScreenSpot, or asks about evaluating this task. Reports success rate.Votes: 0GitHub stars: 3
- Fashion Compatibility EvalEvaluates a model's ability to score the visual-semantic compatibility of items within a complete outfit, and to recommend a missing item that best completes a partial outfit. It probes the model's capacity to learn non-transitive, type-aware relationships across different fashion categories. Use when the user wants to benchmark on Maryland Polyvore, Polyvore Outfits, Polyvore Outfits-D, or asks about evaluating this task. Reports AUC.Votes: 0GitHub stars: 3
- Fashion Ner El EvalEvaluates a BERT-based Named Entity Recognition pipeline and a binary classifier for candidate entity disambiguation on fashion product descriptions. It probes the model's ability to extract attribute mentions (e.g., material, color) and correctly link them to a knowledge graph ontology under severe data scarcity. Use when the user wants to benchmark on Fashion Product Descriptions (In-house), Fashion EL Disambiguation Dataset, or asks about evaluating this task. Reports f1-score.Votes: 0GitHub stars: 3
- Fashionpedia EvalEvaluates joint instance segmentation and fine-grained attribute localization on fashion apparel. It measures how well a model can detect objects, segment them accurately, and correctly assign multiple localized attributes to each instance. Use when the user wants to benchmark on Fashionpedia, or asks about evaluating this task. Reports AP_{IoU + F_1}.Votes: 0GitHub stars: 3
- Fast Gshare EvalEvaluates the performance of a spatio-temporal GPU sharing architecture for serverless deep learning inference. It measures how well the system manages resource multiplexing, isolation, and auto-scaling under varying workloads and allocation configurations. Use when the user wants to benchmark on MLPerf, or asks about evaluating this task. Reports throughput.Votes: 0GitHub stars: 3
- Fast Vgan Vc EvalEvaluates a GAN-based voice conversion model's ability to transfer speaker timbre while explicitly controlling prosodic features like F0 and duration. It tests static prosodic manipulation, dynamic expressive transfer without expressive training data, and real-time inference efficiency. Use when the user wants to benchmark on VCTK Corpus, Expresso Dataset, or asks about evaluating this task. Reports timbre transfer quality.Votes: 0GitHub stars: 3
- Fastat Benchmark EvalThis benchmark evaluates the adversarial robustness and computational efficiency of Fast Adversarial Training (FastAT) methods. It measures how well models maintain accuracy under strong adversarial attacks (PGD, AutoAttack, CR Attack) while tracking training time and memory usage, ensuring fair comparison by controlling architecture, training settings, and data sources. Use when the user wants to benchmark on CIFAR-10, CIFAR-100, Tiny-ImageNet, or asks about evaluating this task. Reports Aut...Votes: 0GitHub stars: 3
- Fastfact EvalEvaluates the long-form factuality of LLM-generated responses by extracting claims, verifying them against evidence, and comparing the system's factuality scores against human annotations. Use when the user wants to benchmark on FaStfact-Bench, or asks about evaluating this task. Reports F₁@K′.Votes: 0GitHub stars: 3
- Fastlongspeech EvalThis evaluation protocol assesses the ability of Large Speech-Language Models to process and understand both short and long-form audio inputs across multiple tasks. It specifically probes speech comprehension, spoken question answering, dialogue understanding, emotion recognition, automatic speech recognition, and long-speech information retrieval under varying compression ratios. Use when the user wants to benchmark on LongSpeech-Eval, speech_QA_iemocap (AIR-Bench), LibriSQA, LibriTTS (OpenA...Votes: 0GitHub stars: 3
- Fasttrack EvalEvaluates multi-object tracking performance in highly crowded, complex urban traffic environments. It probes a tracker's ability to maintain identity consistency under severe occlusions, varying lighting conditions, and across diverse object classes using motion and structural cues rather than appearance models. Use when the user wants to benchmark on FastTrack, or asks about evaluating this task. Reports HOTA.Votes: 0GitHub stars: 3
- Fastturn Turn Detection EvalEvaluates a full-duplex turn detection model's ability to classify speech turn states (complete, incomplete, backchannel, wait) in real-time streaming conditions, measuring robustness to acoustic ambiguity, noise, and speech overlap. Use when the user wants to benchmark on FastTurn test set, Easy Turn, Smart Turn, or asks about evaluating this task. Reports Accuracy.Votes: 0GitHub stars: 3
- Faviq EvalEvaluates a model's ability to verify factual claims against retrieved evidence, specifically focusing on claims derived from ambiguous information-seeking questions. It also measures the effectiveness of transfer learning from crowdsourced fact-checking data to professional fact-checking benchmarks. Use when the user wants to benchmark on FaVIQ, Snopes, SciFACT, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Fbeta ScoreCompute the fbeta_score metric — provided by scikit-learn. Use when the user has predictions and ground-truth and needs to compute fbeta_score, or asks how to score with fbeta_score.Votes: 0GitHub stars: 3
- FbetascoreCompute the FBetaScore metric — provided by torchmetrics. Use when the user has predictions and ground-truth and needs to compute FBetaScore, or asks how to score with FBetaScore.Votes: 0GitHub stars: 3
- Fch Gradient Flow Bench EvalEvaluates the accuracy and stability of numerical schemes for solving the functionalized Cahn-Hilliard gradient flow under varying morphological complexity regimes. It tests how well different time-stepping methods (IMEX, SAV, PSD, ETD) capture defect formation, pearling, and interface dynamics in stiff, nonlinear PDE regimes. Use when the user wants to benchmark on FCH Gradient Flow Benchmarks, or asks about evaluating this task. Reports L2 relative error.Votes: 0GitHub stars: 3
- Fcmbench EvalEvaluates vision-language models on financial credit document understanding, covering perception tasks like document type recognition and key information extraction, as well as reasoning tasks such as validity checking and numerical calculation under strict low-latency constraints. Use when the user wants to benchmark on FCMBench, or asks about evaluating this task. Reports F1 score.Votes: 0GitHub stars: 3
- Fcn Ctr EvalEvaluates the effectiveness, efficiency, and interpretability of explicit feature interaction models for click-through rate (CTR) prediction on large-scale, highly sparse advertising and recommendation datasets. Use when the user wants to benchmark on Avazu, Criteo, ML-1M, KDD12, iPinYou, KKBox, or asks about evaluating this task. Reports AUC.Votes: 0GitHub stars: 3