All authors
Claude Skills by qhjqhj00
github.com/qhjqhj007,572 skills29 installs10,436 views
- Transz Sbert CosineCompute transZ/sbert_cosine via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of transZ/sbert_cosine.Votes: 0GitHub stars: 3
- Transz Test ParascoreCompute transZ/test_parascore via the HuggingFace `evaluate` library. Use when the user has predictions + references and wants the canonical implementation of transZ/test_parascore.Votes: 0GitHub stars: 3
- Trap EvalThis benchmark evaluates the vulnerability of web agents to prompt injection attacks that redirect their intended tasks. It probes how well agents maintain task fidelity under benign conditions versus how susceptible they are to social-engineering and persuasion-based adversarial injections embedded in web interfaces. Use when the user wants to benchmark on TRAP, or asks about evaluating this task. Reports Attack Success Rate (ASR).Votes: 0GitHub stars: 3
- Traveler Temporal Reasoning EvalThis benchmark evaluates large language models' ability to perform event-temporal reasoning by resolving explicit, implicit, and vague temporal references across synthetic household event chains. It systematically probes how model performance degrades with increasing event set length and varying levels of temporal explicitness. Use when the user wants to benchmark on TRAVELER, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Travelfraudbench EvalEvaluates graph neural networks and tabular baselines on detecting fraudulent user rings in travel booking networks. It probes the models' ability to classify individual fraud accounts and recover entire fraud ring structures using heterogeneous graph topology and co-occurrence signals. Use when the user wants to benchmark on TravelFraudBench (TFG), or asks about evaluating this task. Reports AUC-ROC.Votes: 0GitHub stars: 3
- Travelplanner EvalEvaluates language agents' ability to perform long-horizon, multi-constraint real-world travel planning. It probes their capacity for dynamic tool use, constraint tracking, commonsense reasoning, and maintaining task coherence across complex decision-making steps. Use when the user wants to benchmark on TravelPlanner, or asks about evaluating this task. Reports macro pass rate.Votes: 0GitHub stars: 3
- Treb EvalEvaluates large language models' table reasoning capabilities across six core skills: natural language understanding, table understanding, basic/computational operations, data analysis, and advanced data analysis. It tests both direct textual reasoning and code-based reasoning (Program-of-Thought and Interleaved CoT) for extracting and processing tabular information. Use when the user wants to benchmark on TReB, or asks about evaluating this task. Reports ROUGE-L.Votes: 0GitHub stars: 3
- Trec 2019 Dl Track EvalEvaluates ad-hoc information retrieval systems on document and passage ranking tasks using large-scale human-labeled judgments. It probes the ability of neural and traditional models to rank relevant items highly for a set of test queries. Use when the user wants to benchmark on TREC 2019 Deep Learning Track, or asks about evaluating this task. Reports NDCG@10.Votes: 0GitHub stars: 3
- Trec 2020 Podcast Summarisation EvalEvaluates the ability of summarization systems to generate concise, accurate, and informative summaries of long-form spoken podcast episodes. It probes handling of speech-specific challenges like redundancy, speaker turns, and informal language, as well as factual recall of key entities and events. Use when the user wants to benchmark on TREC 2020 Podcast Summarisation Track, or asks about evaluating this task. Reports Avg.Votes: 0GitHub stars: 3
- Trec 2021 Dl EvalEvaluates the ability of ranking models to retrieve and order relevant passages or documents for given queries. It probes multi-stage retrieval pipelines, including dense/sparse retrieval, query expansion, and re-ranking capabilities. Use when the user wants to benchmark on TREC 2021 Deep Learning Track, or asks about evaluating this task. Reports NDCG@5.Votes: 0GitHub stars: 3
- Trec Cast 2019 EvalEvaluates a system's ability to perform multi-turn conversational information retrieval by selecting relevant passages for user utterances while leveraging prior dialogue history, including handling coreference, omissions, and topic shifts. Use when the user wants to benchmark on TREC CAsT 2019, or asks about evaluating this task. Reports NDCG@3.Votes: 0GitHub stars: 3
- Trec Cast21 EvalEvaluates conversational search retrieval pipelines by reproducing baseline and top-performing systems from the TREC CAsT 2021 track. It probes the effectiveness of query rewriting, sparse/dense retrieval fusion, and re-ranking strategies in multi-turn conversational settings. Use when the user wants to benchmark on TREC CAsT 2021, or asks about evaluating this task. Reports NDCG@3.Votes: 0GitHub stars: 3
- Trec Covid EvalEvaluates information retrieval systems on pandemic-related queries using a dynamically evolving corpus. It probes a system's ability to retrieve relevant medical literature under real-world conditions where terminology and document availability change rapidly. Use when the user wants to benchmark on TREC-COVID Round 1, or asks about evaluating this task. Reports NDCG@10.Votes: 0GitHub stars: 3
- Trec Data Fusion EvalEvaluates the effectiveness of linear combination data fusion methods for information retrieval when trained on partial relevance judgments. It probes whether multiple linear regression can learn near-optimal fusion weights using only 20% or 50% of relevant documents instead of full official qrels. Use when the user wants to benchmark on TREC 2018-2021 Precision Medicine & Deep Learning Tracks, or asks about evaluating this task. Reports MAP.Votes: 0GitHub stars: 3
- Trec Dl 2019 EvalEvaluates document ranking models on a small set of test queries from the TREC 2019 Deep Learning Track. It probes how effectively models trained on large-scale clicked query-document pairs can rerank documents according to human relevance judgments. Use when the user wants to benchmark on TREC 2019 Deep Learning Track, or asks about evaluating this task. Reports MRR.Votes: 0GitHub stars: 3
- Trec Dl EvalEvaluates document retrieval models on a large-scale corpus by ranking millions of documents against a set of queries. It probes the model's ability to perform full-document retrieval and produce accurate ranked lists using explicit and latent matching signals. Use when the user wants to benchmark on TREC Deep Learning track (MS MARCO), or asks about evaluating this task. Reports MRR.Votes: 0GitHub stars: 3
- Trec Dl Hole Filling EvalEvaluates the ability of LLMs to accurately predict missing relevance judgments (holes) in information retrieval test collections. It measures how well synthetic incomplete judgments can be restored to match ground truth relevance labels across varying hole percentages. Use when the user wants to benchmark on TREC DL 2019/2020/2021, or asks about evaluating this task. Reports Kendall τ.Votes: 0GitHub stars: 3
- Trec Dl Passage Relevance EvalThis benchmark evaluates LLMs on their ability to assign relevance scores to document passages given a search query, comparing their outputs against human judgements. It specifically probes systematic biases such as query-term injection gullibility and instruction manipulation in information retrieval labelling tasks. Use when the user wants to benchmark on TREC DL21+DL22, or asks about evaluating this task. Reports MAE.Votes: 0GitHub stars: 3
- Trec Dl Track EvalEvaluates the reliability and best practices for using TREC Deep Learning test collections for ranking model evaluation. It probes whether researchers properly separate model selection from final evaluation and accounts for training variance. Use when the user wants to benchmark on TREC Deep Learning Track, or asks about evaluating this task. Reports NDCG@10.Votes: 0GitHub stars: 3
- Trec Dragun EvalEvaluates assistive RAG systems that support news trustworthiness assessment by generating investigative questions and context-rich reports. Probes the model's ability to identify critical aspects of source bias, motivation, and alternative viewpoints, and to synthesize attributed summaries that help readers evaluate credibility. Use when the user wants to benchmark on TREC DRAGUN 2025 Track, or asks about evaluating this task. Reports Kendall's τ.Votes: 0GitHub stars: 3
- Trec Ikat 2023 EvalThis benchmark evaluates a system's ability to perform personalized conversational search by retrieving relevant passages and generating fluent, grounded responses. It specifically probes how well an agent can adapt its output to user-specific context encoded in a Personal Text Knowledge Base (PTKB) while maintaining provenance traceability. Use when the user wants to benchmark on TREC iKAT 2023, ClueWeb22-B Subset, or asks about evaluating this task. Reports groundedness.Votes: 0GitHub stars: 3
- Trec Ir EvalEvaluates information retrieval ranking methods on ad hoc and routing tasks across diverse text collections. It probes how well term weighting schemes capture semantic relevance and handle difficult or verbose queries compared to classic baselines like BM25 and TF-IDF. Use when the user wants to benchmark on TREC Collections and Topics, or asks about evaluating this task. Reports MAP.Votes: 0GitHub stars: 3
- Trec Microblog EvalEvaluates the effectiveness of query expansion methods for real-time microblog search by measuring how well ranked document lists match relevance judgments for short-form social media queries. It specifically probes the model's ability to handle vocabulary mismatch and temporal relevance in noisy, short-text retrieval scenarios. Use when the user wants to benchmark on TREC Microblog Track, or asks about evaluating this task. Reports MAP.Votes: 0GitHub stars: 3
- Trec Rag Support EvalEvaluates the ability of LLM judges versus human annotators to assess sentence-level grounding (support) in RAG-generated answers. It measures how well models cite relevant passages and whether the cited text actually supports the generated claims. Use when the user wants to benchmark on TREC 2024 RAG Track, or asks about evaluating this task. Reports weighted precision.Votes: 0GitHub stars: 3
- Trec Rts MetricsEvaluates real-time tweet summarization systems by measuring their ability to push relevant, non-redundant tweets within fixed temporal windows, while penalizing system latency and irrelevant outputs. Use when the user has predictions and gold and needs to compute Expected Gain (EG).Votes: 0GitHub stars: 3
- Trec Session Tracks EvalEvaluates the effectiveness of a learning-to-rank personalization approach for web search sessions by measuring relevance prediction quality across multiple years of session track data. It probes the model's ability to leverage historical query sequences, document rankings, and user click behavior to improve session-level relevance ranking. Use when the user wants to benchmark on TREC 2011-2014 Session Tracks, or asks about evaluating this task. Reports nDCG@k.Votes: 0GitHub stars: 3
- Trec2020 Deep Learning EvalEvaluates ad hoc information retrieval ranking methods on document and passage retrieval tasks using large-scale training data and human-labeled relevance judgments. It compares neural language models, neural networks, and traditional methods under blind single-shot conditions to assess ranking quality in the top-k results. Use when the user wants to benchmark on TREC 2020 Deep Learning Track, or asks about evaluating this task. Reports NDCG@10.Votes: 0GitHub stars: 3
- Trec2020 Fairness Ranking EvalEvaluates information retrieval and re-ranking systems on their ability to balance document relevance with demographic fairness. It probes how well algorithms maintain ranking utility while ensuring equitable representation across inferred gender and country groups. Use when the user wants to benchmark on TREC 2020 Fairness Ranking Track dataset, or asks about evaluating this task. Reports utility.Votes: 0GitHub stars: 3
- Trec2022 Fair Ranking EvalEvaluates retrieval systems on balancing topical relevance with intersectional fairness in Wikipedia article rankings. It probes static single-query ranking for coordinators and dynamic multi-query ranking for editors under fairness constraints across demographic attributes. Use when the user wants to benchmark on TREC 2022 Fair Ranking Track, or asks about evaluating this task. Reports M1, EE-L.Votes: 0GitHub stars: 3
- Trec2022 Neuclir EvalEvaluates neural cross-language information retrieval systems on ad hoc, reranking, and monolingual tasks across Chinese, Persian, and Russian newswire collections. It measures how well models retrieve and rank relevant documents when queries are in English and documents are in other languages, or when queries are human-translated. Use when the user wants to benchmark on TREC 2022 NeuCLIR Collections, or asks about evaluating this task. Reports nDCG@10.Votes: 0GitHub stars: 3
- Trec2024 Rag Nugget EvalEvaluates the factual accuracy and content grounding of RAG-generated answers by checking for the presence of key factual claims (nuggets) extracted from source documents. It also measures answer length to assess the trade-off between conciseness and completeness in system outputs. Use when the user wants to benchmark on TREC 2024 RAG Track, or asks about evaluating this task. Reports V_strict.Votes: 0GitHub stars: 3
- Trec2025 Rag EvalProbes retrieval-augmented generation systems on complex, narrative-driven queries by decomposing information needs into sub-narratives. It evaluates document relevance based on sub-narrative coverage, measures response quality via strict vital recall of fully supported information nuggets, and assesses sentence-level factual grounding against cited documents. Use when the user wants to benchmark on MS MARCO V2.1, or asks about evaluating this task. Reports strict_vital_recall.Votes: 0GitHub stars: 3
- Tree Vs Sequence EvalThis evaluation protocol compares recursive tree-based neural models against recurrent sequence-based models across multiple NLP tasks. It probes whether syntactic tree structures are necessary for learning representations, particularly for tasks requiring long-distance dependency modeling or hierarchical composition. Use when the user wants to benchmark on Stanford Sentiment Treebank, Pang Sentiment Dataset, UMD-QA, SemEval-2010 Task 8, Discourse Parsing, or asks about evaluating this task. ...Votes: 0GitHub stars: 3
- Treebench EvalEvaluates visual grounded reasoning by requiring models to precisely localize target objects in cluttered scenes and perform second-order reasoning about their interactions. It measures both the correctness of the final answer and the spatial accuracy of the traceable bounding box evidence. Use when the user wants to benchmark on TreeBench, or asks about evaluating this task. Reports Accuracy.Votes: 0GitHub stars: 3
- Treeeval EvalTreeEval probes an LLM's ability to handle complex, adaptive reasoning through dynamically generated hierarchical questions. It evaluates how well a model's relative performance ranking aligns with established leaderboards like AlpacaEval2.0, while testing the framework's efficiency in distinguishing fine-grained capability differences without relying on static datasets. Use when the user wants to benchmark on TreeEval (Dynamic/Benchmark-Free), or asks about evaluating this task. Reports Spea...Votes: 0GitHub stars: 3
- Treereview Peer Review EvalThis benchmark evaluates an LLM's ability to perform deep, structured scientific peer review by generating comprehensive reviews and actionable feedback comments. It probes the model's capacity for hierarchical question decomposition, context-aware analysis of long documents, and alignment with human reviewer judgments across multiple quality dimensions. Use when the user wants to benchmark on TreeReview Benchmark (ICLR-2024, NeurIPS-2023, Nature Communications), or asks about evaluating this...Votes: 0GitHub stars: 3
- Treesatai Ts EvalEvaluates a model's ability to perform fine-grained tree species identification using multimodal Earth observation data, specifically leveraging temporal dynamics from optical and radar time series alongside high-resolution imagery. Use when the user wants to benchmark on TreeSatAI-TS, or asks about evaluating this task. Reports weighted F1.Votes: 0GitHub stars: 3
- Trek 150 EvalEvaluates single-object visual tracking performance in first-person vision videos, specifically testing robustness to object manipulation, occlusions, and dynamic interactions under real-time execution constraints. Use when the user wants to benchmark on TREK-150, or asks about evaluating this task. Reports SS, NPS, GSR.Votes: 0GitHub stars: 3
- Trendfact EvalEvaluates fact-checking systems across three sub-tasks: retrieving relevant evidence, verifying claim truthfulness, and generating natural language explanations. It specifically probes a model's Hotspot Perception Ability (HPA) by measuring how effectively it allocates reasoning effort and computational resources based on the real-world influence of trending claims. Use when the user wants to benchmark on TrendFact, or asks about evaluating this task. Reports F1-macro.Votes: 0GitHub stars: 3
- Trglue EvalThis benchmark evaluates Turkish natural language understanding across multiple task types, including grammaticality judgment, sentiment analysis, paraphrase detection, and semantic textual similarity. It probes a model's ability to handle agglutinative morphology, culturally adapted expressions, and nuanced semantic equivalence in Turkish. Use when the user wants to benchmark on TrCoLA, TrSST-2, TrMRPC, TrSTS-B, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Trilemma Of Truth EvalEvaluates large language models' ability to distinguish factually true statements from factually false and unverifiable ('neither') statements. It probes both prompt-based output probabilities and internal hidden activations to measure veracity classification accuracy and uncertainty quantification. Use when the user wants to benchmark on Trilemma of Truth Datasets, or asks about evaluating this task. Reports MCC.Votes: 0GitHub stars: 3
- Triplesumm Video Summarization EvalEvaluates a model's ability to perform video summarization by predicting frame-level importance scores across visual, textual, and audio modalities. It probes the model's capacity for adaptive multimodal fusion and temporal dependency modeling to identify salient segments in long videos. Use when the user wants to benchmark on MoSu, Mr. HiSum, SumMe, TVSum, or asks about evaluating this task. Reports Kendall’s τ (kTau), Spearman’s ρ (sRho).Votes: 0GitHub stars: 3
- Triviaqa EvalThis benchmark evaluates reading comprehension on complex, compositional trivia questions that require multi-sentence reasoning and handling high lexical variability. It tests a model's ability to locate and extract precise answers from large, noisy evidence documents across different domains. Use when the user wants to benchmark on TriviaQA, or asks about evaluating this task. Reports exact match (EM).Votes: 0GitHub stars: 3
- Truck Driving Risk EvalBinary classification of truck driving risk on specific highway segments based on historical behavior, short-term trip dynamics, and real-time traffic conditions. It probes a model's ability to predict forward collision warning events using a small, highly imbalanced dataset of real-world trajectory data. Use when the user wants to benchmark on Truck Driving Risk Dataset, or asks about evaluating this task. Reports Accuracy.Votes: 0GitHub stars: 3
- Truemicl EvalThis benchmark evaluates a model's ability to perform true multimodal in-context learning by requiring it to solve tasks that depend on both visual and textual information from provided demonstrations. It probes whether models can correctly attend to and utilize visual context in few-shot examples rather than relying on superficial textual patterns or prior knowledge. Use when the user wants to benchmark on TrueMICL, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Trustbench EvalEvaluates the ability of autonomous LLM agents to perform safe, domain-specific actions in real-time by measuring how effectively a trust verification framework reduces harmful actions while maintaining task completion. It probes epistemic calibration, runtime safety intervention, and domain-specific verification reliability. Use when the user wants to benchmark on MedQA, FinQA, TruthfulQA, or asks about evaluating this task. Reports harmful_actions.Votes: 0GitHub stars: 3
- Truthfulqa Biogen Factuality EvalEvaluates an LLM's factual accuracy and hallucination mitigation across multiple-choice, short-form, and long-form generation tasks. It measures the trade-off between truthfulness and informativeness, and quantifies the exact number of supported versus unsupported facts in generated text. Use when the user wants to benchmark on TruthfulQA, BioGEN, or asks about evaluating this task. Reports Accuracy, True*Info, FActScore.Votes: 0GitHub stars: 3
- Truthfulqa EvalEvaluates the factual accuracy and truthfulness of large language models by measuring their ability to select correct answers over common misconceptions. It probes the model's capacity to resist generating plausible but false statements across diverse categories like health, law, and politics. The benchmark specifically tests whether models can identify and output factually correct responses when presented with multiple candidate answers. Use when the user wants to benchmark on TruthfulQA, or...Votes: 0GitHub stars: 3
- Ts Forecasting EvalEvaluates the forecasting accuracy and computational efficiency of deep learning models on multivariate time series data. It probes how architectural choices, preprocessing steps, and spatial-temporal processing configurations impact performance across varying forecasting horizons. Use when the user wants to benchmark on Weather, Solar-Energy, ECL, Traffic, or asks about evaluating this task. Reports MAE.Votes: 0GitHub stars: 3
- Ts Insights EvalEvaluates the ability of large multimodal models to generate accurate, domain-agnostic natural language descriptions of time series trends. It probes cross-modal alignment between visual time series plots (or extracted features) and textual trend explanations. Use when the user wants to benchmark on TS-Insights, or asks about evaluating this task. Reports final_score.Votes: 0GitHub stars: 3