Research

Evaluates a model's ability to predict the next item in a user session based on sequential click or watch history. It probes the capability to capture short-term sequential dependencies and maintain context within session-based recommendation scenarios. Use when the user wants to benchmark on RSC15, VIDEO, or asks about evaluating this task. Reports recall@20.

This evaluation probes a model's ability to perform personalized session-based sequential recommendation by predicting the next item a user will interact with. It measures how well the model leverages both intra-session behavior and cross-session user history to rank relevant items in a top-5 list. Use when the user wants to benchmark on XING, VIDEO, or asks about evaluating this task. Reports MRR@5.

This evaluation probes a model's ability to predict the next item in a user session based on historical interactions. It measures ranking quality across multiple benchmark datasets, testing how well the model captures temporal patterns and prospective user preferences without relying on fixed recency heuristics. Use when the user wants to benchmark on Six session-based recommendation benchmarks (DG, GA, YC, TM, LF, NP), or asks about evaluating this task. Reports recall@k, MRR@k, NDCG@k.

Evaluates a model's ability to predict the next item in a user's shopping session based on sequential item interactions and cross-session collaborative signals. It probes how well the model captures dynamic user interests and leverages historical session data for accurate recommendations. Use when the user wants to benchmark on Diginetica, Tmall, Yoochoose1_64, or asks about evaluating this task. Reports P@20.

Evaluates the predictive performance of session-based and session-aware recommendation models in ranking the next item a user will interact with. It benchmarks both neural and non-neural approaches across multiple real-world interaction datasets to assess accuracy, coverage, and popularity bias. Use when the user wants to benchmark on RETAIL, XING, COSMETICS, LASTFM, or asks about evaluating this task. Reports MAP@20.

Evaluates the commercial viability and economic performance of text-to-image and image editing models in real-world design workflows. It measures how well generated images meet baseline requirements, visual quality standards, and commercial intent, linking outputs directly to human payment decisions and platform revenue. Use when the user wants to benchmark on ServImage, or asks about evaluating this task. Reports Task Acceptance (%).

Evaluates the accuracy and robustness of Sérsic profile fitting algorithms (GIM2D and GALFIT) on simulated HST/ACS galaxy images. It probes how well these codes recover true structural parameters under varying signal-to-noise ratios and surface brightness levels. Use when the user wants to benchmark on GEMS Bulge0001, GEMS Disk0001, or asks about evaluating this task. Reports magnitude_residual.

Evaluates deep learning models for speech emotion recognition (SER) using a multi-window data augmentation strategy. It probes the model's ability to classify categorical emotions from speech audio under varying feature extraction window sizes and class configurations. Use when the user wants to benchmark on IEMOCAP, RAVDESS, SAVEE, or asks about evaluating this task. Reports Unweighted Accuracy (UA).

Evaluates pre-trained speech models' ability to recognize emotions in audio across multiple languages. It specifically tests how internal layer representations and feature aggregation strategies impact classification performance. Use when the user wants to benchmark on AESDD, CaFE, EmoDB, EMOVO, IEMOCAP, RAVDESS, ShEMO, or asks about evaluating this task. Reports accuracy.

Evaluates the capability of speech emotion recognition models to classify emotional states from audio recordings using spectral features and attention mechanisms. The protocol measures classification performance across multiple standard SER benchmarks to assess robustness and generalization. Use when the user wants to benchmark on SAVEE, RAVDESS, CREMA-D, TESS, EMO-DB, EMOVO, or asks about evaluating this task. Reports accuracy.

Evaluates the performance and reproducibility of sequential recommender system (SRS) models across multiple user-item interaction datasets. It probes how architectural choices, hyperparameter settings, and training configurations affect ranking metrics and computational emissions. Use when the user wants to benchmark on Beauty, FS-NYC, FS-TKY, ML-100k, ML-1M, ML-20M, or asks about evaluating this task. Reports NDCG@10.

Evaluates a model's ability to predict the next item in a user's sequential interaction history by leveraging both temporal proximity across users and within-user sequence dynamics. Use when the user wants to benchmark on Amazon (Beauty, Book, Video), Steam, or asks about evaluating this task. Reports NDCG@10.

Evaluates the ability of sequential recommendation models to predict the next item in a user's interaction history by distilling semantic user profiles from pre-trained LLMs into the recommender's internal representations. The protocol tests whether knowledge distillation improves recommendation accuracy while maintaining inference efficiency without requiring real-time LLM calls. Use when the user wants to benchmark on Beauty, ML20M, Kion, Amazon M2, or asks about evaluating this task. Repor...

Evaluates a model's ability to predict the next item in a user's sequential interaction history. It probes how well the system captures temporal user preferences and handles discrete recommendation data under a strict chronological split. Use when the user wants to benchmark on Amazon Beauty, Amazon Toys, MovieLens-1M, or asks about evaluating this task. Reports NDCG@K.

Evaluates the stability and performance of sequential knowledge editing methods on large language models over long horizons. It probes whether editing techniques can maintain factual accuracy, preserve general capabilities, and avoid norm blow-up or catastrophic forgetting across thousands of atomic updates. Use when the user wants to benchmark on CounterFact, ZsRE, WikiBigEdit, GLUE-style tasks (SST, MRPC, RTE, CoLA, MNLI), MMLU, or asks about evaluating this task. Reports Efficacy.

Evaluates sequence modeling capabilities, specifically long-term memory retention and contextual understanding across synthetic stress tests and real-world benchmarks. It compares generic temporal convolutional networks against canonical recurrent architectures (LSTM, GRU, RNN) on tasks requiring prediction of sequential data. Use when the user wants to benchmark on Adding problem, Sequential MNIST, P-MNIST, Copy memory, Nottingham, JSB Chorales, PTB, Wikitext-103, LAMBADA, text8, or asks abo...

This evaluation protocol benchmarks sequential recommendation models by comparing sequence-level data augmentation strategies against contrastive learning baselines. It probes a model's ability to capture user intent from interaction sequences and generate accurate item rankings under varying data sparsity, sequence lengths, and cold-start conditions. Use when the user wants to benchmark on Amazon Beauty, Amazon Sports, Yelp, ML-1m, or asks about evaluating this task. Reports Recall@K / NDCG@...

This survey evaluates and categorizes methodologies for sequence-aware recommender systems, focusing on offline evaluation protocols, data partitioning strategies, and ranking metrics used to assess prediction accuracy and list quality. It highlights how temporal dependencies and session boundaries require specialized splitting and target definition compared to traditional matrix completion. Use when the user wants to benchmark on Amazon, RecSys Chall. 2015, Delicious, or asks about evaluatin...

Evaluates deep learning models for forecasting solar energetic particle (SEP) events using solar magnetic field parameters and historical eruptive features. It probes the model's ability to classify general and operational SEP occurrences under different feature sets and temporal conditions. Use when the user wants to benchmark on SEPVAL, CLEAR, or asks about evaluating this task. Reports F1.

Evaluates a sentence-level early intervention framework for reducing object hallucinations in multimodal large language models (MLLMs) while preserving or enhancing general vision-language capabilities across multiple standard benchmarks. Use when the user wants to benchmark on Object HalBench, AMBER, HallusionBench, VQAv2, TextVQA, ScienceQA, MM-Vet, or asks about evaluating this task. Reports response-level hallucination rate (Resp.).

Evaluates a model's ability to jointly classify sentiment (negative, neutral, positive) from healthcare transcripts and generate semantically coherent rationales explaining the classification. It probes multimodal sentiment analysis, explainable AI, and chain-of-thought reasoning in a clinical dialogue setting. Use when the user wants to benchmark on Sentiment Reasoning dataset, or asks about evaluating this task. Reports accuracy.

Evaluates sentiment classification and justification generation capabilities for the low-resource Maithili language. It probes a model's ability to accurately predict sentence-level sentiment labels and generate culturally grounded, linguistically correct explanations in Maithili. Use when the user wants to benchmark on SentiMaithili, or asks about evaluating this task. Reports F1-score.

Evaluates large language models on fine-grained emotion classification across English and Chinese dialogues and social media text. It probes the model's ability to handle complex, multilingual contexts, long sequences, and imbalanced emotion categories using a self-analytical negotiation mechanism. Use when the user wants to benchmark on MELD, EmoryNLP, IEMOCAP, CPED, CH-SIMS, Twitter2015, Twitter2017, or asks about evaluating this task. Reports Accuracy.

Evaluates the transferability and quality of universal sentence embeddings across a standardized suite of downstream tasks. It probes capabilities in sentiment classification, natural language inference, semantic textual similarity, and cross-modal image-caption retrieval using fixed hyperparameters and consistent preprocessing. Use when the user wants to benchmark on MR, CR, SUBJ, MPQA, TREC, SST-2, SST-5, SNLI, SICK-E, SICK-R, STS14, MRPC, COCO, or asks about evaluating this task. Reports a...