Claude Skills by wenmin-wu

Merges multiple training datasets while keeping all positive examples and downsampling negatives to control class imbalance.

Reranks candidate answer spans by subtracting the CLS token's start+end logit sum as a null-answer baseline score.

Resolves mixed cp1252/utf-8 encoding artifacts in text via round-trip encode/decode with custom error handlers and unidecode normalization.

Merge candidate pools from multiple independent seq2seq models before MBR selection to reduce shared failure modes

Trains a Byte-Pair Encoding tokenizer on the task corpus to capture domain-specific vocabulary, typos, and subword patterns.

Cyclical learning rate (CLR) Keras callback that oscillates LR between base and max each batch for faster convergence

Fine-tunes DeBERTa-v3 for text classification tasks. Use when building text classifiers, sentiment analysis, or multi-label classification on domain-specific data.

Add domain-specific categorical values as new special tokens, resize embeddings, and prepend them to input so the model learns domain-aware representations

Trains a WordPiece tokenizer on in-domain text, then feeds its subword token IDs into TF-IDF vectorization for domain-adapted sparse features.

Explicitly zero all dropout probabilities in transformer config at load time for fully deterministic inference

Concatenate GloVe and FastText embedding matrices along feature axis for richer 600d word representations

Pads each batch to its actual max sequence length instead of the global max_len, reducing wasted computation.

Use pretrained embedding vocab to decide which punctuation to keep, split, or remove before tokenization

Measure pretrained embedding coverage over dataset vocab and return OOV words sorted by frequency for targeted preprocessing

Initialize out-of-vocabulary word embeddings with Gaussian noise matching the pretrained embedding distribution

Use separate learning rates for pretrained backbone (low) and randomly initialized classification head (high)

Uses a fine-tuned masked language model to classify candidate entity spans by comparing fill-mask probabilities for positive vs negative marker tokens.

GPU-accelerated TF-IDF vectorization via RAPIDS cuML with chunked cosine similarity for large-scale text matching

Trades compute for memory by recomputing intermediate activations during backprop instead of storing them, reducing memory from O(n) to O(sqrt(n)).

Preserves both the start and end of long text sequences when truncating for transformer input limits.

Concatenate multi-level categorical fields into a compound label (Category:Subcategory) for flat multiclass classification

Weight BCE loss by identity subgroup membership to debias predictions — upweight samples where identity conflicts with label

Further pretrains a transformer with masked language modeling on the target task's own text before fine-tuning.

Deduplicates extracted entity predictions by filtering out candidates whose Jaccard word-overlap with already-accepted labels exceeds a threshold.

Computes micro F-beta for NER where true positives are determined by Jaccard word-overlap >= threshold rather than exact string match.

Converts Japanese scripts (Hiragana, Katakana, Kanji) to romanized ASCII using pykakasi for cross-script entity matching.

Dual-head transformer model that jointly predicts start/end span logits from sequence output and answer type from pooled CLS output.

Evaluates predicted sequence ordering quality using Kendall Tau correlation via efficient O(n log n) inversion counting.

Replay keystroke activity logs (Input/Replace/Paste/Remove/Move) against a string buffer to reconstruct the evolving essay text

Applies label smoothing to binary cross-entropy loss to reduce overconfidence and improve generalization in text classification.

Concatenate the last two transformer hidden states along the feature dimension before the task head for richer token representations

Applies different learning rates to transformer encoder vs task-specific head, with no weight decay on bias and LayerNorm.

Latent Dirichlet Allocation on CountVectorizer bag-of-words to discover latent topics with per-document topic distributions for feature engineering or EDA

Replace mean pooling with a trainable attention network (Linear-Tanh-Linear-Softmax) that learns token importance weights over transformer hidden states

Sort texts by length before batching with dynamic padding to minimize wasted padding tokens and speed up transformer inference

Feeds CLS token embeddings from each transformer layer into a BiLSTM to learn an optimal combination across layer depth.

Custom MAP@K evaluation metric — scores top-K classification predictions with reciprocal rank weighting for HuggingFace Trainer

Binary cross-entropy loss with mask to ignore special and padding tokens in token-level span classification

Minimum Bayes Risk decoding — select the candidate with highest average chrF++ agreement against all others in the pool

Concatenates token-level mean pooling and max pooling from the last hidden state for a richer sequence representation.

Computes attention-mask-weighted mean of token embeddings, excluding padding tokens from the average.

Min-max normalizes each model's predictions to [0,1] before averaging, ensuring equal contribution regardless of score distribution scale.

Computes multiple complementary string similarity scores (Gestalt, Levenshtein, Jaro-Winkler, LCS) per field pair as features for entity matching classifiers.

Run multiple independent retrieve-rerank pipelines and union-merge their predicted IDs per query via explode-groupby-unique

Generate diverse translation candidates by running nucleus sampling at multiple temperatures then pooling for MBR selection

Applies multiple dropout masks to the same hidden state and averages predictions for regularization and variance reduction.

Downsamples documents with no entity labels while keeping all positive samples, balancing class distribution in NER training without discarding entity-bearing examples.

Override model span predictions with full text for neutral sentiment or very short inputs where sub-span extraction is unreliable

Thresholds the O-class (non-entity) softmax probability in NER: if below threshold, overrides with the best non-O class to boost entity recall.

Exports a HuggingFace transformer to ONNX with dynamic axes, then auto-converts to BF16 mixed precision for 30-200% GPU inference speedup with 2x memory reduction.