All categories
DevOps & Cloud
Deployment, cloud, containers, CI/CD, monitoring, and infrastructure
- 20,288
- 846
Security grades appear on each card once the skill has been scanned. Newly imported skills may briefly show without a grade until the backfill job runs.
Open in full browserBrowse devops & cloud skills
Showing 6,769–6,792 of 20,288 skills
- Ffdl Platform OverheadEvaluates the runtime performance overhead and scalability of a containerized deep learning platform (FfDL) compared to bare-metal and specialized hardware, using standard image classification benchmarks. It measures how much throughput degrades when running DL training jobs in a Kubernetes-based multi-tenant environment versus direct execution. Use when the user has predictions and gold and needs to compute performance_overhead_pct.Votes: 0GitHub stars: 3
- Fast Gshare EvalEvaluates the performance of a spatio-temporal GPU sharing architecture for serverless deep learning inference. It measures how well the system manages resource multiplexing, isolation, and auto-scaling under varying workloads and allocation configurations. Use when the user wants to benchmark on MLPerf, or asks about evaluating this task. Reports throughput.Votes: 0GitHub stars: 3
- Dnn Inference Latency EvalProbes the inference latency and execution time variability of four industrial DNN models across heterogeneous cloud compute instances (CPU, GPU, and inference-optimized) on AWS and Chameleon Cloud. It evaluates how hardware heterogeneity impacts real-time processing requirements for safety-critical industrial applications. Use when the user wants to benchmark on Fire Detection Dataset, Oil Spill Detection Dataset, UCI Human Activity Recognition Using Smartphones, Marmousi 2 Dataset, or asks ...Votes: 0GitHub stars: 3
- Demographic DisparityEvaluates the alignment between expected demographic proportions in a target population and the actual representation in a dataset or model outputs. It probes sampling, deployment, and structural biases by quantifying discrepancies across protected attributes. Use when the user has predictions and gold and needs to compute demographic_disparity.Votes: 0GitHub stars: 3
- Deepfake Voice Detection EvalEvaluates deepfake voice detection models on their ability to generalize from controlled lab synthetic speech to real-world presentation distortions like loudspeaker playback and telephony injection. It measures robustness against realistic signal dynamics and spoofing pipelines that degrade audio quality. Use when the user wants to benchmark on ASVspoof19 LA, ASVspoof21 LA, ASVspoof21 LA-HT, ASVspoof21 DF, ASVspoof5 w/o Enc., In-the-wild, SpoofCeleb, Realworld, or asks about evaluating this ...Votes: 0GitHub stars: 3
- Deception EvalProbes the model's tendency to exhibit deceptive alignment, including alignment faking in chain-of-thought reasoning, jailbreak success rates, and strategic behavior shifts between evaluation and deployment stages. Use when the user wants to benchmark on DECEPTIONBENCH, StrongReject, JailbreakBench, BeaverTails, HarmfulQA, or asks about evaluating this task. Reports DTR.Votes: 0GitHub stars: 3
- CostThis evaluation probes a containerized framework's capability to automate the deployment and execution of distributed big data workloads. It specifically measures how execution time and monetary cloud costs scale as the cluster size increases from four to eight nodes. Use when the user has predictions and gold and needs to compute cost.Votes: 0GitHub stars: 3
- Cosoco Malware Detection EvalThis benchmark evaluates a model's ability to detect malware within Docker container file systems by treating the entire file system as a large-scale RGB image. It probes the capability to identify subtle, localized malicious byte patterns against a largely benign background, simulating real-world container security threats. Use when the user wants to benchmark on COSOCO, or asks about evaluating this task. Reports F1 score.Votes: 0GitHub stars: 3
- Bench360 EvalEvaluates local LLM inference across multiple dimensions, including task-specific quality (e.g., accuracy, F1, ROUGE) and system-level performance (latency, throughput, energy, memory, cold-start) under simulated workloads (single-stream, batch, server). Use when the user wants to benchmark on mmlu, squad_v2, cnn_dailymail, or asks about evaluating this task. Reports accuracy.Votes: 0GitHub stars: 3
- Auto Split EvalEvaluates the latency, accuracy, and model size of a collaborative edge-cloud DNN splitting framework (Auto-Split) compared to baselines like QDMP and Neurosurgeon across image classification and object detection tasks. Use when the user wants to benchmark on ImageNet, COCO 2017, or asks about evaluating this task. Reports End-to-end latency (normalized).Votes: 0GitHub stars: 3
- Anomaly Detection Telemetry EvalEvaluates the robustness and calibration stability of anomaly detection models across heterogeneous cloud telemetry datasets under strict no-leakage conditions. Probes how architectures handle distribution shift, high dimensionality, and label sparsity without test-time label access. Use when the user wants to benchmark on NAB, Microsoft Cloud Monitoring, Exathlon, IBM Console dataset, or asks about evaluating this task. Reports normalized NAB score.Votes: 0GitHub stars: 3
- Aipperf EvalEvaluates the end-to-end performance and weak scalability of heterogeneous AI-HPC systems using AutoML workloads. It measures how efficiently clusters execute dynamically scaling machine learning training and inference tasks across varying numbers of nodes. Use when the user wants to benchmark on CIFAR10, or asks about evaluating this task. Reports cumulative OPS.Votes: 0GitHub stars: 3
- MDBI MTBIEvaluates the robustness and safety of autonomous driving systems by quantifying how far and how long the vehicle operates between human interventions (disengagements). It enables unbiased comparison across different AV platforms and road environments by normalizing disengagement frequency with spatial and temporal data. Use when the user has predictions and gold and needs to compute MDBI, MTBI.Votes: 0GitHub stars: 3
- 1panel-app-builderUse when packaging Docker deployments as 1Panel local app store apps, including GitHub projects, docker-compose.yml files, docker run commands, app metadata, version directories, icons, README files, and 1Panel validation.Votes: 0GitHub stars: 213
- OpenrlhfHigh-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× faster than DeepSpeedChat with distributed architecture and GPU resource sharing.Votes: 0GitHub stars: 4
- Nemo EvaluatorEvaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.Votes: 0GitHub stars: 4
- Nemo CuratorGPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.Votes: 0GitHub stars: 4
- MlflowTrack ML experiments, manage model registry with versioning, deploy models to production, and reproduce experiments with MLflow - framework-agnostic ML lifecycle platformVotes: 0GitHub stars: 4
- Knowledge DistillationCompress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.Votes: 0GitHub stars: 4
- VllmServes LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.Votes: 0GitHub stars: 4
- Tensorrt LlmOptimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.Votes: 0GitHub stars: 4
- PhoenixOpen-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.Votes: 0GitHub stars: 4
- Llama CppRuns LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.Votes: 0GitHub stars: 4
- SentencepieceLanguage-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.Votes: 0GitHub stars: 4