All authors
Claude Skills by AgenttiX
github.com/AgenttiX2 skills0 installs0 views
- Optimize Llama CppOptimize a llama.cpp (llama-server) configuration of this repository for maximum speed and context size while keeping the output quality, by measuring each setting (KV cache type, context, n-cpu-moe, ubatch, flash attention, MTP, vision encoder placement, threads, split mode) with the scripts in ./benchmark. Use when asked to tune, optimize or create a llama.cpp preset or configuration, or to fit a larger context or a model on a GPU.Votes: 0GitHub stars: 3
- Optimize VllmOptimize a vLLM configuration of this repository for maximum speed and context size while keeping the output quality, by measuring each setting (kv-cache-memory-bytes, int8 KV cache and attention backend, CUDA graph sizes, max-num-seqs and prefill chunk, speculative decoding, CPU KV offloading, quantized embeddings, CPU vision towers, tensor parallelism) with the scripts in ./benchmark. Use when asked to tune, optimize or create a vLLM config YAML or configuration, or to fit a larger context ...Votes: 0GitHub stars: 3