Timebill Inference

Name: Timebill Inference
Author: ADu2021

ASecurity

Guarantee LLM inference completes within time budgets via response length prediction, execution time estimation, and adaptive KV cache eviction. Three-stage pipeline predicts response length, estimates end-to-end time with 1.22% accuracy, adjusts cache eviction ratio—enabling time-critical deployment in robotics and autonomous systems.

6 stars
0 votes
0 copies
1 view
Added September 9, 2026

devopspython

Security analysis

A100/100

Scanned September 9, 2026

npx -y skills add ADu2021/skillXiv --skill timebill-inference --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Timebill Inference?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Timebill Inference

[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/adu2021-timebill-inference/badge)](https://www.skillsdirectory.com/skills/adu2021-timebill-inference)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro

SKILL.md

---
name: timebill-inference
title: "TimeBill: Time-Budgeted Inference for LLMs"
version: 0.0.2
engine: skillxiv-v0.0.2-claude-opus-4.6
license: MIT
url: https://arxiv.org/abs/2512.21859
keywords: [inference, efficiency, time-budget, adaptive, latency]
description: "Guarantee LLM inference completes within time budgets via response length prediction, execution time estimation, and adaptive KV cache eviction. Three-stage pipeline predicts response length, estimates end-to-end time with 1.22% accuracy, adjusts cache eviction ratio—enabling time-critical deployment in robotics and autonomous systems."
---

## Overview

TimeBill solves the problem of unpredictable LLM inference latency in time-critical systems.

## Core Technique

**Three-Stage Pipeline:**

```python
# Stage 1: Response Length Prediction
length_predictor = fine_grained_response_length_model()
predicted_length = length_predictor.predict(input)

# Stage 2: Execution Time Estimation
ete = execution_time_estimator()
worst_case_time = ete.estimate(predicted_length)

# Stage 3: Adaptive KV Cache Eviction
cache_eviction_ratio = calculate_optimal_eviction(worst_case_time, time_budget)
```

## When to Use

Use when: Robotics, autonomous driving, time-critical inference.

## References

- Response length prediction
- FLOPs-based execution time modeling
- Adaptive KV cache management