Smart patrol inspector for production systems. Scheduled randomized sampling checks based on BACKLOG requirements. Cost-controlled AI validation with intelligent spot-checking logic.
Scanned 5/27/2026
Install via CLI
Skills Directoryopenskills install seanyao/roll---
name: roll-sentinel
license: MIT
allowed-tools: "Read, Edit, Write, Bash, WebFetch"
description: Smart patrol inspector for production systems. Scheduled randomized sampling checks based on BACKLOG requirements. Cost-controlled AI validation with intelligent spot-checking logic.
---
# Sentinel
**Smart Patrol Inspector** - Scheduled, randomized, cost-controlled patrol and acceptance checks for production systems.
## When Not to Use
- One-off debugging of a reported bug (use `$roll-debug`)
- Full-coverage regression testing (sentinel samples, does not cover)
- Dev/staging environment checks (use CI tests instead)
- Pre-commit self-review of diffs (use `$roll-.review`)
## Core Principle
```
┌─────────────────────────────────────────────────────────────┐
│ Smart Patrol Logic │
├─────────────────────────────────────────────────────────────┤
│ │
│ Not full-coverage checks! Think of it like a security │
│ guard on patrol: │
│ │
│ 🕐 Scheduled Triggers - Auto-patrol on schedule │
│ └── "Patrol once every 6 hours" │
│ │
│ 🎲 Random Sampling - Different samples each time │
│ └── "Check Stories 1-10 this time, 50-60 next time" │
│ │
│ 💰 Cost Control - AI checks are expensive, use sparingly │
│ └── "Only check 10 each time, not all 100" │
│ │
│ 🎯 BACKLOG-Based - Validate against requirements │
│ └── "US-001 says login works, so verify login" │
│ │
└─────────────────────────────────────────────────────────────┘
```
## Patrol Strategy
### Sampling Logic
```javascript
// Sampling logic for each patrol
function selectSamples(backlog, strategy = 'smart') {
const completedStories = backlog.filter(s => s.status === '✅');
switch(strategy) {
case 'random':
// Fully random: randomly select N from all completed Stories
return shuffle(completedStories).slice(0, 10);
case 'weighted':
// Weighted random: prioritize recently modified and frequently used
return completedStories
.sort((a, b) => b.lastModified - a.lastModified)
.slice(0, 5) // 5 most recent
.concat(shuffle(completedStories).slice(0, 5)); // + 5 random
case 'coverage':
// Coverage sampling: ensure different modules are all covered
const byModule = groupBy(completedStories, 'module');
return Object.values(byModule).map(
stories => randomPick(stories)
);
}
}
```
### Cost Control
| Strategy | Sample Size | Frequency | Use Case |
|----------|-------------|-----------|----------|
| **Light** | 5 Stories | Once daily | Stable period |
| **Normal** | 10 Stories | Every 6 hours | General monitoring |
| **Intensive** | 20 Stories | Every hour | Post-release period |
| **Full** | All | Once weekly | Weekly patrol |
```yaml
# sentinel.config.yml
cost_control:
daily_budget: 100 # AI call budget
light_patrol:
samples: 5
schedule: "0 9 * * *" # Daily at 9am
normal_patrol:
samples: 10
schedule: "0 */6 * * *" # Every 6 hours
# Intensive patrol after deployment
post_deploy:
trigger: "after_deploy"
samples: 20
duration: "2h" # Lasts 2 hours
```
### Uncertainty Handling
```javascript
// Systems have uncertainty; a single check may be inaccurate.
// Use multiple random checks to increase confidence.
class UncertaintyHandler {
// Track check result history
history = new Map(); // storyId -> [check1, check2, ...]
// Determine if an issue is real
isRealIssue(storyId, currentResult) {
const pastResults = this.history.get(storyId) || [];
pastResults.push(currentResult);
// Only consider it a real issue if it fails 3 times consecutively
const recent3 = pastResults.slice(-3);
if (recent3.every(r => r.status === 'FAIL')) {
return true; // Real issue
}
// If it fails occasionally, it may be intermittent; keep observing
if (recent3.filter(r => r.status === 'FAIL').length === 1) {
return false; // Likely intermittent, don't alert yet
}
return false;
}
}
```
## When to Patrol
### Scheduled Patrols
```bash
# Daily patrol - randomly check a few each day
$roll-sentinel patrol --mode=normal
# Late-night patrol - full check during off-peak hours
$roll-sentinel patrol --mode=full --schedule="0 3 * * *"
# Weekend walkthrough - check the week's accumulation on Sunday
$roll-sentinel patrol --mode=weekly --schedule="0 10 * * 0"
```
### Event-Triggered
```bash
# Intensive patrol for 2 hours after deployment
$roll-sentinel patrol --mode=intensive --duration=2h --after-deploy
# Emergency check after an alert
$roll-sentinel patrol --mode=focus --target=US-XXX
```
## Patrol Report
```markdown
## 🛡️ Sentinel Patrol Report #247
**Time**: 2024-01-15 14:00 UTC
**Patrol ID**: patrol-20240115-1400
**Mode**: Normal (Random Sampling)
### 📊 Sampling Info
| Metric | Value |
|--------|-------|
| Total Stories | 150 |
| Sample Size | 10 |
| Sampling Rate | 6.7% |
| Cost Estimate | $0.07 |
### 🎲 Random Sample
| # | Story | Module | Last Checked | Result |
|---|-------|--------|--------------|--------|
| 1 | US-LOGIN-001 | Auth | 6h ago | ✅ |
| 2 | US-STORY-042 | Content | 12h ago | ✅ |
| 3 | US-AUDIO-015 | Player | 2h ago | 🟡* |
| 4 | US-SEARCH-003 | Search | 18h ago | ✅ |
| 5 | ... | ... | ... | ... |
\* US-AUDIO-015: Occasional playback stuttering (2nd occurrence, under observation)
### 🔴 Issues Found
None (no confirmed issues found in this sample)
### 📈 Patrol Statistics (7 days)
| Metric | Value |
|--------|-------|
| Total Patrols | 28 |
| Stories Checked | 280 |
| Issues Found | 3 |
| False Positives | 1 |
| Coverage | 85% of all stories |
### 💰 Cost Report
| Item | Usage |
|------|-------|
| AI Checks | 280 calls |
| Playwright Runs | 28 sessions |
| Total Cost | $2 |
| Budget Used | 15% of monthly |
```
## Smart Detection Logic
### Pattern 1: Intermittent vs Real Issue
```javascript
// Don't alert on the first failure; look at the trend
const checks = [
{ time: 'T-6h', status: 'PASS' },
{ time: 'T-12h', status: 'FAIL' }, // Intermittent?
{ time: 'T-18h', status: 'PASS' },
{ time: 'Now', status: 'FAIL' }, // Failed again!
];
// 2 consecutive failures → Create Issue
if (lastN(checks, 2).all(c => c.status === 'FAIL')) {
createBacklogItem('FIX-XXX');
}
// Occasional failure → Log for observation
else if (checks.filter(c => c.status === 'FAIL').length <= 1) {
logForObservation('Might be flaky, continue monitoring');
}
```
### Pattern 2: Hotspot Detection
```javascript
// Some Stories frequently show issues when sampled.
// Automatically increase their check frequency.
const hotSpots = analyzeHistory();
// hotSpots = [
// { story: 'US-AUDIO-015', failRate: 0.3 }, // 30% failure rate
// { story: 'US-SEARCH-003', failRate: 0.1 },
// ]
// Increase weight for hotspots
if (hotSpots.some(h => h.story === selectedStory)) {
// If it's a hotspot, even if not randomly selected, add extra check probability
if (Math.random() < 0.3) {
extraCheck(story);
}
}
```
## Cost Optimization
### Tiered Checking
```
Level 1: Lightweight Check (cheap)
└── HTTP ping / API health check
└── Cost: $0.001 per check
Level 2: Functional Check (moderate)
└── Playwright critical path
└── Cost: $0.01 per check
Level 3: AI Deep Check (expensive)
└── AI-powered content quality analysis
└── Cost: $0.07 per check
Strategy:
- Each patrol: 90% Level 1 + 10% Level 2
- Once weekly: Level 3 deep inspection
```
### Smart Batching
```javascript
// Batch checks to reduce cost.
// Instead of 10 separate checks, open one browser for all 10.
async function batchCheck(stories) {
const browser = await chromium.launch();
const context = await browser.newContext();
// Reuse browser session to check multiple Stories
for (const story of stories) {
const page = await context.newPage();
await checkStory(page, story);
await page.close();
}
await browser.close();
// Cost: 1 session for 10 checks
}
```
## Workflow: Find Issue → Backlog
```
┌─────────────────────────────────────────────────────────────┐
│ Issue Discovery Workflow via Patrol │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. Sentinel Patrol (scheduled random sampling) │
│ └── Sample: US-AUDIO-015 │
│ │
│ 2. Check Result │
│ └── Status: FAIL (playback stuttering) │
│ │
│ 3. Uncertainty Check │
│ └── Check history: this is the 2nd failure │
│ └── 1st was 6h ago (possibly intermittent) │
│ └── Decision: continue observing, don't create Issue │
│ │
│ 4. Next Patrol │
│ └── US-AUDIO-015 sampled again (hotspot weighting) │
│ └── Status: FAIL (failed again!) │
│ └── Check history: 2 consecutive failures │
│ └── Decision: create FIX-AUDIO-015 │
│ │
│ 5. Create Backlog Item │
│ └── Add FIX-AUDIO-015 to .roll/backlog.md │
│ └── Status: 📋 Todo │
│ └── Awaiting human fix │
│ │
│ 6. Human Fix │
│ └── User: "Fix FIX-AUDIO-015" │
│ └── $roll-fix FIX-AUDIO-015 │
│ │
│ 7. Verification │
│ └── Next patrol will prioritize verifying this FIX │
│ └── Status: ✅ Fixed │
│ │
└─────────────────────────────────────────────────────────────┘
```
## Integration with Other Skills
```
┌─────────────────────────────────────────────────────────────┐
│ Complete Monitoring System │
├─────────────────────────────────────────────────────────────┤
│ │
│ $roll-sentinel patrol Scheduled random patrol (main) │
│ ↓ │
│ Issue found? ──┬── Yes ──→ Create BACKLOG item │
│ │ Await $roll-fix │
│ │ │
│ └── No ──→ Continue patrolling │
│ │
│ $roll-debug On-demand deep diagnosis (aux) │
│ (When Sentinel finds an issue, manually trigger deep dive) │
│ │
│ $roll-build Post-fix regression verify │
│ │
└─────────────────────────────────────────────────────────────┘
```
## Best Practices
1. **Don't do full-coverage checks** - Expensive and unnecessary
2. **Random + Hotspots** - Balance coverage and cost
3. **Multi-check confirmation** - Avoid false positives from intermittent failures
4. **Budget control** - Set daily/monthly AI call limits
5. **Progressive intensity** - Light during stable periods, Intensive after releases
No comments yet. Be the first to comment!
