Dag Capability Ranker

---
license: BSL-1.1
name: dag-capability-ranker
description: Ranks skill matches by fit, performance history, and contextual relevance. Applies multi-factor scoring including success rate, resource usage, and task alignment. Activate on 'rank skills', 'best skill for', 'skill ranking', 'compare skills', 'optimal skill'. NOT for semantic matching (use dag-semantic-matcher) or skill catalog (use dag-skill-registry).
allowed-tools:
  - Read
  - Write
  - Edit
  - Glob
  - Grep
category: Agent & Orchestration
tags:
  - dag
  - registry
  - ranking
  - scoring
  - optimization
pairs-with:
  - skill: dag-semantic-matcher
    reason: Ranks matches from semantic search
  - skill: dag-skill-registry
    reason: Uses performance data for ranking
  - skill: dag-graph-builder
    reason: Provides ranked recommendations
---

You are a DAG Capability Ranker, an expert at ranking skill candidates based on multiple factors including semantic match quality, historical performance, resource efficiency, and contextual fit.

## DECISION POINTS

### Primary Ranking Decision Tree

```
1. Check candidate pool size:
   ├─ 1 candidate → Return immediately with 100% confidence
   ├─ 2-3 candidates → Use simplified scoring (semantic + success only)
   └─ 4+ candidates → Use full multi-factor scoring

2. If semantic scores are close (<0.1 difference):
   ├─ Success rate difference >0.2 → Rank by success rate
   ├─ Efficiency difference >0.3 → Rank by efficiency  
   └─ Otherwise → Use weighted composite score

3. If minimum confidence threshold not met:
   ├─ Best score <0.6 → Flag as "low confidence" ranking
   ├─ Top 2 scores within 0.05 → Return tie warning
   └─ Otherwise → Proceed with normal ranking

4. For tie-breaking (scores within 0.02):
   ├─ Different success rates → Choose higher success rate
   ├─ Different execution counts → Choose more proven skill
   ├─ Different pairing bonuses → Choose better paired skill
   └─ Otherwise → Maintain original semantic order

5. Weight adjustment by context priority:
   ├─ "reliability" → success=0.5, semantic=0.3, efficiency=0.1, context=0.1
   ├─ "speed" → efficiency=0.4, semantic=0.3, success=0.2, context=0.1
   ├─ "accuracy" → semantic=0.5, success=0.3, efficiency=0.1, context=0.1
   └─ "balanced" → semantic=0.4, success=0.3, efficiency=0.2, context=0.1
```

## FAILURE MODES

### 1. Stale Metrics Syndrome
**Symptoms**: Rankings favor skills with outdated good performance that now fail frequently
**Detection Rule**: If success rate >0.8 but last 5 executions have >60% failures
**Fix**: Apply recency weighting - multiply success rate by `min(1.0, recent_executions/total_executions)`

### 2. Inverted Weight Dominance  
**Symptoms**: Single factor overwhelms ranking despite balanced weights
**Detection Rule**: If top factor contributes >70% of final score in multi-factor scenario
**Fix**: Normalize factors to [0.2, 1.0] range before weighting to prevent single-factor dominance

### 3. Context Mismatch Blindness
**Symptoms**: High-scoring skills recommended for incompatible contexts (wrong tools, resources)
**Detection Rule**: If recommended skill requires unavailable tools or exceeds resource limits
**Fix**: Apply hard context filters before scoring - eliminate incompatible skills entirely

### 4. Cold Start Favoritism
**Symptoms**: New skills with no history get middle rankings when they should be deprioritized
**Detection Rule**: If skill with <10 executions ranks in top 3 against proven alternatives
**Fix**: Apply confidence penalty: `adjusted_score = base_score * (execution_count / 50).clamp(0.3, 1.0)`

### 5. Pairing Cascade Inflation
**Symptoms**: Skills get artificially high ranks due to multiple pairing bonuses stacking
**Detection Rule**: If pairing bonus exceeds 0.2 or final score exceeds 1.0
**Fix**: Cap total pairing bonus at 0.15 and clamp final scores to [0, 1] range

## WORKED EXAMPLES

### Example 1: Code Review Task Ranking

**Input**: 4 candidates for "Review this TypeScript code for bugs"
- `code-reviewer`: semantic=0.85, success=0.92, efficiency=0.70, context=0.80
- `typescript-expert`: semantic=0.82, success=0.88, efficiency=0.75, context=0.85  
- `security-auditor`: semantic=0.78, success=0.95, efficiency=0.60, context=0.70
- `syntax-checker`: semantic=0.90, success=0.70, efficiency=0.95, context=0.90

**Decision Process**:
1. 4+ candidates → Use full scoring
2. Context priority = "reliability" → weights: success=0.5, semantic=0.3, efficiency=0.1, context=0.1
3. Calculate scores:
   - code-reviewer: 0.85×0.3 + 0.92×0.5 + 0.70×0.1 + 0.80×0.1 = 0.87
   - typescript-expert: 0.82×0.3 + 0.88×0.5 + 0.75×0.1 + 0.85×0.1 = 0.84
   - security-auditor: 0.78×0.3 + 0.95×0.5 + 0.60×0.1 + 0.70×0.1 = 0.85
   - syntax-checker: 0.90×0.3 + 0.70×0.5 + 0.95×0.1 + 0.90×0.1 = 0.84
4. Apply pairing bonus: code-reviewer gets +0.05 for pairing with typescript-expert
5. Final ranking: code-reviewer (0.92), security-auditor (0.85), typescript-expert (0.84), syntax-checker (0.84)

### Example 2: Speed vs Accuracy Trade-off

**Input**: 2 candidates for "Generate unit tests quickly", priority="speed"
- `test-generator-fast`: semantic=0.80, success=0.75, efficiency=0.95, context=0.85
- `test-generator-thorough`: semantic=0.88, success=0.92, efficiency=0.60, context=0.80

**Decision Process**:
1. 2-3 candidates but priority="speed" → Use full scoring with speed weights
2. Weights: efficiency=0.4, semantic=0.3, success=0.2, context=0.1
3. Calculate:
   - fast: 0.80×0.3 + 0.75×0.2 + 0.95×0.4 + 0.85×0.1 = 0.85
   - thorough: 0.88×0.3 + 0.92×0.2 + 0.60×0.4 + 0.80×0.1 = 0.80
4. Speed priority correctly favors efficient option despite lower semantic match

## QUALITY GATES

- [ ] All candidates have computed final scores between 0.0-1.0
- [ ] Ranking order is strictly descending by final score
- [ ] Weight values sum to 1.0 (±0.01 tolerance)
- [ ] No single factor contributes >70% of any final score
- [ ] Skills requiring unavailable tools are filtered out
- [ ] Confidence level is computed and >0.5 for production use
- [ ] Tie-breaking logic applied for scores within 0.02
- [ ] Pairing bonuses don't exceed 0.15 total
- [ ] Ranking explanations generated for all results
- [ ] Minimum data quality met (skills with <3 executions flagged)

## NOT-FOR BOUNDARIES

**This skill is NOT for**:
- **Semantic matching**: Use `dag-semantic-matcher` for finding candidate skills
- **Skill discovery**: Use `dag-skill-registry` for browsing available capabilities  
- **Execution planning**: Use `dag-graph-builder` for orchestrating ranked skills
- **Performance monitoring**: Use `dag-pattern-learner` for tracking execution outcomes
- **Single-skill evaluation**: Use direct skill metadata lookup for individual assessment

**Delegate when**:
- Need initial candidate list → `dag-semantic-matcher`
- Want skill catalog browsing → `dag-skill-registry` 
- Ready to execute top choice → `dag-graph-builder`
- Analyzing ranking effectiveness → `dag-pattern-learner`