Ideate

---
description: Multi-phase research idea generation pipeline: landscape scan → dual-model brainstorm → first-pass filter → deep validation → write to wiki
argument-hint: "[research-direction-or-topic] [--max-ideas N] [--skip-validation] [--auto]"
---

# /ideate

> Generates high-quality research ideas through a 5-phase pipeline, grounded in the wiki knowledge base and external search.
> Phase 1 scans the research landscape (wiki + WebSearch + S2), Phase 2 runs a dual-model brainstorm (Claude + Review LLM independently),
> Phase 3 applies a first-pass filter (feasibility + quick novelty check), Phase 4 performs deep validation (calls /novelty + /review),
> Phase 5 writes to the wiki (ideas/ + graph edges), including eliminated ideas (failure reasons recorded as anti-repetition memory).

## Inputs

- `direction` (optional): research direction, keywords, or specific problem description. If omitted, automatically selects the most valuable direction from open_questions.md.
- `--max-ideas N` (optional, default 3): maximum number of ideas to write to the wiki
- `--skip-validation`: skip Phase 4 deep validation (fast mode: Phase 1–3 + Phase 5 only)
- `--auto`: fully automatic mode, no pause for user confirmation (used when called by /research)

## Outputs

- `wiki/ideas/{slug}.md` — one page per idea (status: proposed), covering both top ideas and eliminated ideas
- `wiki/graph/edges.jsonl` — new idea → claim/gap relationship edges
- `wiki/graph/context_brief.md` — rebuilt compressed context
- `wiki/graph/open_questions.md` — rebuilt knowledge gap map
- **IDEA_REPORT** (printed to terminal) — pipeline execution summary, ranked results, novelty scores

## Wiki Interaction

### Reads
- `wiki/graph/context_brief.md` — global context
- `wiki/graph/open_questions.md` — knowledge gaps, drives idea direction
- `wiki/ideas/*.md` — existing ideas, especially status=failed ideas and their failure_reason (banlist)
- `wiki/claims/*.md` — current claims status, identifies weakly_supported and challenged claims
- `wiki/papers/*.md` — existing paper methods and results
- `wiki/concepts/*.md` — technical concepts, find cross-domain combination opportunities
- `wiki/topics/*.md` — research direction maps, SOTA and open problems
- `wiki/experiments/*.md` — existing experiment results, avoid duplication

### Writes
- `wiki/ideas/{slug}.md` — create new idea pages
- `wiki/graph/edges.jsonl` — add idea → claim/gap relationship edges (addresses_gap, inspired_by)
- `wiki/graph/context_brief.md` — rebuild
- `wiki/graph/open_questions.md` — rebuild
- `wiki/log.md` — append operation log

### Graph edges created
- `addresses_gap`: idea → claim/topic (knowledge gap the idea targets)
- `inspired_by`: idea → paper/concept (source of inspiration for the idea)

## Workflow

**Pre-conditions**:
1. Confirm working directory is the wiki project root (directory containing `wiki/`, `raw/`, `tools/`).
2. **Check wiki maturity**:
   ```bash
   python3 tools/research_wiki.py maturity wiki/ --json
   ```
   Adjust subsequent behavior based on maturity level:
   - **cold**: expand Phase 1 external search (WebSearch queries from 5 to 8, S2/DeepXiv limit from 20 to 30),
     skip wiki internal context loading (empty, no value), annotate "cold-start mode: heavier external search"
   - **warm**: standard behavior (current default)
   - **hot**: reduce Phase 1 external search (WebSearch queries from 5 to 2, S2/DeepXiv limit from 20 to 10),
     raise Phase 3 gap_alignment_bonus from +2 to +3, prioritize resolving weak claims already in the wiki
3. **Snapshot wiki state** (for the Growth Report at the end):
   Save the JSON returned by maturity to memory variable `maturity_before`

### Phase 1: Landscape Scan

Goal: build a comprehensive view of the target domain, including existing work, knowledge gaps, and recent advances.

1. **Load wiki internal context**:
   - Read `wiki/graph/context_brief.md` (global compressed context)
   - Read `wiki/graph/open_questions.md` (knowledge gap list)
   - Read all `wiki/ideas/*.md`, extract:
     - status=failed ideas → **banlist** (with failure_reason)
     - status=proposed/in_progress ideas → **active list** (avoid duplication)
   - Read `wiki/claims/*.md`, find claims with status=weakly_supported or challenged → **weak claims list**
   - If `direction` is specified, filter to the relevant subset

2. **External search** (run in parallel using Agent tool):
   - **WebSearch**: search for recent 6-month papers and advances in the target direction (3–5 queries)
   - **Semantic Scholar**:
     ```bash
     python3 tools/fetch_s2.py search "<direction-keywords>" --limit 20
     ```
     Fetch details for the top 5 highly-cited papers
   - **DeepXiv semantic search**:
     ```bash
     python3 tools/fetch_deepxiv.py search "<direction-keywords>" --mode hybrid --limit 20
     ```
     Fetch TLDR and keywords for top 5 most relevant results:
     ```bash
     python3 tools/fetch_deepxiv.py brief <arxiv_id>
     ```
     Semantic search supplements S2 keyword search for conceptually related papers that keyword search may miss.
   - **DeepXiv trending papers**:
     ```bash
     python3 tools/fetch_deepxiv.py trending --days 14
     ```
     Trending papers indicate community focus areas, useful for discovering trend-driven gaps.
   - **arXiv latest**: `site:arxiv.org <direction> 2025 2026`
   - **If DeepXiv is unavailable**: skip DeepXiv search and trending, rely on S2 + WebSearch only (fallback to original behavior).

3. **Compile landscape report** (internal use, not written to wiki):
   - Current SOTA methods and performance
   - Known open problems / unresolved challenges
   - Recent trends and hot topics
   - Knowledge gaps in the wiki (from gap_map)
   - Prohibited directions (from banlist)

### Phase 2: Dual-Model Brainstorm

Goal: generate ideas independently with Claude and Review LLM, exploiting the diversity that comes from different model perspectives.

**Follow `shared-references/cross-model-review.md`**: Claude and Review LLM generate independently without seeing each other's output.

1. **Claude generates 6–10 ideas**:
   - Input: landscape report + wiki gaps + weak claims + banlist
   - Strategies:
     - Cross-domain combination (method from Topic A + problem from Topic B)
     - Fill gaps in the gap_map
     - Strengthen weakly_supported claims
     - Alternative hypotheses that challenge challenged claims
     - Known limitations of SOTA → improvement directions
   - Each idea includes: title, hypothesis (1–2 sentences), approach sketch (3–5 sentences), target claims, estimated feasibility (high/medium/low)

2. **Review LLM independently generates 4–6 ideas** (run in parallel):
   ```
   mcp__llm-review__chat:
     system: "You are a creative ML researcher brainstorming research ideas.
              Generate novel, concrete, and feasible ideas based on the given context.
              For each idea, provide: title, hypothesis (1-2 sentences),
              approach sketch (3-5 sentences), and feasibility assessment."
     message: |
       ## Research Landscape
       {landscape report from Phase 1 — gaps, SOTA, trends}

       ## Knowledge Gaps
       {gap_map entries}

       ## Banlist (DO NOT revisit these)
       {failed ideas with failure_reason}

       ## Active Ideas (avoid duplicating)
       {proposed/in_progress ideas}

       Generate 4-6 novel research ideas that address the gaps above.
       Focus on ideas that are: (1) genuinely novel, (2) feasible within 3-6 months,
       (3) directly address a knowledge gap.
   ```

3. **Merge and deduplicate**:
   - Combine Claude's and Review LLM's ideas (10–16 candidates)
   - Remove highly similar ideas (merge ideas with the same core method, keep the more specific version)
   - Remove ideas that overlap with the banlist
   - Remove ideas that heavily duplicate the active list
   - Output: 8–12 candidate ideas

### Phase 3: First-Pass Filter

Goal: quickly eliminate ideas that are clearly infeasible or insufficiently novel.

Apply the following checks to each candidate idea:

1. **Feasibility check**:
   - Are GPU/compute requirements within reasonable range? (reference experiment setups already in the wiki)
   - Data availability (public datasets vs. private data)
   - Implementation complexity (achievable within 3–6 months?)
   - Label as feasibility: high/medium/low

2. **Quick novelty screening** (2–3 WebSearch queries per idea):
   - `"<idea-core-method>" + "<task>"` exact-match search
   - `<component-1> + <component-2>` component-combination search
   - If a highly similar published work is found → eliminate or flag

3. **Wiki alignment check**:
   - Does the idea address a known gap in the gap_map? (+score)
   - Does the idea target a weakly_supported claim? (+score)
   - Does the idea build on existing wiki knowledge? (+score)

4. **Filter decision**:
   - Eliminate if: feasibility=low AND quick novelty screening found similar published work
   - Eliminate if: highly correlated with a failure_reason in the banlist
   - Retain if: feasibility >= medium AND not eliminated
   - Output: 4–6 surviving ideas (ranked)

### Phase 4: Deep Validation

(Skip if `--skip-validation` is set; proceed directly to Phase 5.)

Apply deep validation to the top 3 ideas from Phase 3:

1. **Call /novelty** (one at a time):
   ```
   For each top idea:
   Skill: novelty
   Args: "<idea-title-and-hypothesis>"
   ```
   Record novelty score (1–5) and recommendations

2. **Call /review** (for top 2 ideas):
   ```
   Skill: review
   Args: "<idea-full-description>" --difficulty hard --focus method
   ```
   Record review score (1–10) and weaknesses

3. **Composite ranking**:
   - Final score = novelty_score × 2 + review_score + gap_alignment_bonus
   - gap_alignment_bonus: +2 if the idea directly targets a gap_map entry
   - If novelty_score <= 2 → downgrade to "modify needed"
   - If review_score <= 4 → downgrade to "major issues"

4. **If `--auto` is not set**: display ranked results in terminal, wait for user confirmation or adjustment

### Phase 5: Write to Wiki

Write the validated ideas to the wiki (including eliminated ideas, with their elimination reasons recorded).

1. **Write top ideas** (status: proposed):
   For the top `--max-ideas` ideas:
   ```bash
   # generate slug
   python3 tools/research_wiki.py slug "<idea-title>"
   ```
   Create `wiki/ideas/{slug}.md` **following the CLAUDE.md ideas template exactly** (all fields required; `lint.py` enforces `status` and `priority`):
   ```yaml
   ---
   title: "<idea title>"
   slug: "<idea-slug>"
   status: proposed
   origin: "ideate: <short description of the driving gap / weak claim / paper>"
   origin_gaps: []           # [[claim-slug]] list — claims or topics this idea targets
   tags: []                  # 2-5 topic tags (inherit from target claims / direction)
   domain: ""                # NLP / CV / ML Systems / Robotics (inherit from direction)
   priority: 3               # 1-5 — see Priority computation below
   pilot_result: ""          # empty until /exp-eval fills it
   failure_reason: ""        # empty for proposed ideas
   linked_experiments: []    # empty until /exp-design creates experiments
   date_proposed: YYYY-MM-DD
   date_resolved: ""         # empty until validated/failed
   ---
   ```

   **Priority computation** (maps Phase 4 signals into the 1-5 scale):
   - If `--skip-validation`: default `priority = 3`
   - Otherwise start from `novelty_score` (1-5 from /novelty)
   - `+1` if `gap_alignment_bonus > 0` (directly targets a gap_map entry)
   - `-1` if `review_score <= 4` (major issues downgrade)
   - Clamp to `[1, 5]`

   **Body sections** (exactly match the CLAUDE.md template — do not rename):
   ```markdown
   ## Motivation
   Which gap / weakly_supported claim / paper limitation drives this idea. Reference wiki pages via `[[slug]]`.

   ## Hypothesis
   1-2 sentences stating the testable proposition.

   ## Approach sketch
   3-5 sentences on the proposed method. Reference `[[paper-slug]]` or `[[concept-slug]]` for any component borrowed from existing work.

   ## Expected outcome
   What success looks like (metric / claim status change), plus the Phase 4 novelty & review summary:
   - Novelty score: N/5 — <one-line reason from /novelty>
   - Review score: M/10 — <one-line summary from /review>

   ## Risks
   Feasibility rating (high/medium/low) + top 2-3 risks. Include the main weaknesses surfaced by /review.

   ## Pilot results
   (empty — filled by /exp-eval after running the experiment)

   ## Lessons learned
   (empty — filled by /exp-eval after the idea reaches a terminal status)
   ```

2. **Write eliminated ideas** (status: failed):
   For ideas eliminated in Phase 3/4, also create `wiki/ideas/{slug}.md` using the **same template above**, with these overrides:
   - `status: failed`
   - `priority: 1` (eliminated ideas never block higher-priority work)
   - `date_resolved: YYYY-MM-DD` (today)
   - `failure_reason: "[filter] <specific elimination reason>"` — the `[filter]` prefix distinguishes ideate-stage eliminations from post-experiment failures (which /exp-eval tags differently). Examples: `"[filter] highly similar published work exists: <paper-title>"`, `"[filter] insufficient feasibility: GPU requirements too high"`
   - Body `## Motivation` and `## Hypothesis` should still be filled (so future banlist matching has content); `## Approach sketch` may be brief; `## Expected outcome` and `## Risks` can note why the idea was eliminated
   - These failed ideas become the banlist for future ideate runs

3. **Add graph edges**:
   ```bash
   # for each idea
   python3 tools/research_wiki.py add-edge wiki/ \
     --from "ideas/{slug}" --to "claims/{target-claim}" \
     --type addresses_gap --evidence "Generated by ideate"

   python3 tools/research_wiki.py add-edge wiki/ \
     --from "ideas/{slug}" --to "papers/{source-paper}" \
     --type inspired_by --evidence "Inspired by method in {paper-title}"
   ```

4. **Rebuild derived data**:
   ```bash
   python3 tools/research_wiki.py rebuild-context-brief wiki/
   python3 tools/research_wiki.py rebuild-open-questions wiki/
   ```

5. **Append log**:
   ```bash
   python3 tools/research_wiki.py log wiki/ \
     "ideate | {N} ideas proposed, {M} ideas filtered out | direction: {direction}"
   ```

6. **Print IDEA_REPORT to terminal**:
   ```markdown
   # Idea Generation Report

   ## Pipeline Summary
   - Direction: {direction}
   - Phase 1: Scanned {N} external papers, {M} wiki gaps identified
   - Phase 2: Generated {X} candidates (Claude: {a}, Review LLM: {b})
   - Phase 3: {Y} survived initial filter (from {X})
   - Phase 4: Deep validation on top {Z}
   - Phase 5: {K} ideas written to wiki

   ## Top Ideas (ranked)

   | Rank | Idea | Novelty | Review | Gap Align | Status |
   |------|------|---------|--------|-----------|--------|
   | 1 | [[slug]] | 4/5 | 7/10 | +2 | proposed |
   | 2 | [[slug]] | 3/5 | 6/10 | +0 | proposed |

   ## Filtered Out
   | Idea | Reason | Status |
   |------|--------|--------|
   | [[slug]] | Similar published work exists | failed |
   | [[slug]] | GPU requirements too high | failed |

   ## Suggested Next Steps
   - Run `/exp-design {top-idea-slug}` to design experiments
   - Run `/novelty` on any idea before investing time

   ## Wiki Growth
   | Metric | Before | After | Delta |
   |--------|--------|-------|-------|
   | Papers | {before} | {after} | +{delta} |
   | Claims | {before} | {after} | +{delta} |
   | Ideas | {before} | {after} | +{delta} |
   | Edges | {before} | {after} | +{delta} |
   | Maturity | {before_level} | {after_level} | {unchanged/upgraded} |
   (Only rows with delta != 0 are shown. Data is computed by comparing `maturity_before` from the pre-condition step against a fresh `maturity --json` call here.)
   ```

## Constraints

- **Auto-switch to cold-start mode when wiki is cold**: expand external search (WebSearch 8 queries, S2/DeepXiv limit 30), do not block execution
- **Every idea must have wiki grounding**: each idea must reference at least 2 wiki pages (paper/concept/claim)
- **Banlist must be loaded**: Phase 1 must read failed ideas' failure_reason; Phase 2/3 must check for overlap
- **Review LLM independence**: in Phase 2, Review LLM does not see Claude's idea list (cross-model-review.md)
- **Eliminated ideas are also written to wiki**: status=failed + failure_reason, as anti-repetition memory
- **No fabrication**: all ideas must be derived from existing wiki knowledge or external search results; do not invent non-existent papers or methods
- **Slug uniqueness**: check whether the same slug already exists in wiki/ideas/ before creating
- **Graph edges via tools/research_wiki.py**: do not manually edit edges.jsonl

## Error Handling

- **Wiki is empty**: proceed with external search (Phase 1 sources B/C/D), but skip wiki internal context; prompt user to build the knowledge base first
- **WebSearch unavailable**: skip external search, generate ideas from wiki internal knowledge only (degraded mode, noted in report)
- **Semantic Scholar API unavailable**: skip S2 search, rely on DeepXiv + WebSearch for compensation
- **DeepXiv API unavailable**: skip DeepXiv search and trending, fall back to S2 + WebSearch (original behavior)
- **Review LLM unavailable**: Phase 2 uses Claude only (no dual-model diversity, noted in report)
- **/novelty fails**: if novelty fails for a single idea in Phase 4, mark "novelty unverified" and continue
- **/review fails**: if review fails in Phase 4, mark "unreviewed" and continue; recommend user manually runs /review
- **Slug conflict**: if the same slug already exists in wiki/ideas/, append a numeric suffix (e.g. `sparse-lora-v2`)
- **All ideas eliminated**: still write to wiki (status: failed); report recommends user broaden the search direction or /ingest more papers

## Dependencies

### Tools（via Bash）
- `python3 tools/research_wiki.py maturity wiki/ --json` — check wiki maturity + Growth Report
- `python3 tools/research_wiki.py slug "<title>"` — generate slug
- `python3 tools/research_wiki.py add-edge wiki/ ...` — add graph edge
- `python3 tools/research_wiki.py rebuild-context-brief wiki/` — rebuild query_pack
- `python3 tools/research_wiki.py rebuild-open-questions wiki/` — rebuild gap_map
- `python3 tools/research_wiki.py log wiki/ "<message>"` — append log
- `python3 tools/fetch_s2.py search "<query>" --limit 20` — Semantic Scholar search
- `python3 tools/fetch_deepxiv.py search "<query>" --mode hybrid --limit 20` — DeepXiv semantic search
- `python3 tools/fetch_deepxiv.py brief <arxiv_id>` — fetch paper TLDR
- `python3 tools/fetch_deepxiv.py trending --days 14` — trending paper trends

### Skills（via Skill tool）
- `/novelty` — Phase 4 deep novelty validation
- `/review` — Phase 4 cross-model review

### MCP Servers
- `mcp__llm-review__chat` — Phase 2 Review LLM independent brainstorm

### Claude Code Native
- `WebSearch` — Phase 1 external search, Phase 3 quick novelty screening
- `Agent` tool — Phase 1 parallel search, Phase 2 parallel brainstorm

### Shared References
- `.claude/skills/shared-references/cross-model-review.md` — Phase 2 Review LLM independence principle