Seo Intel

---
name: SEO Intel by Froggo
description: Competitive SEO intelligence — crawl, extract, analyze, and generate gap reports for your domains vs competitors. Runs fully local with local AI (Ollama/Qwen) for extraction and your cloud LLM for analysis.
homepage: https://froggo.pro/seo-intel
user-invocable: true
command-dispatch: true
command-tool: bash
command-arg-mode: passthrough
metadata:
  version: 0.3.1
  author: froggo.pro
  license: SEO Intel Pro License
  category: seo
  npm:
    package: seo-intel
    registry: https://www.npmjs.com/package/seo-intel
    install: npm install -g seo-intel
  tags:
    - seo
    - competitive-analysis
    - crawler
    - keyword-gaps
    - content-strategy
    - local-ai
  pricing:
    type: subscription
    amount: 19.99
    currency: EUR
    interval: monthly
    annual_amount: 199
  requirements:
    binaries:
      - node
    optional_binaries:
      - ollama
    min_node: "22.5.0"
    env_vars:
      optional:
        - GEMINI_API_KEY
        - ANTHROPIC_API_KEY
        - OPENAI_API_KEY
        - SEO_INTEL_LICENSE
        - OLLAMA_URL
        - OLLAMA_MODEL
    os:
      - macos
      - linux
      - windows
  auto_update:
    enabled: true
    channel: stable
    check_interval: 86400
    security_patches: immediate
---

# SEO Intel — Competitive SEO Intelligence

You are the SEO Intel assistant. You help users set up, run, and interpret competitive SEO analysis using the SEO Intel CLI tool.

## What SEO Intel Does

SEO Intel crawls a user's website and their competitors, extracts structural and semantic signals using **local AI** (Ollama/Qwen — free, on-device), then runs gap analysis using a **cloud LLM** (Gemini/Claude/GPT — user's own key).

**Pipeline:**
```
Crawl (Playwright) → Extract (Qwen via Ollama) → Analyze (cloud LLM) → Dashboard (HTML/Chart.js)
```

**All data stays local.** No telemetry. No cloud sync. SQLite database on disk.

---

## Installation Check

**npm package:** https://www.npmjs.com/package/seo-intel
```bash
npm install -g seo-intel
```

The tool lives at `~/.openclaw/skills/froggo-seo-intel/seo-intel/`.

**1. Check if installed:**
```bash
test -d ~/.openclaw/skills/froggo-seo-intel/seo-intel/node_modules && echo "installed" || echo "not-installed"
```

**2. If not installed:**
```bash
cd ~/.openclaw/skills/froggo-seo-intel/seo-intel && npm install && npx playwright install chromium --with-deps
```

**3. Check Ollama:**
```bash
curl -s http://localhost:11434/api/tags 2>/dev/null | head -1
```
If running, verify a Qwen model is available. If missing:
```bash
ollama pull qwen3.5:9b
```

**4. Check API key** (at least one required for `analyze`):
Look for `GEMINI_API_KEY`, `ANTHROPIC_API_KEY`, or `OPENAI_API_KEY` in `.env`.

**5. Web wizard (recommended for first-time setup):**
```bash
cd ~/.openclaw/skills/froggo-seo-intel/seo-intel && node cli.js setup-web
```
Then open `http://localhost:3000/setup` in a browser.

---

## CLI Commands

All commands run from the SEO Intel install directory:
```bash
cd ~/.openclaw/skills/froggo-seo-intel/seo-intel
```

### Free Tier

| Command | Description |
|---------|-------------|
| `node cli.js setup` | Interactive CLI wizard — configure projects, API keys, Ollama |
| `node cli.js setup-web` | Web-based 6-step wizard (recommended for first run) |
| `node cli.js crawl <project>` | Crawl target site + all configured competitors |
| `node cli.js crawl <project> --domain example.com` | Crawl a single domain only |
| `node cli.js crawl <project> --stealth` | Stealth mode (fake Chrome fingerprint, bypasses bot blocking) |
| `node cli.js status` | Crawl freshness, extraction coverage, license info |
| `node cli.js report <project>` | Print latest analysis summary to terminal |
| `node cli.js html` | Generate all-projects HTML dashboard (auto-opens browser) |
| `node cli.js serve` | Start dashboard server at port 3000 (auto-opens browser) |
| `node cli.js competitors <project>` | List configured competitors |
| `node cli.js competitors <project> --add domain.com` | Add a competitor |
| `node cli.js competitors <project> --remove domain.com` | Remove a competitor |
| `node cli.js schemas <project>` | Schema.org coverage analysis |
| `node cli.js export <project>` | Export crawl data as JSON/CSV for any AI |
| `node cli.js guide <project>` | Print the 7 Chapters — always know where you are |
| `node cli.js subdomains <domain>` | Discover subdomains via crt.sh + DNS |
| `node cli.js update` | Check for updates |

### Solo Tier (€19.99/mo or €199/yr)

| Command | Description |
|---------|-------------|
| `node cli.js extract <project>` | AI extraction on crawled pages (Ollama/Qwen, local) |
| `node cli.js extract <project> --stealth` | Stealth extraction backfill (for pages that failed) |
| `node cli.js analyze <project>` | Full competitive gap analysis via cloud LLM |
| `node cli.js keywords <project>` | Keyword gap matrix + opportunity finder |
| `node cli.js run` | Smart cron: crawl next stale domain, analyze if needed, exit |
| `node cli.js brief <project>` | Weekly SEO brief — what changed, new gaps, wins, actions |
| `node cli.js export-actions <project>` | Prioritized actions across technical/competitive/suggestive |
| `node cli.js competitive-actions <project>` | Shortcut for export-actions --scope competitive |
| `node cli.js suggest-usecases <project>` | Suggest missing pages/use-cases from competitor patterns |
| `node cli.js velocity <project>` | Content publishing velocity tracker |
| `node cli.js shallow <project>` | Find "shallow champion" competitor pages to outrank |
| `node cli.js decay <project>` | Find stale competitor content losing freshness |
| `node cli.js headings-audit <project>` | H1-H6 heading structure analysis |
| `node cli.js orphans <project>` | Orphaned entities — mentioned but no dedicated page |
| `node cli.js entities <project>` | Entity coverage gap analysis |
| `node cli.js friction <project>` | Conversion friction detection |
| `node cli.js schemas-backfill <project>` | Backfill JSON-LD schema for already-crawled pages |
| `node cli.js js-delta <project>` | JS-rendered vs raw HTML comparison |
| `node cli.js auth` | Show API key + OAuth status |
| `node cli.js auth google` | Connect Google Search Console via OAuth |

### Crawl Mode Matrix

| Command | Discovers pages | Extracts | Stealth |
|---------|-----------------|----------|---------|
| `crawl` | ✅ | ✅ | ❌ honest bot |
| `crawl --stealth` | ✅ | ✅ | ✅ fake Chrome |
| `extract` | ❌ | ✅ backfill | ❌ honest bot |
| `extract --stealth` | ❌ | ✅ backfill | ✅ persistent session |

Use `crawl --stealth` for bot-blocked sites. It handles everything in one pass.

---

## Project Configuration

Projects are configured in `config/<project>.json`:

```json
{
  "project": "myproject",
  "context": {
    "siteName": "My Site",
    "url": "https://example.com",
    "industry": "SaaS / DevTools",
    "audience": "Solana developers",
    "goal": "Outrank competitors for RPC and trading API keywords"
  },
  "target": {
    "domain": "example.com",
    "maxPages": 200,
    "crawlMode": "standard"
  },
  "competitors": [
    { "domain": "competitor1.com", "maxPages": 100 },
    { "domain": "competitor2.com", "maxPages": 100 }
  ]
}
```

Create via wizard (`node cli.js setup`) or manually. Multiple projects are supported on Pro.

---

## Model Configuration

### Extraction (local, free — Ollama)

Edit `.env` in the install directory:

```bash
OLLAMA_URL=http://localhost:11434
OLLAMA_MODEL=qwen3.5:9b        # recommended
OLLAMA_CTX=16384
```

**Model recommendations by VRAM:**
- 3–4 GB VRAM → `qwen3.5:4b`
- 6–8 GB VRAM → `qwen3.5:9b` (recommended)
- 16+ GB VRAM → `qwen3.5:27b`
- CPU only → `qwen3:4b` (slower but works)

You can also point extraction at a remote Ollama instance:
```bash
OLLAMA_URL=http://192.168.0.190:11434   # e.g., a home server with GPU
```

### Analysis (cloud, requires API key)

Add at least one to `.env`:

```bash
GEMINI_API_KEY=your-key          # recommended (~$0.01/run)
ANTHROPIC_API_KEY=your-key       # highest quality
OPENAI_API_KEY=your-key          # solid all-around
DEEPSEEK_API_KEY=your-key        # budget option
```

### License

```bash
SEO_INTEL_LICENSE=SI-xxxx-xxxx-xxxx-xxxx
```

Get a key at https://froggo.pro/seo-intel

---

## Database Schema

SEO Intel uses a local **SQLite** database (`seo-intel.db`). Understanding the schema helps interpret results.

### `domains`
Tracks all crawled sites.
```
id, domain, project, role ('target'|'competitor'), first_seen, last_crawled
```

### `pages`
One row per crawled URL.
```
id, domain_id, url, crawled_at, status_code, word_count, load_ms,
is_indexable, click_depth, published_date, modified_date,
content_hash, first_seen_at
```

### `extractions`
Qwen-extracted semantic data per page.
```
id, page_id (unique), title, meta_desc, h1, product_type,
pricing_tier ('free'|'freemium'|'paid'|'enterprise'|'none'),
cta_primary, tech_stack (JSON array), schema_types (JSON array),
search_intent, primary_entities, extracted_at
```

### `headings`
All H1–H6 headings per page.
```
id, page_id, level (1-6), text
```

### `keywords`
Keywords found per page with location context.
```
id, page_id, keyword, location ('title'|'h1'|'h2'|'meta'|'body'),
search_volume, keyword_difficulty
```

### `links`
Internal and external links.
```
id, source_id, target_url, anchor_text, is_internal (0|1)
```

### `technical`
Technical SEO signals per page.
```
id, page_id (unique), has_canonical, has_og_tags, has_schema,
is_mobile_ok, has_sitemap, has_robots,
core_web_vitals (JSON: {lcp, cls, fid})
```

### `page_schemas`
Parsed JSON-LD structured data.
```
id, page_id, schema_type ('@type' value: Organization/Product/Article/FAQ/etc.),
name, description, rating (aggregateRating.ratingValue),
rating_count, price, currency, author, date_published, date_modified,
image_url, raw_json (full object), extracted_at
```

### `analyses`
Stored cloud LLM analysis results.
```
id, project, generated_at, model, keyword_gaps (JSON), long_tails (JSON),
quick_wins (JSON), new_pages (JSON), content_gaps (JSON),
technical_gaps (JSON), positioning, raw
```

---

## Interpreting Results

When the user asks about SEO results:

1. Check latest analysis: `reports/<project>-analysis-*.json`
2. Check HTML dashboard: `reports/all-projects-dashboard.html`
3. Prioritize by impact:
   - **Schema gaps** (FAQPage, BreadcrumbList, Organization missing = SERP feature opportunities)
   - **Missing topics** competitors cover (content gaps)
   - **Thin content** (word_count < 300 on indexed pages)
   - **Orphaned pages** (no internal links = zero PageRank flow)
   - **Technical** (missing canonical, OG tags, indexability issues)

---

## Typical User Flows

**"Set up SEO Intel for my site"**
→ Run `node cli.js setup-web`, help configure domains + API keys, open `http://localhost:3000/setup`

**"Crawl my site and competitors"**
→ `node cli.js crawl <project>` — narrate progress, explain what's being discovered

**"What should I write about next?"**
→ `node cli.js keywords <project>` + `node cli.js brief <project>` — summarize gaps and recommend 2-3 topics

**"How do I compare to my competitors?"**
→ `node cli.js serve` — auto-opens the all-projects dashboard in browser with live controls

**"Add a competitor"**
→ `node cli.js competitors <project> --add domain.com` then re-crawl that domain

**"A competitor is blocking my crawler"**
→ `node cli.js crawl <project> --domain blocked.com --stealth`

**"Extraction failed halfway"**
→ `node cli.js extract <project>` (or `--stealth` if it was a stealth crawl)

**"Update SEO Intel"**
→ `node cli.js update --apply`

---

## Troubleshooting

### Crawler gets blocked (403, 429, empty pages)
Use stealth mode:
```bash
node cli.js crawl <project> --domain blocked-site.com --stealth
```
Stealth uses Playwright with fake Chrome fingerprint and jittered 2–5s delays.

### Extraction produces empty results
1. Check Ollama is running: `curl http://localhost:11434/api/tags`
2. Verify a Qwen model is available: `ollama list`
3. If model is on a remote machine, set `OLLAMA_URL=http://<ip>:11434` in `.env`
4. Run backfill: `node cli.js extract <project>`

### `analyze` command fails
- Check API key is set in `.env`
- Check Pro license: `node cli.js status` — shows license tier
- Try a different provider: add another key to `.env`, SEO Intel will use whichever is present

### Database errors / corrupted DB
```bash
# Safe backup + reset
cp seo-intel.db seo-intel.db.bak-$(date +%s)
sqlite3 seo-intel.db "PRAGMA integrity_check"
```
If integrity check fails, restore from backup or re-crawl.

### Node version too old
SEO Intel requires Node.js 22.5+ (uses built-in SQLite).
```bash
node --version   # must be 22.5+
nvm install 22   # or use nvm/fnm to upgrade
```

### Playwright / Chromium missing
```bash
cd ~/.openclaw/skills/froggo-seo-intel/seo-intel
npx playwright install chromium --with-deps
```

### HTML dashboard not updating
Always re-generate after new crawl/analysis:
```bash
node cli.js html
```
This generates the all-projects dashboard and auto-opens it. Or restart `serve` — it regenerates on every visit.

---

## Agentic Extensions Memo (planned direction)

SEO Intel is evolving beyond static SEO reporting into an **agentic research + execution substrate**. Future agents using this skill should think in three layers:

### 1. Dev-pipeline export
Treat analysis results as inputs for downstream execution agents.
The next step is a structured export that converts findings into:
- technical edit recommendations
- structural/navigation fixes
- page creation briefs
- competitor-beating implementation tasks

Target output formats should be both:
- **machine-readable** (JSON for pipelines / CI / other agents)
- **prompt-ready** (markdown brief for Claude Code / Codex / Gemini)

### 2. Competitive action export
The dashboard and SQLite data already contain enough information to generate a condensed **you vs competitor → action list**.
Prefer exports that say:
- what competitors have
- what the target lacks
- why it matters
- what page/edit should be built next

This should be execution-first, not just descriptive analytics.

### 3. Suggestive use cases (important)
The local crawl + extraction database is a **general intelligence layer for builders**, not only an SEO report source.
Agents should use it to suggest:
- documentation pages to create
- product pages to improve
- landing pages to build
- onboarding gaps to close
- feature comparison pages
- developer trust pages (rate limits, auth, security, changelog, quickstarts)

Implement this in **two forms**:
- a CLI surface (for example a future `suggest-usecases <project>` command)
- an agent-native pattern where strong cloud models query the SQLite data directly and reason from it

The CLI version makes the feature discoverable and pipeline-friendly.
The agent-native version allows deeper open-ended reasoning from the same local dataset.

### Working principle
Do **not** build one rigid hook for every scenario.
Instead, teach agents to leverage the DB correctly:
- inspect competitor docs/guides/reference coverage
- compare against target coverage
- synthesize concrete page/build/use-case recommendations
- export them into implementation briefs

This is especially strong for docs, product pages, landing pages, onboarding, dashboards, and competitive positioning work.

## Google Search Console

Upload GSC data for ranking insights:

1. Go to https://search.google.com/search-console
2. Export Performance data as CSV (date range: last 3 months recommended)
3. Place CSVs in `gsc/<project>/` or upload via the web wizard
4. Or connect via OAuth: `node cli.js auth google`

---

## Auto-Update Protocol

SEO Intel checks for updates in the background (cached 24h). On new version:

**Security patches** → apply immediately:
```bash
cd ~/.openclaw/skills/froggo-seo-intel/seo-intel && npm install seo-intel@latest
```

**Feature updates** → notify user, apply on confirmation:
```
SEO Intel update available: 0.2.0 → 0.3.0
Changelog: [description]
Apply now? (Won't affect your data or configs)
```

**Manual check:**
```bash
node cli.js update
node cli.js update --apply   # auto-apply
```