Web scraping via progressive escalation and social media platform scrapers. USE WHEN scraping, crawl, scrape URL, bot detection, CAPTCHA, spider.
Scanned 6/6/2026
Install via CLI
Skills Directoryopenskills install danielmiessler/Personal_AI_Infrastructure---
name: scraping
description: Web scraping via progressive escalation and social media platform scrapers. USE WHEN scraping, crawl, scrape URL, bot detection, CAPTCHA, spider.
metadata:
author: pai
version: 1.0.0
---
# Scraping Skill
## Progressive Escalation
1. **Direct fetch** — curl/fetch with standard headers
2. **Browser rendering** — Headless browser for JS-heavy sites
3. **Proxy rotation** — For rate-limited or geo-restricted content
4. **Specialized APIs** — Platform-specific scrapers
## Rules
- Respect robots.txt
- Rate limit requests
- Handle pagination
- Extract structured data
- Store raw + processed versions
No comments yet. Be the first to comment!
