Crawlee Smart Crawler Agent

Name: Crawlee Smart Crawler Agent
Author: agentskillexchange

ASecurity

Implements intelligent web crawling using the Crawlee framework with adaptive request routing between CheerioCrawler and PlaywrightCrawler. Manages request queues, handles anti-bot challenges, and exports structured data.

36 stars
0 votes
0 copies
1 view
Added June 8, 2026

ai-agentsjavascriptpythongojavanodegitapisecuritydocumentation

Works with

cursor
cli
api

Security analysis

A92/100

mediumInstalls packages at runtime which could introduce malicious dependencies

Pro shows the line behind each finding and how to fix it

Scanned June 8, 2026

npx -y skills add agentskillexchange/skills --skill crawlee-smart-crawler-agent --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Crawlee Smart Crawler Agent?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Crawlee Smart Crawler Agent

[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/agentskillexchange-crawlee-smart-crawler-agent/badge)](https://www.skillsdirectory.com/skills/agentskillexchange-crawlee-smart-crawler-agent)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro

SKILL.md

---
name: "Crawlee Smart Crawler Agent"
slug: "crawlee-smart-crawler-agent"
description: "Implements intelligent web crawling using the Crawlee framework with adaptive request routing between CheerioCrawler and PlaywrightCrawler. Manages request queues, handles anti-bot challenges, and exports structured data."
github_stars: 22922
verification: "security_reviewed"
source: "https://github.com/apify/crawlee"
author: "Apify"
category: "Research & Scraping"
framework: "Cursor"
tool_ecosystem:
  github_repo: "apify/crawlee"
  github_stars: 22922
  npm_package: "crawlee"
  npm_weekly_downloads: 96161
---

# Crawlee Smart Crawler Agent

Implements intelligent web crawling using the Crawlee framework with adaptive request routing between CheerioCrawler and PlaywrightCrawler. Manages request queues, handles anti-bot challenges, and exports structured data.

## Installation

Use the upstream install or setup path that matches your environment:
- npx crawlee create my-crawler
- npm start
- npm install crawlee playwright
- npm install crawlee@next

Requirements and caveats from upstream:
- Do you prefer 🐍 Python instead of JavaScript? [👉 Checkout Crawlee for Python 👈](https://github.com/apify/crawlee-python).
- Crawlee requires **Node.js 16 or higher**.

Basic usage or getting-started notes:
- We recommend visiting the [Introduction tutorial](https://crawlee.dev/js/docs/introduction) in Crawlee documentation for more information.
- ### With Crawlee CLI
- The fastest way to try Crawlee out is to use the **Crawlee CLI** and choose the **Getting started example**. The CLI will install all the necessary dependencies and add boilerplate code for you to play with.

- Source: https://github.com/apify/crawlee
- Extracted from upstream docs: https://raw.githubusercontent.com/apify/crawlee/HEAD/README.md

## Documentation

- https://crawlee.dev/

## Source

- [Agent Skill Exchange](https://agentskillexchange.com/skills/crawlee-smart-crawler-agent/)