Builds scalable web collection pipelines with Crawlee, Apify’s open-source crawling and browser automation library. Useful for request queue management, Playwright or Puppeteer crawling, structured dataset export, and resilient scraping across large sets of pages.
Installs into .claude/skills of the current project.
Are you the author of Crawlee Web Crawling Browser Automation Library?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/agentskillexchange-crawlee-web-crawling-browser-automation-library)
---
name: "Crawlee Web Crawling and Browser Automation Library"
slug: "crawlee-web-crawling-browser-automation-library"
description: "Builds scalable web collection pipelines with Crawlee, Apify’s open-source crawling and browser automation library. Useful for request queue management, Playwright or Puppeteer crawling, structured dataset export, and resilient scraping across large sets of pages."
github_stars: 22591
verification: "security_reviewed"
source: "https://github.com/apify/crawlee"
category: "Research & Scraping"
framework: "Multi-Framework"
tool_ecosystem:
github_repo: "apify/crawlee"
github_stars: 22591
npm_package: "crawlee"
npm_weekly_downloads: 72475
---
# Crawlee Web Crawling and Browser Automation Library
Builds scalable web collection pipelines with Crawlee, Apify’s open-source crawling and browser automation library. Useful for request queue management, Playwright or Puppeteer crawling, structured dataset export, and resilient scraping across large sets of pages.
## Installation
Use the upstream install or setup path that matches your environment:
- npx crawlee create my-crawler
- npm start
- npm install crawlee playwright
- npm install crawlee@next
Requirements and caveats from upstream:
- Do you prefer 🐍 Python instead of JavaScript? [👉 Checkout Crawlee for Python 👈](https://github.com/apify/crawlee-python).
- Crawlee requires **Node.js 16 or higher**.
Basic usage or getting-started notes:
- We recommend visiting the [Introduction tutorial](https://crawlee.dev/js/docs/introduction) in Crawlee documentation for more information.
- ### With Crawlee CLI
- The fastest way to try Crawlee out is to use the **Crawlee CLI** and choose the **Getting started example**. The CLI will install all the necessary dependencies and add boilerplate code for you to play with.
- Source: https://github.com/apify/crawlee
- Extracted from upstream docs: https://raw.githubusercontent.com/apify/crawlee/HEAD/README.md
## Source
- [Agent Skill Exchange](https://agentskillexchange.com/skills/crawlee-web-crawling-browser-automation-library/)
