gitpulse

crawling

6 个已发布仓库。

仓库
1
D4Vinci/Scrapling 🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
103.2k +0.5%+3.6%+18.0% 135
2
scrapy/scrapy Scrapy, a fast high-level web crawling & scraping framework for Python.
65.4k +0.0%+0.2%+0.9% 11
3
apify/crawlee Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
26.5k +0.0%+0.3%+1.2% 7
4
gocolly/colly Elegant Scraper and Crawler Framework for Golang
25.6k +0.0%+0.1%+0.2% 8
5
getmaxun/maxun Turn any website into a structured API. Extract, automate, search and monitor the web.
18.4k +0.3%+1.5%+3.4% 16
6
codelucas/newspaper newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
15.2k +0.0%+0.0%+0.1% 3