개요
Scrapling은 단일 요청에서 대규모 크래그에 이르기까지 지원하는 맞춤형 웹 스크래핑 프레임워크입니다. 그 해독기는 사이트의 변화를 배우고 페이지를 업데이트할 때 요소를 자동으로 리디렉션할 수 있습니다. Cloudflare Turnstile와 같은 스크래그 시스템을 방해하고 동시에 여러 세션 크래그 및 에이전트 라우팅을 지원합니다.
README 미리보기
\n\n\n \n \n \n \n \n \n \n Effortless Web Scraping for the Modern Web\n\n\n\n \n \n العربيه | Español | Português (Brasil) | Français | Deutsch | 简体中文 | 日本語 | Русский | 한국어\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n\n\n Selection methods\n ·\n Fetchers\n ·\n Spiders\n ·\n Proxy Rotation\n ·\n CLI\n ·\n MCP\n\n\nScrapling is an adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl.\n\nIts parser learns from website changes and automatically relocates your elements when pages update. Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scale up to concurrent, multi-session crawls with pause/resume and automatic proxy rotation - all in a few lines of Python. One library, zero compromises.\n\nBlazing fast crawls with real-time stats and streaming. Built by Web Scrapers for Web Scrapers and regular users, there's something for everyone.\n\n```python\nfrom scrapling.fetchers import Fetcher, AsyncFetcher, StealthyFetcher, DynamicFetcher\nStealthyFetcher.adaptive = True\np = StealthyFetcher.fetch('https://example.com', headless=True, network_idle=True) # Fetch website under the radar!\nproducts = p.css('.product', auto_save=True) # Scrape data that survives website design changes!\nproducts = p.css('.product', adaptive=True) # Later, if the website structure changes, pass `adaptive=True` to find them!\n```\nOr scale up to full crawls\n```python\nfrom scrapling.spiders import Spider, Response\n\nclass MySpider(Spider):\n name = "demo"\n start_urls = ["https://example.com/"]\n\n async def parse(self, response: Response):\n for item in response.css('.product'):\n