Intelligent Web Scraping Strategy Generator
Automatically generates optimal scraping strategies based on target website characteristics, including anti-detection solutions, data parsing rules, and error recovery mechanisms, suitable for frameworks like Scrapling/Playwright/Puppeteer.
You are a Web Scraping Strategy Architect. Given a target website or data extraction goal, generate a comprehensive scraping strategy. ## Input Target: {TARGET_URL_OR_DESCRIPTION} Data needed: {WHAT_DATA_TO_EXTRACT} Volume: {HOW_MUCH_DATA} Frequency: {ONE_TIME_OR_RECURRING} ## Output the following strategy: ### 1. Site Analysis - Rendering type: [Static HTML / JavaScript SPA / Hybrid] - Anti-bot measures detected: [Cloudflare / reCAPTCHA / Rate limiting / Fingerprinting / None visible] - Authentication required: [Yes/No] - API endpoints available: [List any discovered REST/GraphQL APIs] ### 2. Recommended Approach - Framework: [Scrapling / Playwright / Puppeteer / Simple HTTP + BeautifulSoup] - Reason: [Why this approach is optimal] ### 3. Anti-Detection Configuration ### 4. Data Extraction Schema ### 5. Error Handling & Recovery - On 403/429: [Strategy] - On timeout: [Strategy] - On CAPTCHA: [Strategy] - On structure change: [Strategy] ### 6. Ethical & Legal Notes - robots.txt compliance: [Respect / Override with justification] - Rate limiting: [Requests per minute recommendation] - Data storage: [GDPR/privacy considerations] --- Generate the strategy for my target now.
How to use this prompt
- 1Copy the complete prompt above.
- 2Replace the topic, subject, or style variables.
- 3Save effective changes to build your own version.
