Scrapy, a fast high-level web crawling & scraping framework for Python.

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Scrape the Twitter Frontend API without authentication.

A Python module to bypass Cloudflare's anti-bot page.

Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!

pagodo (Passive Google Dork) - Automate Google Hacking Database scraping and searching

A Python program to scrape secrets from GitHub through usage of a large repository of dorks.

Library to scrape and clean web pages to create massive datasets.

Scrape job websites into a single spreadsheet with no duplicates.

Async Python 3.6+ web scraping micro-framework based on asyncio

Python package for scraping recipes data

SoundCloud (and Bandcamp and Mixcloud) downloader in Python.

101 real world web scraping exercises in Python 3 for data journalists

A Python library to utilize AWS API Gateway's large IP pool as a proxy to generate pseudo-infinite IPs for web scraping and brute forcing.

Easy to use fansly.com content downloading tool. Written in python, but ships as a standalone Executable App for Windows too. Enjoy your Fansly content offline anytime, anywhere in the highest possible content resolution! Fully customizable to download in bulk or single: photos, videos & audio from timeline, messages, collection & specific posts 👍

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

Creating Scrapy scrapers via the Django admin interface

OnionSearch is a script that scrapes urls on different .onion search engines.