An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Scrape the Twitter Frontend API without authentication.

A Python module to bypass Cloudflare's anti-bot page.

Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!

pagodo (Passive Google Dork) - Automate Google Hacking Database scraping and searching

Library to scrape and clean web pages to create massive datasets.

A Python program to scrape secrets from GitHub through usage of a large repository of dorks.

Async Python 3.6+ web scraping micro-framework based on asyncio

Scrape job websites into a single spreadsheet with no duplicates.

SoundCloud (and Bandcamp and Mixcloud) downloader in Python.

101 real world web scraping exercises in Python 3 for data journalists

Python package for scraping recipes data

Creating Scrapy scrapers via the Django admin interface

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

A Python library to utilize AWS API Gateway's large IP pool as a proxy to generate pseudo-infinite IPs for web scraping and brute forcing.

Non-official client to get some info about products sold on Amazon

OnionSearch is a script that scrapes urls on different .onion search engines.

source for Open States scrapers