Alternative Scraping tools and utilities for Python
1
2
3
4
5
scrapy
Github stargazers
52918
Github forks
10
Commits
10209
Code contributors Contributors
564
Scrapy, a fast high-level web crawling & scraping framework for Python.
Created
Feb. 22, 2010
Updated
Sept. 10, 2024
License
bsd-3-clause
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
676
Homepage
twint
Read-only repository, archived by owner Archived
Github stargazers
15791
Github forks
2724
Commits
845
Code contributors Contributors
60
An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
Created
June 10, 2017
Updated
March 2, 2021
License
mit
Github repo
Type
Module/library
Primary Language, based on Github DataLanguage
Python
Issues
589
pattern
Github stargazers
8737
Github forks
1578
Commits
1434
Code contributors Contributors
19
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
Created
May 3, 2011
Updated
April 25, 2020
License
bsd-3-clause
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
173
Homepage
autoscraper
Github stargazers
6249
Github forks
656
Commits
137
Code contributors Contributors
7
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Created
Aug. 31, 2020
Updated
Sept. 24, 2024
License
mit
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
1
twitter-scraper
Read-only repository, archived by owner Archived
Github stargazers
3927
Github forks
602
Commits
208
Code contributors Contributors
28
Scrape the Twitter Frontend API without authentication.
Created
Feb. 22, 2018
Updated
Dec. 17, 2021
License
mit
Github repo
Type
Module/library
Primary Language, based on Github DataLanguage
Python
Issues
57
cloudflare-scrape
Github stargazers
3381
Github forks
459
Commits
156
Code contributors Contributors
23
A Python module to bypass Cloudflare's anti-bot page.
Created
Feb. 28, 2013
Updated
March 23, 2020
License
mit
Github repo
Type
Module/library
Primary Language, based on Github DataLanguage
Python
Issues
130
Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE
Github stargazers
3148
Github forks
545
Commits
615
Code contributors Contributors
24
Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!
Created
Sept. 12, 2020
Updated
Feb. 21, 2024
License
gpl-3.0
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
29
pagodo
Github stargazers
2796
Github forks
495
Commits
136
Code contributors Contributors
7
pagodo (Passive Google Dork) - Automate Google Hacking Database scraping and searching
Created
Aug. 19, 2016
Updated
June 8, 2024
License
gpl-3.0
Github repo
Type
Module/library
Primary Language, based on Github DataLanguage
Python
Issues
2
GitDorker
Github stargazers
2283
Github forks
419
Commits
113
Code contributors Contributors
4
A Python program to scrape secrets from GitHub through usage of a large repository of dorks.
Created
July 13, 2020
Updated
May 7, 2021
Github repo
Type
Tool/utility
Primary Language, based on Github DataLanguage
Python
Issues
20
lazynlp
Github stargazers
2159
Github forks
311
Commits
14
Code contributors Contributors
4
Library to scrape and clean web pages to create massive datasets.
Created
Feb. 27, 2019
Updated
Oct. 7, 2019
Github repo
Type
Module/library
Primary Language, based on Github DataLanguage
Python
Issues
10
JobFunnel
Github stargazers
1853
Github forks
216
Commits
413
Code contributors Contributors
13
Scrape job websites into a single spreadsheet with no duplicates.
Created
Aug. 25, 2017
Updated
Sept. 17, 2024
License
mit
Github repo
Type
Cli
Primary Language, based on Github DataLanguage
Python
Issues
8
ruia
Github stargazers
1749
Github forks
181
Commits
440
Code contributors Contributors
14
Async Python 3.6+ web scraping micro-framework based on asyncio
Created
July 10, 2018
Updated
Aug. 21, 2022
License
apache-2.0
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
8
Homepage
recipe-scrapers
Github stargazers
1720
Github forks
525
Commits
1257
Code contributors Contributors
182
Python package for scraping recipes data
Created
Sept. 14, 2015
Updated
Sept. 20, 2024
License
mit
Github repo
Type
Module/library
Primary Language, based on Github DataLanguage
Python
Issues
128
SoundScrape
Github stargazers
1417
Github forks
146
Commits
250
Code contributors Contributors
12
SoundCloud (and Bandcamp and Mixcloud) downloader in Python.
Created
Dec. 29, 2013
Updated
Nov. 22, 2020
License
mit
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
71
requests-ip-rotator
Github stargazers
1336
Github forks
139
Commits
44
Code contributors Contributors
6
A Python library to utilize AWS API Gateway's large IP pool as a proxy to generate pseudo-infinite IPs for web scraping and brute forcing.
Created
July 17, 2021
Updated
Nov. 13, 2023
License
gpl-3.0
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
3
Homepage
fansly
Read-only repository, archived by owner Archived
Github stargazers
1275
Github forks
63
Commits
187
Code contributors Contributors
3
Easy to use fansly.com content downloading tool. Written in python, but ships as a standalone Executable App for Windows too. Enjoy your Fansly content offline anytime, anywhere in the highest possible content resolution! Fully customizable to download in bulk or single: photos, videos & audio from timeline, messages, collection & specific posts 👍
Created
Oct. 8, 2021
Updated
Sept. 2, 2023
License
gpl-3.0
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
26
Homepage
OnionSearch
Github stargazers
1271
Github forks
163
Commits
44
Code contributors Contributors
4
OnionSearch is a script that scrapes urls on different .onion search engines.
Created
March 18, 2020
Updated
Dec. 29, 2023
License
gpl-3.0
Github repo
Type
Script
Primary Language, based on Github DataLanguage
Python
Issues
14
search-script-scrape
Github stargazers
1235
Github forks
227
Commits
56
Code contributors Contributors
1
101 real world web scraping exercises in Python 3 for data journalists
Created
June 7, 2015
Updated
Oct. 5, 2015
Github repo
Type
Script
Primary Language, based on Github DataLanguage
Python
Issues
2
Homepage
scrapy-cluster
Github stargazers
1180
Github forks
323
Commits
747
Code contributors Contributors
25
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
Created
April 14, 2015
Updated
April 7, 2021
License
mit
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
17
Homepage
django-dynamic-scraper
Github stargazers
1151
Github forks
311
Commits
552
Code contributors Contributors
9
Creating Scrapy scrapers via the Django admin interface
Created
Dec. 16, 2011
Updated
June 25, 2021
License
bsd-3-clause
Github repo
Primary Language, based on Github DataLanguage
Python
Issues
40
Homepage
1
2
3
4
5
Ad
Github Data
Privacy
Terms & Conditions
About
Contact
© 2020 rights reserved