Top 50 open source web crawlers
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

Comment only
 
 
ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
NameLanguagePlatform
2
HeritrixJavaLinux
3
NutchJava
Cross-platform
4
ScrapyPython
Cross-platform
5
DataparkSearchC++
Cross-platform
6
GNU WgetCLinux
7
GRUB
C#, C, Python, Perl
Cross-platform
8
ht://DigC++Unix
9
HTTrackC/C++
Cross-platform
10
ICDL CrawlerC++
Cross-platform
11
mnoGoSearchCWindows
12
Norconex HTTP CollectorJava
Cross-platform
13
Open Source ServerC/C++, Java PHP
Cross-platform
14
PHP-CrawlerPHP
Cross-platform
15
YaCyJava
Cross-platform
16
WebSPHINXJava
Cross-platform
17
WebLechJava
Cross-platform
18
AraleJava
Cross-platform
19
JSpiderJava
Cross-platform
20
HyperSpiderJava
Cross-platform
21
ArachnidJava
Cross-platform
22
SpindleJava
Cross-platform
23
SpiderJava
Cross-platform
24
LARMJava
Cross-platform
25
MetisJava
Cross-platform
26
SimpleSpiderJava
Cross-platform
27
GrunkJava
Cross-platform
28
CAPEKJava
Cross-platform
29
ApertureJava
Cross-platform
30
Smart and Simple Web Crawler
Java
Cross-platform
31
Web HarvestJava
Cross-platform
32
AspseekC++Linux
33
BixoJava
Cross-platform
34
crawler4jJava
Cross-platform
35
EbotErlandLinux
36
HounderJava
Cross-platform
37
Hyper EstraierC/C++
Cross-platform
38
OpenWebSpiderC#, PHP
Cross-platform
39
PavukCLunix
40
SphiderPHP
Cross-platform
41
XapianC++
Cross-platform
42
Arachnode.netC#Windows
43
CrawwwlerC++Java
44
Distributed Web CrawlerC, Java, Python
Cross-platform
45
iCrawlerJava
Cross-platform
46
pycreepJava
Cross-platform
47
OpeseC++Linux
48
AndjingJava
49
CcrawlerC#Windows
50
WebEaterJava
Cross-platform
51
JoBoJava
Cross-platform
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...
Main menu