This is a simple domain crawler built in ruby using nokogiri library
To crawl website geeksforgeeks.org, run
--> crawl_domain("http://www.geeksforgeeks.org","geeksforgeeks")
-
Notifications
You must be signed in to change notification settings - Fork 1
This is a simple domain crawler built in ruby. The primary purpose of this crawler is to get store all the valid urls(a tags) of any domain(website). Furthurmore, it tries to avoid certain redundant webpages which does not contain important content.
papriwalprateek/domain-crawler
About
This is a simple domain crawler built in ruby. The primary purpose of this crawler is to get store all the valid urls(a tags) of any domain(website). Furthurmore, it tries to avoid certain redundant webpages which does not contain important content.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published