Monthly Archives: November 2010

1.Save the code below as spyder.erl 2.Run as : erl spyder.erl  -s spyder start http://site_to_crawl.com -s init stop http://pastebin.com/Qw9f11bi This is a very crude crawler – will crawl all the links in a page, and then further. It doesn’t protect you … Continue reading →