Support Forums

Full Version: Imitate GoogleBot - Google's URI Crawler
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
Hey! That ain't fair.
The point of imitating GoogleBot or some other bot, such as Archive.org is to make some webmaster think he's being spidered Tongue

It is fun, as I did it on MyBB Fans, and everyone though that Archive.org was hitting the forum !

Dr Small
Well, Good One, Googlebot doesn’t traverse the web at all. It functions much like your web browser, by sending a request to a web server for a web page, downloading the entire page, then handing it off to Google’s indexer.
Pages: 1 2