Imitate GoogleBot - Google's URI Crawler

[mike_mybb_import3284]

For this tutorial you need the following extension(s)

User Agent Switcher - https://addons.mozilla.org/firefox/59

IMITATE GOOGLEBOT - GOOGLE'S URI CRAWLER

1. Install the extension(s) listed above.

2. Restart Firefox.

3. Open the Tools Menu, Highlight User Agent Swicher, Highlight Options, Then click Options.

4. In the Options Dialogue, select User Agents.

5. Click Add.

6. In The Name box on the Add User Agent dialogue, type Googlebot 2.1

7. In the User Agents box on the Add User Agent dialogue, type Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) .

8. Click OK.

9. Close the Options Dialogue.

10. Open the Tools Menu.

11. Highlight User Agent Switcher.

12. Select Googlebot 2.1

Congratulations, you should now be identifyed as a GoogleBot! To Test:

1. Log out of these Forums.

2. Scroll down to the Who's Online List. Check if it says "GoogleBot" there.
If not, go back to the tutorial and check for mistakes.

3. If it is there, Click Complete List.

4. Check if it Says Viewing Who's Online under GoogleBot.
If so, Congrats! It worked!
If not, go back to the tutorial and check for mistakes.

If you want to switch back to being identfied as a Firefox user, go to Tools > User Agent Switcher > Default.

If their is a mistake, or a part you don't understand, reply to this post. I will try to fix/answer your notice.

[Dr Small]

Very cool.
I tried it, and it works.

You can also imitate Yahoo! Slup

Code:

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

Dr Small

[Dr Small]

I got Googlebot, Yahoo! Slurp and MSN Search User Agents, so download the xml file, and import using the User Agent Switcher Extension, and then you can imitate either one of those


PS> You need to download the attachment below - v v v v v v
Dr Small

[mike_mybb_import3284]

Update:
Added Ask Jeeve's Crawler.

[Dr Small]

Update:

Not an XML file, but here's some webpages I found, that you can get a LOAD of different user agents of bots from:
http://www.psychedelix.com/agents/index.shtml
http://support.free-conversant.com/2701

Dr Small

[Dr Small]

Here is a list of the bots that MyBB Forums will recognize.
You can use just the text on the left (taken from the php file) as your user agent.
MyBB will automatically display you as that bot then.

PHP Code:

        
        'google' => 'GoogleBot',

        'lycos' => 'Lycos.com',

        'ask jeeves' => 'Ask Jeeves',

        'slurp@inktomi' => 'Hot Bot',

        'whatuseek' => 'What You Seek',

        'is_archiver' => 'Archive.org',

        'scooter' => 'Altavista',

        'fast-webcrawler' => 'AlltheWeb',

        'grub.org' => 'Grub Client',

        'turnitinbot' => 'Turnitin.com',

        'msnbot' => 'MSN Search',

        'yahoo' => 'Yahoo! Slurp' 


[blh765]

Cool. Thanks for the tutorial!

[FirefoxWiz]

Watch the spam. I edited your post.

[BETA_mybb_import3230]

Wouldn't that be mean to go around to forums as "Googlebot" and make people think you were actually googlebot caching their site?

[Papa Spot]

Wouldn't that be mean to go around to forums as "Googlebot" and make people think you were actually googlebot caching their site?

Yeah, I don't really see the point m'self.