I want to make my site not appear in any search engines

I want to make my site not appear in any search engines - Printable Version

+- Support Forums (https://www.supportforums.net)
+-- Forum: Categories (https://www.supportforums.net/forumdisplay.php?fid=87)
+--- Forum: Webmaster Support (https://www.supportforums.net/forumdisplay.php?fid=36)
+---- Forum: Domaining (https://www.supportforums.net/forumdisplay.php?fid=54)
+---- Thread: I want to make my site not appear in any search engines (/showthread.php?tid=7448)

Pages: 1 2

I want to make my site not appear in any search engines - Fallenour - 05-29-2010

How would I go about this? Is there any way I could make it so that no search engines can find my site?

RE: I want to make my site not appear in any search engines - timestandstill - 06-03-2010

Yes. Make a file called "robots.txt" and put it in your domain's top-level directory, so its accessible at yourdomain.com/robots.txt, then add this:

Code:
User-Agent: *

Disallow: /

RE: I want to make my site not appear in any search engines - Riverthief - 06-04-2010

If you don't mind saying, howcome?

RE: I want to make my site not appear in any search engines - KraZi - 06-04-2010

(06-04-2010, 07:12 AM)Riverthief Wrote: If you don't mind saying, howcome?

I was about to ask the same question?

Illegal content... or?

RE: I want to make my site not appear in any search engines - Fallenour - 06-07-2010

(06-04-2010, 07:56 AM)KraZi Wrote: I was about to ask the same question?

Illegal content... or?

1) Illegal content
2) Torrenting
3) Private closed community forum
4) I dont want people to be able to search for me at all, I want it to be completely hidden so that the only way possible to find it is if your invited Oui

RE: I want to make my site not appear in any search engines - Cida - 06-07-2010

If your host supports individual custom .htaccess you can make it so it can only be accessed with user and pass (there would be a prompt), search engines would have nothing to index, only the 401 error page.

http://www.elated.com/articles/password-protecting-your-pages-with-htaccess/

RE: I want to make my site not appear in any search engines - Fallenour - 06-07-2010

I was gonna do something far more hardcore, I was just gonna do a full range ip block, and add exceptions to the list as I went. 8)

RE: I want to make my site not appear in any search engines - Omniscient - 06-07-2010

Probably htaccess password would work best. Members would feel exclusive and it would protect your site very well.

RE: I want to make my site not appear in any search engines - Fallenour - 06-08-2010

ooo I like the look of this 8D. Does this stop them from even being able to contact the server though? Thats more along the lines of what Im aiming for, I dont even want them to be able to open a connection to it unless I want them to.

RE: I want to make my site not appear in any search engines - Cida - 06-08-2010

(06-08-2010, 05:51 AM)Fallenour Wrote: ooo I like the look of this 8D. Does this stop them from even being able to contact the server though? Thats more along the lines of what Im aiming for, I dont even want them to be able to open a connection to it unless I want them to.

It will be similar to a 403, but it will give them the opportunity to authenticate their self, if wrong, they will get a 401 error, the same as 403 access denied, but meaning the reason why is because authentication is needed.

I use it on my WAMP server, that I use for small projects and sending files usually.

You'd have it like this in your htaccess

Code:
AuthUserFile "C:\WebServer\.htpasswd"

AuthName "name of choice"

AuthType Basic

Require valid-user

And in the .htpasswd, you'd have it in this format

Code:
username:password

username1:password

Username is case sensitive as well.