block a subdomain with .htaccess or robots.txt

  • I installed a subdirectory on my site: subdomain.mysite.com and this url appears in search results which I want to avoid since it’s only for members. I’m unable to use google remove tool because it will remove it for 90 days. Could someone offer a simple way to prevent this url from being crawled by search engine robots.

    I was reading https://moz.com/community/q/block-an-entire-subdomain-with-robots-txt but I’m unsure how to block this url from being crawled by search engine robots.

Viewing 1 replies (of 1 total)
  • Moderator James Huff

    (@macmanx)

    Just add a robots.txt file into the subdomain’s file directory with the following content:

    User-agent: *
Disallow: /

    Or, if the subdomain site is running WordPress, you can accomplish the same thing by checking “Discourage search engines from indexing this site” at Settings -> Reading in the site’s Dashboard.

Viewing 1 replies (of 1 total)
  • The topic ‘block a subdomain with .htaccess or robots.txt’ is closed to new replies.