Multisite crawl without subdirectory adress
-
Hey guys,
N.B. that my problem is different to this one:
https://www.ads-software.com/support/topic/subdomains-indexed-by-google-while-mapped-to-domainbecause I have subdirectory *not subdomains* install of my multisite sites. I guess that′s why it either works totally different or I can′t keep the same syntax in the htaccess.
I want to stop google to index my subdirectory link like https://motherpage.com/daughtersite because I already mapped this to a TLD (unfortunately AFTER the site has already been indexed).
So I tried to adopt the common trick proposed in the link above:my htaccess
RewriteCond %{HTTP_HOST} =oled-tv-testcenter.de/bosch-akkuschrauber-test RewriteRule ^robots\.txt$ /robots-subdomains.txt [L]
my robots-subdomains.txt
User-agent: * Disallow: / User-agent: Googlebot Noindex: /
But it doesn′t work: Even when I advise google to crawl my subdirectory site oled-tv-testcenter.de/bosch-akkuschrauber-test again and then check the robots.txt tester it just shows the old robots.txt with the current date.
So I guess sth is wrong in my htaccess syntax? Some tiny bit:)
I also installed the plugin “Multisite Robots.txt Manager” to change the robots.txt per site. But when I let it show me the seperate robots.txt for the daughterpage, it shows it to me under the already mapped domain, so I guess it also doesn′t work for the subdir-option…
- The topic ‘Multisite crawl without subdirectory adress’ is closed to new replies.