• Hey guys,
    N.B. that my problem is different to this one:
    https://www.ads-software.com/support/topic/subdomains-indexed-by-google-while-mapped-to-domain

    because I have subdirectory *not subdomains* install of my multisite sites. I guess that′s why it either works totally different or I can′t keep the same syntax in the htaccess.
    I want to stop google to index my subdirectory link like https://motherpage.com/daughtersite because I already mapped this to a TLD (unfortunately AFTER the site has already been indexed).
    So I tried to adopt the common trick proposed in the link above:

    my htaccess

    RewriteCond %{HTTP_HOST} =oled-tv-testcenter.de/bosch-akkuschrauber-test
    RewriteRule ^robots\.txt$ /robots-subdomains.txt [L]

    my robots-subdomains.txt

    User-agent: *
    Disallow: /
    
    User-agent: Googlebot
    Noindex: /

    But it doesn′t work: Even when I advise google to crawl my subdirectory site oled-tv-testcenter.de/bosch-akkuschrauber-test again and then check the robots.txt tester it just shows the old robots.txt with the current date.

    So I guess sth is wrong in my htaccess syntax? Some tiny bit:)

    I also installed the plugin “Multisite Robots.txt Manager” to change the robots.txt per site. But when I let it show me the seperate robots.txt for the daughterpage, it shows it to me under the already mapped domain, so I guess it also doesn′t work for the subdir-option…

Viewing 1 replies (of 1 total)
Viewing 1 replies (of 1 total)
  • The topic ‘Multisite crawl without subdirectory adress’ is closed to new replies.