Viewing 1 replies (of 1 total)
  • Plugin Author George Pattichis

    (@pattihis)

    Hello,

    This plugin supports multisite if you’re running a subdomain configuration but unfortunately not with subdirectories.

    If you’re adding sites as subdirectories, you can only have one robots.txt file simply because web agents only look for robots.txt files at the root of a domain/web server.

    The robots.txt specification declares that the robots.txt file may exist only on the root folder: example.com/robots.txt. Outputting the file on a subdirectory isn’t permitted: example.com/invalid/robots.txt.

    If you use subdomains instead of subdirectories for your multisite, you can have multiple robots.txt outputs, such as at valid.example.com/robots.txt.

    From a technical perspective, a multisite installation with subdirectories is considered one large website in that regard. Thus you should consider adding your subsites’ rules as global rules in the main robots.txt

    Hope this helps!

Viewing 1 replies (of 1 total)
  • The topic ‘Error 404 on robots.txt file’ is closed to new replies.