Tip for allowing access to robots.txt

  • We had a use case with Multisite where the root blog is utilizing More Privacy Options to require you to be logged in, but there were some subsites (domain.org/foo) that were open to the public. This had the issue where traffic to domain.org/robots.txt was being intercepted by More Privacy Options.

    To fix this, I simply added this to .htaccess and things work as desired now.

    # Don’t muck with people trying to get robots.txt
    RewriteRule ^robots.txt – [L]

    Just sharing in case it’s helpful to others (or if they have a better idea)

  • The topic ‘Tip for allowing access to robots.txt’ is closed to new replies.