Hitting /robots.txt returns a 404
-
My company has recently launched the first of many sites built on top of WordPress multisite (WPMS). We want to use the Multisite Robots.txt Manager plugin but it’s not behaving as expected. I’d like to try and figure out why. Here’s some debugging info.
WP on Ubuntu: (/var/www/html/efs/wordpress).
Plugin version: 3.0.0
Cleaner results:
* Check For Old Robots.txt File Settings: No old robots.txt file data found.
* Check For Real (physical) Robots.txt File: A physical robots.txt file was not found.
* Check For Robots.txt Rewrite Rule: Proper Rewrite Rule foundNetwork Robots.txt File contents (in the plugin)
# robots.txt
User-agent: *
Disallow:
{APPEND_WEBSITE_ROBOTSTXT}Site One Robots.txt Custom Append Rules
User-Agent: *
Disallow: /When I visit /robots.txt for “Site One” I get a 404 response. Manually checking the server and there is definitely no file there. If I manually create a file in the WPMS directory the request returns the contents of the file for both Site One and Site Two (as expected).
Nothing I do in the plugin gets a robots.txt file to display, even though the plugin indicates that everything is working as intended.
- The topic ‘Hitting /robots.txt returns a 404’ is closed to new replies.