• My company has recently launched the first of many sites built on top of WordPress multisite (WPMS). We want to use the Multisite Robots.txt Manager plugin but it’s not behaving as expected. I’d like to try and figure out why. Here’s some debugging info.

    WP on Ubuntu: (/var/www/html/efs/wordpress).
    Plugin version: 3.0.0
    Cleaner results:
    * Check For Old Robots.txt File Settings: No old robots.txt file data found.
    * Check For Real (physical) Robots.txt File: A physical robots.txt file was not found.
    * Check For Robots.txt Rewrite Rule: Proper Rewrite Rule found

    Network Robots.txt File contents (in the plugin)
    # robots.txt
    User-agent: *
    Disallow:
    {APPEND_WEBSITE_ROBOTSTXT}

    Site One Robots.txt Custom Append Rules
    User-Agent: *
    Disallow: /

    When I visit /robots.txt for “Site One” I get a 404 response. Manually checking the server and there is definitely no file there. If I manually create a file in the WPMS directory the request returns the contents of the file for both Site One and Site Two (as expected).

    Nothing I do in the plugin gets a robots.txt file to display, even though the plugin indicates that everything is working as intended.

Viewing 1 replies (of 1 total)
  • Thread Starter commadelimited

    (@commadelimited)

    Ping.

    I wanted to follow up on this post. The plugin isn’t working at all and I wanted to try and diagnose the issue. Is it perhaps something misconfigured on our end, in our system perhaps?

Viewing 1 replies (of 1 total)
  • The topic ‘Hitting /robots.txt returns a 404’ is closed to new replies.