Robots.txt Is Invalid

  • Resolved supertrooper2

    (@supertrooper2)


    1 year, 2 months ago

    Although the RankMath SEO Analyzer says that our site has a valid robots.txt file, Google Page Speed Insights shows the following error message: “robots.txt is invalid 1,699 errors found”

    The content of the robots.txt is as follows:

    user-agent: *
    allow: /wp-admin/admin-ajax.php
    sitemap: https://websiteURL/sitemap_index.xml
    disallow: /wp-admin/
    disallow: /wp-includes/

    Please advise how this issue can be resolved.

    The page I need help with: [log in to see the link]

Viewing 1 replies (of 1 total)
  • Plugin Support Rank Math Support

    (@rankmathteam)

    Hello @supertrooper2,

    Thank you for contacting support.

    If you are using our plugin to generate this file please note that we use a dynamic file generation and since Google PageSpeed Insights looks for the presence of a physical file it’s expected to see an error there.

    In case you want to test how this looks in the eyes of Google you should test with this instead: https://support.google.com/webmasters/answer/6062598?hl=en

    Don’t hesitate to get in touch if you have any other questions.

Viewing 1 replies (of 1 total)
  • The topic ‘Robots.txt Is Invalid’ is closed to new replies.