Sitemap and robots.txt conflicts

  • tombie

    (@tombie)


    4 months ago

    Hi!

    I have enjoyed this plugin with no problems. But recently I added a AI scrapper robots.txt. plugin and after some troubleshooting realized that if I activate your plugins “Add sitemap URL to the virtual robots.txt file”-function, it blocks out the function of the AI scrapper blocker plugin.

    So I unchecked it and everything seems fine.

    But now my robots.txt links wp-sitemap.xml instead of sitemap.xml. And the wp-sitemap.xml appears to be generated by your plugin, but is blank. Is there anything I can do to fix this without having to stop using the AI scrapper blocker?

    The page I need help with: [log in to see the link]

Viewing 4 replies - 1 through 4 (of 4 total)
  • Plugin Author Auctollo

    (@auctollo)

    Hi, would you please email us the debug results from your website?

    Thread Starter tombie

    (@tombie)

    I’m not very techsavy, but tried to follow a guide on how to debug wordpress and it didn’t generate a debug.log after I enabled it. So I’m a bit at a loss on how to send debug results. D:

    Do you think the problem can be solved if change sitemap to wp-sitemap in the plugin settings under “Override the file name of the sitemap: sitemap” ?

    For now I have deactivated the AI crawler plugin and clicked on Add sitemap URL to the virtual robots.txt file. until I can solve the issue.

    • This reply was modified 3 months, 3 weeks ago by tombie.
    Plugin Author Auctollo

    (@auctollo)

    There’s a button on the settings screen that generates the debug for us.

    Thread Starter tombie

    (@tombie)

    Oh, doh. I have now clicked on that debug (with both plugins turned on) and emailed to your support email. Interesting that the debug also generates information from plugins I deleted more than five years ago. Like Tentblogger, Monsterinsight, supercache and so on. I wonder what causes that code to be around still! :O

    Thanks in advance for your help and I hope emailing that debug was safe!

Viewing 4 replies - 1 through 4 (of 4 total)
  • You must be logged in to reply to this topic.