robots.txt file not updating

  • Resolved italialiving

    (@italialiving)


    4 years, 11 months ago

    How long does it take for robots.txt file to update? I added the blackhole line of code to my txt file and uploaded it to my server. I can see the updated version on my server, however when I go to the URL for the robots.txt file it is still showing the old one without the added line. Does it not update immediately?

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Author Jeff Starr

    (@specialk)

    Yeah robots.txt is just a plain text file, so any changes should be reflected immediately. Sounds like you may want to ask your web host, maybe there is something caching the files or whatever, I have no idea.

    Let me know if you have any questions about the plugin itself, glad to help however possible.

    Thread Starter italialiving

    (@italialiving)

    Thanks Jeff. I spent hours with my hosting company as well as other support and have gotten no where. They manually cleared the cache on my server. I can see the robots.txt file on my server with the added Blackhole line but when go to the URL for the robots.txt file it is not there.

    Will the plugin still work without that line in the txt file? I see I have been receiving a few emails of Bad Bot Alerts.

    Plugin Author Jeff Starr

    (@specialk)

    Technically yes the plugin will continue to work, but the chances for false positive may be higher without robots.txt. The blackhole trigger link includes the nofollow attribute, which good bots will obey and bad bots will ignore. So the plugin should remain effective, but the robots.txt rules still are recommended.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘robots.txt file not updating’ is closed to new replies.