zeuspress

I have had Wordfence (free) installed on many sites for years. Your plugin has kept lots of bad guys from damaging my sites, and alerted me the few times when they got in. A plugin author that has that kind of track record of long time helpfulness, should be cut some slack when the occasional RARE oversight occurs. None of us is perfect, but this plugin is as close as you’re going to get for FREE.

To Newbies: Unless an update to a plugin or theme is an absolute must for security reasons, I always wait a day or 2 to see if lots of “I lost my site” or “I updated and got a blank screen” are posted in a plugins forum, right after an update. If there is, I don’t update until it’s straightened out. If there’s nothing out of the ordinary in the forum after 2 days, then I update. Has worked like a charm for me.

I have absolutely no connection to any plugin authors, except I use their hard work to make my websites/blogs better and safer for free. I just want to say thanks to the people that keep Wordfence one of the best FREE plugins available, on a day they’re taking a little heat. There’s a lot of us out there that really appreciate what you do. Keep up the good work and thank you.

OK, I researched it. This is what I found and this is what I did to solve the blocking problem. It’s easier than I thought.

What I found:
1) More and more people who know a lot more than I do about this subject, say that for most websites, a robot txt file isn’t even needed anymore. Why? Because the only bots that care about what it says are the good guys like Google that it’s blocking. The bad bots don’t say “I want to hack a site but look, they don’t want me to go to this file, shucks”. The robot txt file is just a suggestion, some bots don’t even bother to look at it.

2) Yoast (I don’t use that plugin anymore but I have a lot of respect for them) wrote that he doesn’t block WP Admin or WP includes at all anymore.
See: https://yoast.com/wordpress-robots-txt-example/
I went to yoast.com and looked at their /robots.txt and sure enough, they only block something to do with affiliates. In his post, he said that when you delete the robot txt file, WP will automatically at something. WP isn’t stupid.

So I tested it. I deleted my robot txt file that my seo plugin added and checked my /robots.txt and sure enough, a new file was there:

User-agent: *
Disallow: /wp-admin/

Sitemap: https://www.(removed for privacy).com/sitemap.xml.gz

I went to google webmasters and did a fetch as google AND render (you have to choose that option). Result? No more blocked css or java.

I must say I had the confidence to do this because I have Wordfence (free) an excellent plugin that was not the cause of this issue for me, and Bulletproof Security (free) another excellent security plugin and have made use of BPS’s custom codes. I also have a CDN with firewall (paid) that keeps some bad actors from even getting to my site.

I can’t guarantee this will work for you but it’s easy enough to try and test if you’ve signed up with Google Webmasters. It’s free and you just have to verify you own your website.

With method 2, won’t every hacker using a fake googlebot user agent have access to your admin files?

It worked! Just had to change the “turn off/on all sheduled backups” to off, even though I didn’t have any backups set up. Thank you.