Blocks when Tested, but Not Blocking Bad Bots

  • Resolved nevrsmer

    (@nevrsmer)


    4 years, 3 months ago

    Question for Blackhole

    Hello Jeff,

    I’m hoping you can help determine why Blackhole for Bad Bots Pro is not blocking bad bots on our site.

    I’ve:

    1. Added the code below to robots.txt and confirmed it using both methods on page: https://searchenginepromotionhelp.com/m/robots-text-tester/robots-checker.php (Results: Congratulations! No errors were found.)

    User-agent: *
Disallow: /?blackhole
# END

    2. Confirmed the following source code is on pages on our site:

    Do NOT follow this link or you will be banned from the site!

    3. Opened a page using Tor in the browser “Brave”, inspected the page, and clicked on the link above.

    4. This opened the warning page with this text in the opening part:

    “You have fallen into a trap!
    This site’s robots.txt rules explicitly forbid your presence at this location. Because you did not obey the robots.txt rules, further requests from your IP Address will be denied access. If you feel this is a mistake, you can access the site via proxy service and contact the administrator.

    Your Information
    IP Address: 195.176.3.24
    Host Name: tor5e3.digitale-gesellschaft.ch”

    5. Refreshed the page once and resulted in a “banned” notice. I was not able to access any other pages on the site while using that IP via Tor.

    6. In BBB > Bad Bots, there was an entry for the IP and Host name I had used via Tor

    This shows that all should be working. However, I’ve had the plugin installed for weeks, and it has not blocked on “bad bot”.

    I also have Banhammer Pro installed (as well as BBQ Pro). According to the logs in Armory, lots of bots that are not whitelisted, frequently scan our site. One example is a bot with IP 114.119.164.223 and User Agent “PetalBot”. There are various sequential entries for this bot for the same path, all of which show a 200 server response, meaning they did scan that same page various times. This has occurred with many other bots. This is just one example.

    Not one of these bots has been blocked by BBB.

    These are the relevant settings for BBB:

    Logged-in Users: Enabled for testing,
    Threshold: 1
    Whitelisted Bots and IPs: Default settings.

    I reset the settings to default to see if that was the issue. It still is not blocking any bad bots, and there have been quite a few bad bots scanning our site.

    Also, no application caching plugin is used. We are using only Autoptimize with Perfmatters and Fresould Deactiate Plugins for optimization, none of which have browser caching. We also haven’t yet added a CDN. The only caching we are using is on the server – Varnish and Memcached.

    What do you think the issue could be? How do you recommend making it work?

    PS: I see on the WP Plugins site the most current version of BBB is 2.9. The version of BBQ Pro we have installed is 2.7. Is the version of BBB free the same version as BBB Pro? I have not received any notifications of an update.

    Thank you for your time and have a good day.

    • This topic was modified 4 years, 3 months ago by nevrsmer.

    The page I need help with: [log in to see the link]

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Author Jeff Starr

    (@specialk)

    Hey @nevrsmer, I’m glad to help but the forum rules prohibit any discussion of pro/premium plugins (it’s for free versions only). For help with pro versions, send an email via the contact form at Plugin Planet and I’ll get back to you asap.

    Also just FYI, if it is not too late to edit your post (above), I would recommend removing/deleting or obscuring any/all identifying infos, like IP Address and Host Name, etc. If you can not edit the post, maybe ask the forum moderators to help. Just not a good idea to post that sort of information publicly.

    Thread Starter nevrsmer

    (@nevrsmer)

    Hello Jeff,

    My apologies. I did not know that about premium plugins. Sorry!

    The identifying info above does not identify anything regarding our site. That IP and host name are what the warning page generated for the page we used in Tor. It does not link back to us. Thanks for your concern.

    I’ll email you directly.

    Thanks and have a good night!

    aha..PetalBot. This is one is obeying nothing, and I have several security plugins in place.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Blocks when Tested, but Not Blocking Bad Bots’ is closed to new replies.