Blocking Baidu & Yandex

  • Resolved Ben Kemp

    (@wrc-co)


    7 years, 8 months ago

    Hi Jeff

    I am perplexed as to why I can’t keep Baidu and Yandex out of my site. Perhaps you can suggest a different line of attack?
    1.)WordPress runs in the top level at gt-rider.com
    – Block Bad Bots is installed and both Baidu and Yandex removed from the whitelist. A couple of alerts on naughty bots have been received since I installed it a couple of days ago.

    2.) Xenforo Forums are in a subdirectory /se-asia-motorcycling/

    3.) I’ve added the Baidu crawler range to the WHM Blacklist – 180.76.15.1-180.76.15.255. Those are the only Baidu IP’s that are appearing.

    4.) I’ve added blocking code into the top of .htaccess files (root and forum);
    # block baidu + yandex bots
    <IfModule mod_rewrite.c>
    RewriteCond %{HTTP_USER_AGENT} baidu [NC]
    RewriteCond %{HTTP_USER_AGENT} yandex [NC]
    RewriteRule .* – [F,L]
    </IfModule>

    HOWEVER: Right now, there are 15 or so Baidu bots all over the place, and several Yandex bots – both having ignored the instruction to not index Forum Members…

    I’m perplexed as to how they get past the Server blacklist, then the .htaccess files. Do you have any thoughts on that?

    Kind regards
    Ben

Viewing 3 replies - 1 through 3 (of 3 total)
  • Thread Starter Ben Kemp

    (@wrc-co)

    Since then, I’ve tried blocking the Baidu IP address range on Cloudflare.

    That appears to have instantly ended their residency.

    There were up to 20 of their bots at any one time, with IP addresses in the same range, all hammering away at the site day and night.

    Plugin Author Jeff Starr

    (@specialk)

    Awesome, glad you got it sorted. I know bad bots can be a pain. In the Pro version of Blackhole for Bad Bots, you can block any IP address directly via the plugin settings. Either way, glad you resolved the issue.

    WRC Co, can you share the IP range you successfully blocked on Cloudflare?

    I can’t stand Yandex and Baidu and these bots do not give up!

    Thanks!

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Blocking Baidu & Yandex’ is closed to new replies.