PSA | Bytedance and Bytespider Bots | Recommend Blocking

  • Resolved Generosus

    (@generosus)


    1 year, 5 months ago

    *** PUBLIC SERVICE ANNOUNCEMENT ***

    I have thoroughly investigated the bots, Bytedance and Bytespider. As a result, I highly recommend blocking them. Main reasons:

    1. They do not respect robots.txt rules.
    2. They are using well-known hosting services to bypass normal blocking channels or methods. For example, they are using AmazonAWS services to crawl websites and mask (or re-route) their origin IPs. Details: https://prnt.sc/7Zst7eUZaxZ3
    3. Their crawling rates are extremely high.
    4. Blocking Bytedance and Bytespider via AS138699 and AS396986 does not help much.
    5. Typically, the bots’ origin IPs geolocation is China. When blocking the bots’ User Agents, the origin IPs geolocation changes to Singapore (another haven for malicious bots or bad actors).

    In short, I recommend blocking the User Agents, Bytedance and Bytespider, for best results.

    Hope this helps a bit.

    Cheers ??

Viewing 16 replies (of 16 total)
  • Thread Starter Generosus

    (@generosus)

    All,

    Another one: BYTEPLUS-AS-AP Byteplus Pte. Ltd. (AS150436)

    This malicious bot (Bytedance) is routing its queries through Singapore.

    Highly recommend blocking the above ASN as well. In fact, blocking Singapore too will be the best countermeasure you can apply against many malicious bot. Singapore loves to harbor these bots.

    Cheers!

    • This reply was modified 5 months, 1 week ago by Generosus.
Viewing 16 replies (of 16 total)
  • You must be logged in to reply to this topic.