Cookiebot crawler doesn’t get blocked

  • Resolved yariwp

    (@yariwp)


    9 months ago

    Hello,

    Recently we noticed some downtime on our website due to a crawl from Cookiebot triggering a large amount of requests to trp-ajax.php. I’ve also attempted the admin-ajax fallback, but this still results in a very large amount of POST requests.

    The ‘Block crawlers’ option is enabled, but unfortunately it seems to miss the Cookiebot.

    The user agent and list of IP addresses for Cookiebot can be found here:
    https://support.cookiebot.com/hc/en-us/articles/360003824153-Whitelisting-the-Cookiebot-scanner

    I would appreciate it if this could be added to the crawler block list, as it’s a bit of a messy solution to manually patch it in.

    Thanks

Viewing 4 replies - 1 through 4 (of 4 total)
  • Andrei

    (@andrei202)

    Hey there,


    Thank you for approaching us with your concern.
    ??
    Block Crawlers simply tries to detect if it’s a bot and not automatically translate that particular page. Since bots can be varied, we can’t guarantee all will be detected.
    ?
    In regards to the large amount of POST requests to trp-ajax.php, you might want to try disabling the ‘Dynamic Translation’ from the Advanced settings section. This could potentially resolve the issue.
    ?
    Please let us know if disabling the dynamic translation helps
    ?
    Best Regards,

    Thread Starter yariwp

    (@yariwp)

    Hey Andrei,

    Will the disabling of the dynamic translations have any effect on my custom ACF fields? They still need to be translated.

    Thanks

    Thread Starter yariwp

    (@yariwp)

    It seems like our ACF fields are not affected, which is good. Unfortunately we are also translating some strings in the Google maps API, which unfortunately will no longer be translated with this option enabled.

    Hello there,

    We deeply apologize for the delayed response.

    There has been a great load on our support team in the past few days, but such a long reply time is very uncommon.

    Unfortunately at the moment we do not have any workaround available for this.

    Thank you for understanding and for your patience!

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘Cookiebot crawler doesn’t get blocked’ is closed to new replies.