IP Blocks for crawlers for server IP

  • Resolved spiros

    (@spiros)


    6 years, 6 months ago

    I get a lot of: “Exceeded the maximum number of requests per minute for crawlers” with a Block Count of 4361. Checking the IP, it is the IP of my server/site. That does not make much sense to me. How could it be? Is the site being crawled by itself?

Viewing 4 replies - 1 through 4 (of 4 total)

  • Hello @spiros,
    The first thing I would examine is to verify the How does Wordfence get IPs value is set to the correct option.

    If the setting on how Wordfence gets IPs is set incorrectly, the server will see all incoming IP addresses the same way and apply the same “Rate Limiting Rules” to all IPs potentially blocking a request from a valid IP address.

    Can you verify that you are running the latest 7.1.7 version of Wordfence? We made an update in version 7.1.6 to automatically whitelist server’s own IP for known safe requests.

    Let us know if adjusting those settings resolves the issue for you.

    Thanks,
    Steve

    Thread Starter spiros

    (@spiros)

    Hi Steve,

    How does Wordfence get IPs
    Let Wordfence use the most secure method to get visitor IP addresses. Prevents spoofing and works with most sites. (Recommended)

    I see it detects my PC’s IP below (Detected IPs), but I do not see server ip there.

    Version 7.1.7

    @spiros,
    Can you please send me a screenshot of your Live Traffic page? I want to see if there is another internal request from a separate plugin being blocked by the firewall. To ensure you don’t have the issue of your server IP address getting blocked again , I would whitelist that IP address.

    Thanks,
    Steve

    Hello!

    I hope we were successful in helping you resolve your issue with Wordfence! Since we have not heard back from you in the past 2 weeks I will now be marking this support thread as resolved. However, if we still haven’t resolved your issue please reach out to us as we would be more than happy to further assist you!

    Thanks and have a great day!
    Chloe

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘IP Blocks for crawlers for server IP’ is closed to new replies.