link validation plugin causing 503 errors

  • craigstanfield

    (@craigstanfield)


    8 years ago

    Hi All,

    We’re developing a link testing plugin for our site, all went great until we put it live on a site with wordfence running where upon we got all bar 230 internal links returning as a 503 error. I then went back to my dev machine enabled wordfence and ran locally but got no 503 errors (probably due to it not running as quick).

    The plugin itself first uses get_file_contents to download a page and parse for all links, once we have a list of links we use curl to test these links 10 at a time in parallel before saving the resulting status codes and details in the database table. The first part runs fine and gathers a list of 838 unique links used on the website, the curl process however runs good for the first 130 odd of these links the rest are 503, obviously I’m being blocked at this point but my question is surely with a local running script this should bypass wordfence right? I’ve added the websites ip address my work ip address and 127.0.0.1 to the whitelisted ip list in wordfence settings. I feel making other settings less strict would be a bad idea as we are very happy with how wordfence runs on this site.

    So presuming that I have no way around this bar refactoring already working fast code the other option we have is temporary disable wordfence somehow while this script runs. Anyone have any suggestions and does anyone know how we can either bypass or trigger an event to temporarily disable wordfence and re-enable it upon completion of the script.

    Thanks

    Craig

Viewing 3 replies - 1 through 3 (of 3 total)
  • wfalaa

    (@wfalaa)

    Hi Craig,
    It’s not clear for me how exactly your plugin runs these cURL requests, but let me ask first, can you see any of these requests getting listed under (Wordfence > Live Traffic) when you set the filter to “Blocked” or “Blocked By Firewall”?

    Thanks.

    Thread Starter craigstanfield

    (@craigstanfield)

    Hi wfalaa,

    Yes this script is being shown in the live traffic log. I found that adding the server ip we progressed further but it wasn’t until i changed the number of tries until throttle that i was able to get the data in. With this in mind I would need a method either to adjust these values for the small time this runs and set them back afterwards or to disable it temporarily, the job itself runs from a cron so having an operative log on at a quiet time to change settings and press buttons is far from ideal.

    wfalaa

    (@wfalaa)

    I’m pretty sure turning the firewall status into “Learning Mode” should work as a treat in this case.

    Let me know how it goes,
    Thanks.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘link validation plugin causing 503 errors’ is closed to new replies.