Wordfence Ouf of Memory on big site with caching

  • Resolved z2zoop

    (@z2zoop)


    10 months, 4 weeks ago

    Hi folks,

    We finally managed to find the root cause why certain sites keep on going down, frequently taking the whole server with it. These are large websites which are frequently visited by all sorts of bots (some legit, many not) to a point they get more traffic from non-humans than from actual visitors.

    To speed it up we installed w3tc total cache, with Page cache set on Disk (enhanced), which uses .htaccess to serve the static content, has increased the performance significantly, however, sporadically, every 10 or 20 uncached pages that get visited result in an out of memory error (so this is almost every second due to the amount of traffic), the php error mentioning it is Wordfence running out. At such requests, the caching succeeds, the user get served a 500 error, but only after a refresh the cached content is shown.

    The problem seems to be gradually increasing, getting 500 errors more frequently. Disabling Wordfence has solved all these issues. When we investigate the database, we notice its keeping track of the new cache files being created (and probably scanned / checked), which is growing exponentially. Preferably, Wordfence should not be adding extra load at this point.

    PHP execution time is set to 60 seconds and a memory limit of 1024MB.

    Before we try to make a switch to a different plugin to see if those handle memory issues better, we figured we would first ask here. The site is also behind a cloudflare, so basically we mostly only want malware protection, likely could do without the firewall if its causing this much extra load.

    Any idea’s or suggestions to make Wordfence less demanding even if it’s at the cost of live protection?

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Support wfphil

    (@wfphil)

    Hi @z2zoop

    There are two types of memory error that we see, with the second one being much rarer (where xxxxx is a numerical value):

    Fatal error: Allowed memory size of xxxxx bytes exhausted (tried to allocate xxxxx bytes)

    Fatal error: Out of memory (allocated xxxxx) (tried to allocate xxxxx bytes)

    The first error above occurs when WordPress, the theme and all plugins on a site consume more memory from the web server than is allowed by the PHP directive memory_limit.

    However, your error specifically mentioned “out of memory” which matches the second error above. This is very likely to be memory exhaustion occurring outside of PHP somewhere on the server and you will need to ask your hosting provider to fix this as they will need to analyse the server operating system log files and the web server log files.

    Thread Starter z2zoop

    (@z2zoop)

    Both PHP value and WordPress Config were set to 1024M, confirmed by checking the site health screen.

    During a test having it set to even 2048M, one of the errors we caught reads “Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 2400848 bytes) in ~/public_html/wp-content/wflogs/rules.php on line 4937.

    The 134217728 bytes translates to roughly 128M, which seems to be a current limit when wordfence is running. Some digging around, 128M is the PHP default setting. In phpinfo() it mentions 2048M as the local value and 128M as the master value. So it’s weird for Wordfence to be using the wrong limit (it should be running as the same process as anything else php). Any ideas?

    This is 1 site on 1 server, so we can increase the global setting and see if that makes a difference.

    Regardless, my original question about making Wordfence less demanding still stands, wanting to allocate more than 128M per view when a frontend user loads a page is simply too much. Can it be set to ignore the newly created cache files? Or at least dont log the file changes, these cache files usually dont exist longer than an hour, so it’s of no use for wordfence to keep track of them.

    Plugin Support wfphil

    (@wfphil)

    Hi @z2zoop

    The PHP error is correct and your web server is only allowing 128MB of server RAM memory to be used for PHP processes so please ask your hosting provider to set the memory_limit PHP directive to 256MB.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Wordfence Ouf of Memory on big site with caching’ is closed to new replies.

Tags