Very slow scan and lags my entire site

Thanks for providing this feed back with enough details that I think I can help you. I would like to start by offering a counterpoint to your assessment of the potential lagging/downtime that you associate with the scan process. Don’t worry, I’m not trying to dismiss the very real and valid concerns you have about the performance impact on your site. I am well aware that, as with any active scanning process, there will be some amount of impact on the server’s resources and that could affect the overall performance of the site while the scan is running, however, I have engineered the Complete Scan specifically with this in mind and I do not feel that it should be having the impact that you have presented. I would like to address what I have identified as three separate issues here and get you followup feedback on each one if you are willing:

1. Duration of the Complete Scan: I can see from the screenshot that you provided that your were less than half way through the scan and was was projected to take around 5 or 6 hours from start to finish. This is partially owing to the seemingly large number of files in the scan path (it looks like you have around 6 times as many directories on your site as a normal WordPress site might). I will acknowledge that the scan time in the screenshot is unusually long even for the number of files to be scanned, and it doesn’t sound like your server is too under-powered for the job so I can’t say why that might be without seeing it run “in-person”. As for the number of sub-directories, I cannot say why you have so many, but I can throw out some guesses and you can get back to me and let me know if any of these ideas helped.

• You might have other site installed on the server inside the public_html directory of this main site. You can try excluding those directories by listing them in the “Skip directories” field.
• You might have a “cache” directory, possibly created by some caching plugin, that has thousands of old and unnecessary files. You should always clear your cache directory when scanning for malware to save time but also because the malware can sometime linger in your cache even after the threat was removed.
• You might also have an archive or backup or multiple copies of the sites directory tree in your path for any number of reasons that could be responsible for tying up the scan with duplicate folders.

2. when you visit your site while running the scan, it lags everything and makes the site inaccessible to you: I admit that I going to make some assumptions about your experience here but I do have a lot of first hand experience with this exact issue and I think I can explain with giving TMI or boring you with the technical details of why the site appears slow to you while scanning. The simplest way to put it is that your browser only wants to load one connection to your site at a time, so it you have the scan running in one tab and you open up your site in another tab or window that it usually queues up the new window and waits for an opening to make the new string of requests to your site that would load some other page on your site (this can sometime take an uncomfortable amount of time or even timeout and fail to load at all). Rest assured that your server does not operate in this manner and it should be capable of loading numerous simultaneous connection to multiple clients at the same time that you are running the complete scan in your browser. For your own peace of mind, and to prove or disprove conclusion here I would like you to have the complete scan open and running on one computer and then open your site up on a completely different computer or device (maybe your phone or tablet). and see if the site loads as expected on the second device.

3. reports from uptime checks setup with site24x7: This one is actually the most concerning to me and so I sincerely hope that you have stayed with me through all my ramblings and still have enough energy to followup with some more info on the for me, as I feel it could shed some light on things and might really help me improve my plugin. I use uptimerobot.com not uptime.com (site24x7) but I believe they have similar reporting details. Could you send me the “Root Cause Analysis” from that site or a screenshot of the down-times and response times for before, during, and after the scan?

You can email me directly if there is anything that you don’t want to post on this public forum:
eli AT gotmls DOT net

Hi Eli,

Thanks for your response.

If my scan time and progress look normal and as intended because of the number of files, then I’m fine with that. There are no cache files or compressed files included in the scan, I have just a large Woocommerce site with several plugins which could be the reason maybe (?). I think most of the time is spent on the plugins folder and I have seen gotmls can take several minutes scanning a certain file as I was observing it from last time, is this normal?

Yes, the plugin was not really using so much of the resources, and often times just utilize 1 CPU core (I’m on a 6-core VPS). That’s why I’m not sure why it slows down my site as I was visiting it in a different browser with a VPN while the scan is in progress.

I’m sorry for the confusion but I did not mean it created downtimes, I mean I am using site24x7 for Real User Monitoring and I set a warning when my site loads in over 15 seconds which it does when I do the scan. Could be okay if it’s just a brief scan time but would be hurtful, especially it’s an ecom site, for a scan time of 5-6 hours.

Taking a long time to scan a single file or folder (if it looks like it stalled for 30-60 seconds) is not normal and could be an indication of the root problem that is causing the scan to take so long. I am not convinced that the scan time you are seeing is normal at all. Furthermore, I think that whatever is causing the scan to hang on certain folders is also contributing to the overall latency of your site and it’s ability to load multiple pages at one time.

Any individual PHP process is typically not capable of utilizing more that one processor, and my scan process is designed to stagger the scan segments in a linear queue so that only one batch is called at a time. This gives other processes a turn in the queue thereby allowing your site to answer normal requests while the scan is running. Even if your server only had one core it should be able to share with at least 5 simultaneous instances of PHP running at the same time (with each getting about 20% of the CPU) with out too much performance degradation. Each PHP call typically last only 1 to 4 seconds for the average page load request, with some intensive processes (like wp-cron calls, or backup jobs, or file scans) lasting longer. Any given directory scan by my plugin should only take about 1 to 10 seconds to execute, if it is taking more than 30 seconds to scan a given folder then there is definitely something else going on here that needs to be addressed.

I would like to look at the load time trends on your up-time monitor showing the fluctuation you witnessed during the scan process and also a base-line of the load time when your server is not busy or at rest and there is minimal activity on your site.