MAX CPU Usage

@nicola.peluchetti Please contact me at nick [at] nick temple dot com and I can provide additional specifics.

@ntemple and @valprestone
one thing to be clear, the memory issue is not related to high cpu usage. The problem we discovered is that wordpress under some settings fetches all events even if they are paginated. But in our tests this doesn’t cause high cpu, because it’s a problem of data transfer size as also all meta content is loaded. We are working on a fix for this, but since this is more a wordpress problem it’s not an easy fix.
Then we have a problem with bots, which i hope my patch will fix ( if not we will need to add redirects ), which crawls lots of useless pages and which we know causes high cpu usage because many pages are accessed very rapidly.
If there are other and different issues, let us know

@nicola.peluchetti
@ntemple

Sorry if I’m interjecting on your conversation, but I figure in an instance like this, more data/info may help.

When I was on Host Gators Level 4 VPS…
Which includes:
CPU 1.98 GHZ
RAM 1344 MB
Disk Space 59 GB
Bandwidth 1050 GB

They told me…
The average cpu consumption of your server should be closer to 1.00 and you can see that it is averaging around 8.80:
07:00:01 AM 6 94 8.43 7.81 7.92
07:10:02 AM 7 102 8.28 7.71 7.77
07:20:01 AM 9 98 8.35 8.62 8.27
07:30:01 AM 8 101 8.97 8.32 8.20
Average: 8 100 8.84 8.90 8.90

Hope that helps you @nicola.peluchetti

Agreed there are three separate (yet interrelated) issues:

– High CPU usage
– High Memory Usage
– Bots spidering everything

And you are correct, I am somewhat “confusing” the issue, in part because they all have similar symptoms.

We, too upgraded our VPS, but that only helped for a little while. Due to various circumstances, I’m unable to provide specifics at this time.

@valpreston numbers definately help, thanks!

nicola.peluchetti Site went down today:( Urgent issues involve ai1ec and require code for line 70. I’ve emailed you details at the time.ly support line. Could you email me at valpreston at verizon dot net? Thanks in advance.

My host simply suggested to look for other calendar solutions, because “time.ly product has known reliability issues”

So you are recommending we add this:

Ok, the patch to have consistent url is the following one.

in file class-ai1ec-href-helper.php ( found in lib/router folder ) change lines from 13 to 25 so that they lok like this

private $used_paramaters = array(
		'action',
		'page_offset',
		'month_offset',
		'oneday_offset',
		'week_offset',
		'time_limit',
		'exact_date',
		'cat_ids',
		'auth_ids',
		'post_ids',
		'tag_ids',
	);

and this: https://support.time.ly/limiting-excessive-google-crawls/
??

Need a solution quickly now as getting hundreds of OOM errors in addition to the 100% CPU.

Has anyone had any luck with the suggestion above to add the code “in file class-ai1ec-href-helper.php ( found in lib/router folder ) change lines from 13 to 25 so that they look like this…”

We’ve already limited excessive google crawls and are still having issues.

Any suggestions of other calendars to look at? I love the functionality of ai1ec, but no matter how much CPU we add, it seems to suck right through it:(

I’m getting to the stage where, despite the large amount of work involved, the problems this plug-in is causing mean that I have to find another.

The 100%CPU I could live with as it didn’t seem to affect the functionality of the site. But the OOM errors are having a major impact on my users’ enjoyment of the site.

And the lack of reply to my question of two weeks ago is troubling to say the least.

Please please could we have a definite date for release of a patch for this, otherwise it will have to go.

PROBLEM IDENTIFIED and Temporary Working Fix: The plugins’ internal caching mechanism is to blame for the high CPU and Memory usage. I fixed my problem by going to the plugin menu to settings -> advanced -> advanced settings and changing the cache flush interval from 5 minutes (5m) to 3 hours (3h). You can use whatever setting you like, just make sure it’s much longer than 5 minutes.

I found the problem when I was rolling back my calendar to version 1.82. While backing up my current plugin folder using version 1.11, I noticed that the cache folder contained over 4,800 files and was nearly 400mb in size. My guess is that the caching mechanism has a bug and loops, causing the plugin to constantly regenerate cache files for each view type, for each category, and for each post/event. Each minute, the plugin was generating nearly 250 new cache files, or about 4 per second. It doesn’t seem to have flushed any of the old ones, because if it did, I wouldn’t have over 4,800 files with unique file names. If you have a lot of calendar events, this REALLY slows down your site and hogs resources. By setting the calendar to flush the cache every 3 hours, it will not continuously use these resources.

After the change, my cache folder is generating between 5-7 files per minute… much lower than 250 per minute. Having said that, I can’t tell if it’s purging old cache files when it generates new ones, as it’s not gone through all of my views, categories, or events yet… I’ll let you know what I find out.

The good news, doing this dropped my CPU usage from over 90% to less than 15%. Memory usage dropped by almost half, from 1.95 GB to 1.1 GB including cache/buffers.

My setup is as follows:

Linode 2GB VPS
Nginx 1.41
Percona SQL Server using XtraDB
PHP-FPM 5.4 with APC caching db requests and opcode via W3 Total Cache
Wordpress 3.52
All in One Event Calendar 1.11

If someone wants to try to fix this, the functions and scripts for generating the cache are in these files in the plugin directory:

\lib\strategy\class-ai1ec-persistence-context.php
\lib\strategy\cache_strategy\class-ai1ec-apc-cache.php
\lib\strategy\cache_strategy\class-ai1ec-db-cache.php
\lib\strategy\cache_strategy\class-ai1ec-file-cache.php
\app\helper\class-ai1ec-events-helper.php
\app\helper\class-ai1ec-events-list-helper.php

I fixed my problem by going to the plugin menu to settings -> advanced -> advanced settings and changing the cache flush interval from 5 minutes (5m) to 3 hours (3h).

I can’t find this option in advanced settings. The only options I have are:

Move calendar into this DOM element:
Skip in_the_loop() check
Turn this blog into an events-only platform
Strict event platform mode

Where might I find this cache option?

Has anyone tried upgrading to 1.11 version? Does it seem to fix the issues?

1.11 does not fix the issues, but it does give you the advanced option I described above. As a follow up, it should be noted that after my site generated all of the cache files… my server CPU usage finally settled around 30%. That’s with the cache delay time set to 4h. I’m going to see if I can get better results by setting it higher.

Ah OK. I have the standard version which is only up to v.1.10 at the moment.

Would applying a caching programme do the same job? I tried SuperCache and W3 Cache but both seemed to conflict (I forget what happened now, just remember it boiled down to some problem with ai1ec). Was considering putting QuickCache on.