high traffic crashing usmagazine.com

MySql 4.1.18

Still checking through the forums. I wonder why I typed that. But I did.

I understand now that it’s not really the kind of thing that can be helped on a support forum. But I’m going to document this problem in case it’s helpful.

Our ISP is mediatemple.net, and I’ve been very happy with their support up till now. I think Zeldman gives them a shout-out in his blog. However, them quickly saying “It’s WordPress” doesn’t cut it for me. Even if they end up being “right”, it smacks too much of “let’s point at the one component we didn’t install.” I should point out they still respond very quickly and are polite, etc.

We’ve had indications of traffic problems before this and have asked about bumping up from a dedicated virtual server to a dedicated physical server. They seem oddly not interested in talking about that.

I’m not rebuilding the site just because my ISP won’t help examine where the bottleneck really is. Even if I abandon WordPress, wouldn’t ANY php-mysql app hit similar limitations with huge traffic?

Last night, I was getting error code I posted above. I was going to rebuild the database, courtesy of the daily backup I get from Scott Merrill’s plugin. But if there were disk errors, I was worried about somehow aggravating the problem by deleting and writing a bunch of data.

Stuff I’ve done/looked into:

* My ISP has asked if I can have a flat HTML front page. I’ve installed WP-cache (which my tech support guy found for me). Not truly a flat page, but I assume this is helping.

* After looking through these forums, I installed Jerome’s Query Diagnostics to see if any queries were holding us up.
This was for the front page.
This was for a single post.
Nothing stood out.

* It seems comments can cause problems, but comments are a big part of the site and I can’t disable them, or not for long, anyway.

* I cleared out the default entry (pingomatic?) in the “update services” box.

* I disabled trackbacks and pings (we never had many anyway).

* We don’t have a sidebar, so it’s not like there’s a list of categories or past entries whose queries could be slowing us down.

* We have plenty of disk space left — 10 gig of 17 available.

——————

This morning, I get this message from mediatemple:

“Due to a previously undocumented problem with the virtualization software, some users have been experiencing situations where they are exceeding their kmemsize limits.”

Not sure what a “kmemsize limit” is, but I assume it is usmagazine’s traffic that has created this problem. I’m hoping they can make hardware adjustments to allow me to keep using WordPress.

Let me say again that it’s amazing how well-documented the support for WordPress is, and amazing how easily I’m able to add plugins, and locate information on problems. I love it.

In the very least, get yourself a dedicated server. Get the fastest one you can, with tons of ram. A site like that needs it. And make sure it’s in a fast data centre with big pipes to the internet backbone. And don’t put email on the same server. You might be alright with that.

If you can spend the money, considering something like this > https://ev1servers.net/Dedicated/RTG/servers/Monster.aspx. Please note I’m not recommending ev1 necessarily, please shop around, I just provided a link for information on what a high end server might feature. If the budget can’t afford that, get the best with the money available.

[Can’t create/write to file ‘/tmp/#sql_16d8_0.MYD’ (Errcode: 17)]

This suggests that it’s possible the temporary directory on the server is filling up which you should contact your host about.

Thank you for the advice.

I have asked repeatedly for prices of a dedicated server. They’re telling me that the software (WordPress) is so inefficient that IT is the bottleneck, not the server. That’s not their words — but they shrug off upgrading as a good solution, pointing to “WordPress.” It’s frustrating.

It seems like it’s subjective to blame either the software or the hardware. But I guess I have to either do what my ISP says or switch ISPs.

I’ll search for an ISP that will stand by WordPress, but I imgaine I’ll have to find a new blogging solution that they’re willing to work with.

I have recently faced a slowing down / hanging issue with WordPress wherein my Apache server was consuming massive amount of CPU. I too am on a Virtual Private Server and I use wp-cache 2 (recommended).
On analysing the SQL queries the problem was isolated and fixed. Today my site runs like a charm.
You can check out the article (linked above) for details.
Few other thoughts:
wp-cron is basically buggy (resulting in server overload). I have tested it in the past couple of days and have fixed it for my use. I haven’t got around to releasing my version yet or inform the author. You may want to switch over to using a cron job instead as you are on a VPS.

Cool. I’ll check these out.

What virtual server plan are you on? Have you paid a tech to adjust your PHP and MySQL settings to tune to the memory limits of your VPS?

WP-Cache should be making the site almost no-query on cached pages. It’ll hit apache, light php, and otherwise return quickly.

I’d agree with westi – sounds like there’s something wrong with the server configuration overall. I’m shocked that mediatemple isn’t handling this better (they’re rated highly by many users…), and the response that WP is ‘so inefficient’ and that moving to a dedi wouldn’t help is just irresponsible.

If you’re going to host, find a host that actually supports/manages their VPS plans heavily. You need a managed service, or a tech who deals with this stuff daily/weekly and can lend a hand (maybe at a reduced cost..). There’s a least one or two folks on the forums here that have dealt with VPS optimization — but you might be beyond what a VPS can handle.

Other thoughts:

serving up lots of big, high res graphics and static content? run something like lighttpd/et al to serve up the static stuff. In some cases (someone who knows what they are doing) you could completely replace apache. There are also commericial/paid servers that have even better performance — if apache performance is impacting things at all.

wp-cache2. make sure it is on, and working properly.

other plugins. be careful what you use, try to ensure they aren’t adding heavily to the pageload time (either in code or in queries).

your second query dump showed:

[0] => SELECT option_value FROM wp_options WHERE option_name = 'siteurl'
[1] => 0.23864006996155
[0] => SELECT * FROM wp_users WHERE user_login = 'Will'
[1] => 0.53467917442322

Obviously, that’s showing an overloaded server.

-d

Oh and you may try using Bad Behavior and / or Akismet instead of Spam Karma.

> * I cleared out the default entry (pingomatic?) in the “update services” box.

This will not help much as it affects only during posting time.

> * I disabled trackbacks and pings (we never had many anyway).

This may moderately help but I would think the real problem is elsewhere.

I looked at your queries. By itself they don’t look much of an issue. However the cumulative effect may be killing you.
Use wp-cache 2 (linked above) and set cache time to at least 2 hours or more.
Also try to get query details from SQL server itself instead of any plugins.

> I’ll search for an ISP that will stand by WordPress, but I imgaine I’ll have to find a new blogging solution that they’re willing to work with.

My ISP supports WordPress, even installs it for you. They worked with me on my server load issue and actually helped me to identify the errant query.

Wow! This is awesome. Thank you so much.

Mediatemple has been nothing but great in many instances before. This is an anamoly. It’s like I pissed them off.

I’m working getting someone to help me isolate queries on the MySQL server.

I have a feeling it’s the comments. Our traffic is being driven up by an “exclusive” post on Britney Spears being pregnant. Yes, this is a hilarious problem. Viewers are commenting like crazy, and that forces a regeneration of the single post page. I’ve disabled comments on that post and are keep analyzing.

No Spam Karma 2? I was acutally worried that using Akismet would slow the site down if Akismet got overwhelmed. But now it’s MY local database which is overwhelmed, so maybe you’re right.

This might not be entirely helpful for your situation, but over the past few weeks I’ve been going over some optimization of WordPress for my own needs, and I thought I would share.

One of the major factors in getting reliable service was keeping the server from dipping too deeply into swap. Having enough memory available to keep swap to a minimum is one of my new priorities with the new virtual server. When the swap started to get used significantly (usually from a torrent of MySQL queries), it began a cascade effect that would bring the server to its knees. Adding more memory to the system solved the issue instantly.

I installed a PHP opcode cache (APC) on the server. This had a significant impact on load, because the server no longer wastes time recompiling PHP scripts every time they are requested. My pages are running nearly 100% from the cache of compiled PHP, meaning that every page request is using a precompiled script and skipping over the load generated by compilation. Before APC my load was near 3 on average during high traffic times, now it’s consistently under 1.

Another thing that may help performance would be a combination of using the MySQL qcache and a plugin that helps optimize queries for it. You can read about and get the plugin from the wp-hackers mailing list archive. Note that the data from this plugin helped drive development, and that WordPress 2.1 (when it is released) will have this sort of optimization already built in.

Of course WP-Cache is a decent plugin for creating static caches, but it still might not be enough by itself to offset the issue completely. Also, it doesn’t work out-of-the-box with APC. You’ll want to review these notes on APC and wp-cache2. Currently, my site does not run both, since I couldn’t get them to play well together, but I’ve been happier with APC’s performance than with WP-Cache’s.

The sad, unspoken truth is that you can optimize your code until your ears bleed, but sometimes you just need a bigger box.

Disabled Spam Karma 2. Enabled Akismet.
Disabled wp-cron.
Disabled comment on the post which is attracting all the traffic.

That might have put out the fire.

I’m going to need to be able to keep comments open while sustaining high traffic, but if it puts out the fire, I’m happy to disable them temporarily.

Awesome help, everyone!

Just noticed LightPress – a frontend for WP that claims to greatly increase the speed.

See: https://forum.textdrive.com/viewtopic.php?id=5716

Talks about a site that reaches 400 requests/min.

For that level of traffic you do not need a 1000/mo server as suggested earlier in the thread.

Lightpress is different software, it doesn’t use WP themes or plugins or…

Properly configured wp-cache can give you 200+ requests per second, or about 17 million pageviews per day, which is more than enough for Mr Hines’ site.

Owen’s advice is sound.

With WP-Cache, WP can certainly handle hundreds of requests per minute on a decent webserver. LightPress might be a bit more optimized in some ways, but behind in features in others.

Apache performance at some point becomes limiting factor, in which case looking into lighttpd, thpptd, etc., might be useful (if only for image serving…).

-d