WP needs to move from dynamic to static HTML pages

Most sites built within the past 10 years including Digg, Facebook, Google, and our very own WordPress.com do not write out static HTML for every page. My own site and all of www.ads-software.com are run “strictly off a database” with no caching, while some other developers like Donncha prefer WP Super Cache, in fact he wrote it (I wouldn’t recommend WP-Cache, it hasn’t been updated recently).

Here is the traffic for WordPress.com, which is 3.8 million WordPress blogs doing tens of millions of fully dynamic db-driven pageviews per day:

https://www.quantcast.com/p-18-mFEk4J448M/traffic

There are lots of solutions and examples of sites that have scaled to millions of pageviews per day, from Techcrunch to icanhascheezburger. The best bet is to find a hosting partner that is supportive of your growth and then choose the method of running WP (dynamic, cached, static) that is most appropriate for your hosting setup. Then just forget about it, so you can focus on what really matters to your blog, the content.

There is a pluggable caching engine built into core that can eliminate most DB queries, but there is no plan for static file publishing in core. If you’ve ever used a system that makes you wait 15 minutes while your blog is “rebuilding” you’d know why.

So while the specific solution you suggest (pre-generating static HTML) has been considered by scaling experts and the WP community and not chosen, if you describe your specific problem perhaps we could help you work through whatever roadblocks or humps you’re hitting.

Matt, I’m glad to hear directly from you, and I appreciate the detailed reply.

The reason behind my recent frustration with WP’s dynamic vs. static file approach is that I can’t seem to find a host that will let me focus on just “the content”, as you put it. It seems that web hosts love to blame WordPress as the reason the web server is overloaded, or why the site has gone down. Naturally, that sort of thing would make one wish for the ability to generate static HTML files, which scale much better as traffic grows, and make it harder for the web host to find an excuse.

I used 1and1 in the past, then experienced significant outages and moved to Media Temple. Now (mt) blames my WP install as the reason the mySQL server is overloaded and goes down, and they’re suggesting that the WP tables aren’t properly indexed, and that the queries aren’t optimized, or that my site’s too slow, not properly configured, etc. I’ve heard all sorts of “suggestions” from them in a recent case, which prompted me to do some research and find out that others are frustrated with WP’s perceived server load as well.

At any rate, I would really love to not have to worry about the best caching method for me, or why my site keeps going down, or why my web hosts always go into CYA mode, and just publish content, which is what I love to do. I’m not getting huge amounts of traffic like the big sites, and keep wondering why I can’t seem to find a web host that will not offer excuses but just keep my site up.

If you’d like to help me tune my WP install a little, I’d love to work with you or one of your team members on that and learn how to keep WP humming along. I believe I’ve already done a fair bit of tuning to my site, but I’m always glad to learn more. I’d also be glad to document the findings for the benefit of others. Btw, I also sent a message to the host feedback email (seen here https://www.ads-software.com/hosting/) detailing my most recent problem.

Many thanks for your reply!

“… if you describe your specific problem perhaps we could help you work through whatever roadblocks or humps you’re hitting.”

Matt, it’s been a week since. I hate to bug people, but what should I make of your invite to help me? Should I post a more specific question here, or should I start a new thread in one of the other forums?

If you could chime in and let me know either way, I’d be grateful. Thanks!

I guess you may need to be more specific ??

Anyhow. If you’re experiencing issues with shared hosting, maybe you need to consider moving up the chain? Perhaps your site(s) have outgrown normal hosting offerings, and you need something larger? The funny thing with weblogs is that if you have a successful one it’s easy to become a victim of your own success.

If you’re in a country that they offer to, you could look at a dedicated server from Ovh. They are reasonable value I think.

Anytime your site becomes even a little bit successful, it easily overloads your host’s shared policy. It’s then time to get a dedicated server. Options include co-location, managed dedicated server, self-managed dedicated server… Unless you’re good at managing your own server, you may want to go the managed dedicated server route…

V

mrmist and valuxes, while my site does okay, it’s not a big site (yet?) and should do just fine on shared hosting, if only the web hosts would live up to the promises they make on their websites. So far, I’ve gone through two different web hosts (1and1 and Media Temple), each of which promised plenty but were short on results and piled on the CYA (Media Temple in particular).

As of yesterday, I am on a VPS at SliceHost, and so far, WP is doing just fine there running uncached. I will continue to monitor the server to see how WP does under normal loads over the next few weeks. At long last, I can tell exactly what’s going on with my site, which was not a possibility when running under shared hosting. If I encounter any specific problems or performance bottlenecks, I may post questions here.

I remember arguing with my shared host about this issue and their reply was that their promise of unlimited bandwidth applies to “me” and what “I” upload and download. It does not apply to the bandwidth the visitors to my website consume… anyway, wish you good luck on your search.

Just checking in to confirm that acquiring a VPS was the right thing for me to do. I ran WP cached for a while, but found that memory quickly got eaten up when I got indexed by search engines, even after trying various tuning approaches. Now I run it uncached, and it does just great.