switch_to_blog() causing memory leak

I’m seeing my requests running out of memory (even with a ridiculous 512M allocated) on a system that does a lot of blog switching.

Seems to have started with WP 3.5, I’m now seeing it with 2.0.1 of this plugin too. Whereas the plugin was definitely ok earlier.

Right now I have to remove the plugin, which is killing performance.

I have not managed to fix the references issue, however when I implemented wp_cache_switch_to_blog things somewhat improve. Doing this means the object cache size (in php memory) will increase whenever switching to a _new_ blog, but restoring to older blogs has no overhead. So this happens:

1. wp_cache_init on blog 1 – memory used: 2mb
2. switch_to_blog( 2 ) – has an object cache of 1.5m – memory used: 3.5mb
3. restore current blog – memory used 3.5 (as blog 1 was already in the object cache)

If switching between a small amount of blogs this is acceptable (and better than this bug), however memory will still increase with the more blogs you switch to this way.

My implementation of wp_cache_switch_to_blog:

function wp_cache_switch_to_blog() {

	global $blog_id, $table_prefix, $wp_object_cache;
	$wp_object_cache->blog_prefix = ( is_multisite() ? $blog_id : $table_prefix ) . ':';
}

Mike: which version of PHP are you running? I am using 5.4, haven’t got round to debugging this under 5.3 yet.

I’m running 5.3.2.

Unfortunately, the site in question does a lot of switch_to_blog, so I don’t know if your solution would work.

And currently the site is having an ‘event’, which means 25 – 50K visits per day over a 8 hour period, much of that live interaction, so I can’t do anything that might destabilize the site!

I’ll see whether I can reproduce the issue on my test rig, and then whether your solution would fix things.

Is this a bug in switch_to_blog()? I know it was changed recently… Or has the change triggered a bug in the plugin?

Ok, I have a fix!

Not sure what triggered this issue, I know switch_to_blog() did change, but looking at what changed I don’t see why this would have broken.

Anyhow, working on my above function, the key is to blow away the in-memory cache everytime switch to blog is called, this lets the memory footprint remain low no matter how many times you switch to different blogs:

if ( ! function_exists( 'wp_cache_switch_to_blog' ) ) :
function wp_cache_switch_to_blog() {

	global $blog_id, $table_prefix, $wp_object_cache;
	$wp_object_cache->cache = array();
	$wp_object_cache->blog_prefix = ( is_multisite() ? $blog_id : $table_prefix ) . ':';
}
endif;

To test this, I have a helper function to eat up memory:

function use_memory() {
	for( $i = 0; $i < 100; $i ++ ) {
		wp_cache_add( $i, str_pad( 'a', 1014 * 1014, 'a') );
	}
}

So to test (in init hook):

use_memory();
echo memory_get_usage() / 1024 / 1024; // should be about 120MB
switch_to_blog( 2 );
echo memory_get_usage() / 1024 / 1024; // should be about 25MB
restore_current_blog();
echo memory_get_usage() / 1024 / 1024; // should be about 120MB

You need to flush memcached every time running the test, so wp_cache_add() works

I am running the above function in production which fixed memory issues I was having, can now switch_blog_blog() 20+ times without issue

This was a deliberate change in core.

wp_cache_reset() used to be used during the switch_to_blog(), which would obliterate the existing object cache. So if you switched from site A, to site B, back to site A, all existing local cache for site A would be gone, and you’d have to re-query everything (notably, options). It was really silly and actually made switch_to_blog() more expensive, especially if you were doing something like get_blog_option() repeatedly, which now does a switch() inside it.

The caching layer was changed in 3.5 to also key storage by site. So if you switched from A to B to C all the way to Z, we’d store anything in local cache for when you switched back.

This isn’t a “memory leak” as much as an overall performance benefit (most of the time).

Of course, there are situations where a way to “free” memory would be pretty useful. We deprecated wp_cache_reset(), but could bring back its functionality for when the situation requires it.

Background: https://core.trac.www.ads-software.com/ticket/21434

I wouldn’t mind bringing back wp_cache_reset() with lots of phpdoc explaining how and when to use it in a plugin. I think the current behavior should remain the default, however, since it saves lots of queries.

nacin: Yeah, I can see how blowing away the object cache for the not-current-site would be a bad idea, if you don’t have persistant object caching. But doesn’t this change mean it’s pretty much not possible to switch_to_blog() more than say, 20 times:

Presuming I have 100 blogs on an ms install, all with ~3mb object caches, switching between them in a loop is going to load 300MB of data into that process’s memory – if I am using Memcached, I would be better off throwing it away from the current process on switch_to_blog and pull it back from Memcached when a blog is next switched to.

That’s presuming the overhead of pulling everything from Memcached is essentially “free”.