BuddyPress Docs adds 3.5 seconds to siteload time

No, it’s not expected behavior.

Do you have many, many Docs? Are you using complex categories?

Does P3 do any more advanced profiling to see exactly what is causing the slowdown? Perhaps you have access to a MySQL slow query log?

I have a very simple site with only one or two test docs and 5 categories.

I have been tuning the site with W3TC cache and Amazon CloudFront and performance is significantly better. It may be that P3 is not reporting the real world situation.

I have installed Newrelic and got a stack trace. This shows that BP_Docs_Attachments::check_is_protected takes 3s to return because it makes a test request.

* please confirm that check_is_protected only runs in the admin panel and not on every call

If that is the case then it might be that P3 is triggering that call somehow and the reported delay is a test artifact.

* what are your thoughts?

Thanks very much for the additional information. I’ll look more closely into this as soon as I can, but in the meantime can you give me a bit more info:

– Are you logged in as an administrator when this is happening? Have you verified that the problem exists for logged-out users or for non-admins?
– Do you have a complete stack trace to BP_Docs_Attachments::check_is_protected()? By default, it’s called from a few different places throughout the plugin; knowing which would help me to debug.
– What’s your server setup? Are you running Apache? Are your Docs, in fact, protected? Do you see the warning notice in your admin panel about Docs being unprotected? Again, this would be another helpful debug point.

Thank you Gorges,

I have spent the evening learning how to use the monitoring. I will do some real testing tomorrow. I will then try to answer your questions.

My set up
* Bitnami AMI on Amazon small instance running WordPress Multisite
* Ubuntu

Dear Gorges,

I have morning looking at traces. The tool gives a sample of long running transactions.

One that keeps coming up is

/wp-admin/admin-ajax.php

and in each case there is the call to BP_Docs_Attachments::check_is_protected.

Here is a typical trace. There is no reason that this should be being called at all. I am just browsing the website and creating posts.

If you would like a login to my systems please send me your email address (james (at symbol) elephantpm.com and I will give you a login.

Thank you for your help. James

https://dl.dropboxusercontent.com/u/34047309/buddypress-docs-trace.jpg

xjamesb – Thanks for this additional info. Very helpful.

From what I can tell, it seems like a couple things are happening here. One is that the AJAX is incorrectly triggering this check. The second is that the check is not properly storing the result of the check in the database, with the result that the external request has to happen each time.

I’ll have to do some more digging to figure out what’s up with the second issue, but the first one is fixed in https://github.com/boonebgorges/buddypress-docs/commit/beeadb50411eacd7738cce87b16482cdaf9e2c0b. I’ll have a release out soon with this fix.

Thank Gorges,

Now you have confirmed the bug I have disabled BuddyPress Docs and await your new release. System performance has returned to normal.

The whole AJAX thing is very difficult to debug and diagnose. I am new to WordPress development but I understand that the normal process is to use tools such as Debug Bar and Query Monitor to look at single pages as they execute. This probably won’t show up AJAX issues.

P3 is better because it does a larger sample. Newrelic is great because it does a very large sample and full stack traces but it is $200 per month per server which is beyond the means of most developers.

* Are you aware of any other tools that are cheaper than Newrelic that we could have used to detect and then diagnose this issue with a stack trace?

Hi,

we have a similar issue which is effecting our site. Our site is large and we use bp_docs extensively within a number of groups.

We have discovered that when we add a featured image to a post and submit the post we get an ‘error establishing database connection’ for around 20-40 mins.

Our IT department has indicated that this is due to “multiple apache worker processes are created when trying to set the featured image”.

We worked extensively to identify the issue, initially we believed the theme to be at fault but we have ruled that out by a process of elimination.

By disabling and re-enabling plugins and re-running the process of setting a featured image we have determined that ‘buddypress docs’ is somehow putting enough of a strain on the system to take it down.

Here are some log entries (edited to remove certain references) that have occurred around the time of the system going down, BP_docs and bp_docs are mentioned a lot:

[error] [client ***.***.***.***] PHP Warning: in_array() expects parameter 2 to be array, null given in /var/www/wp-content/plugins/buddypress-docs/includes/integration-groups.php on line 264

Above repeated multiple times (~20)

[error] [client ***.***.***.***] WordPress database error Lost connection to MySQL server during query for query SELECT option_value FROM wp_options WHERE option_name = 'bp-docs-enable-attachments' LIMIT 1 made by require_once('wp-admin/admin.php'), require_once('wp-load.php'), require_once('wp-config.php'), require_once('wp-settings.php'), do_action('plugins_loaded'), call_user_func_array, BP_Docs->load_hook, do_action('bp_docs_load'), call_user_func_array, BP_Docs->do_integration, BP_Docs_Component->__construct, BP_Docs_Component->setup_hooks, BP_Docs_Attachments->__construct, bp_docs_enable_attachments, get_option, referer: https://***/wp-admin/post.php?post=583&action=edit

[error] [client ***.***.***.***] WordPress database error Lost connection to MySQL server during query for query SELECT post_id, meta_key, meta_value FROM wp_postmeta WHERE post_id IN (1569) ORDER BY meta_id ASC made by require('wp-blog-header.php'), require_once('wp-includes/template-loader.php'), include('/themes/buddy-2014-01/page.php'), the_content, apply_filters('the_content'), call_user_func_array, bp_replace_the_content, apply_filters('bp_replace_the_content'), call_user_func_array, BP_Docs_Theme_Compat->single_content, bp_buffer_template_part, bp_get_template_part, bp_locate_template, load_template, require('/plugins/buddypress-docs/includes/templates/docs/single/index.php'), include('/plugins/buddypress-docs/includes/templates/docs/single/attachments.php'), bp_docs_attachment_item_markup, bp_docs_get_attachment_url, get_attached_file, get_post_meta, get_metadata, update_meta_cache, referer: https://***/wp-admin/post.php?post=583&action=edit

Above repeated multiple times

[error] [client ***.***.***.***] WordPress database error Lost connection to MySQL server during query for query SELECT wp_posts.* FROM wp_posts WHERE 1=1 AND ( wp_posts.ID NOT IN (\n\t\t\t\t\tSELECT object_id\n\t\t\t\t\tFROM wp_term_relationships\n\t\t\t\t\tWHERE term_taxonomy_id IN (39)\n\t\t\t\t) ) AND wp_posts.post_type = 'bp_doc' AND ((wp_posts.post_status = 'publish')) GROUP BY wp_posts.ID ORDER BY wp_posts.post_date DESC made by require('wp-blog-header.php'), require_once('wp-includes/template-loader.php'), include('/themes/buddy-2014-01/homepage.php'), get_footer, locate_template, load_template, require_once('/themes/buddy-2014-01/footer.php'), dynamic_sidebar, call_user_func_array, WP_Widget->display_callback, WP_Widget_Recent_Posts->widget, WP_Query->__construct, WP_Query->query, WP_Query->get_posts, do_action_ref_array, call_user_func_array, bp_docs_general_access_protection, BP_Docs_Access_Query->get_doc_ids, get_posts, WP_Query->query, WP_Query->get_posts

We have a large user base (20,000+ members) but far fewer concurrent users (typically 20-50). The system running wordpress is designed for robust usage and should theoretically be taken out by one active user posting a featured image to a blog post.

I hope this helps in some way. Please let me know if we can provide any further material to help rectify the problem. It should be noted that when we activate/deactivate the plugin there is general very notable change in site responsiveness.

Good luck finding a solution, full list of plugins used is available below.

LB

Littlebloke,

On my installation Newrelic also reports that

* SELECT option_value FROM wp_options

is a high frequency and time consuming query. I am not able to determine if this is due to BuddyPress Docs. However, with my limited knowledge of WordPress I thought it was odd.

littlebloke – Thanks for the detailed report.

The first notice you mention (in_array() expects...) is just a PHP notice that should do no harm. It will be fixed in the next version of Docs; see https://github.com/boonebgorges/buddypress-docs/commit/a5945ba799406dca92d7dd62faa0ddbe44becd0e

It’s unlikely that the second one would be the source of a problem. It’s just the stack trace for a call to get_option(), which is used in thousands of places through WP.

It is possible that the last two would cause a fairly large strain on your system. I’ll have to do some more analysis to see whether and why that would happen in the case of adding featured images.

While I’m working on this, can you please share a few pieces of information:

– WP/BP/BP Docs versions
– Rough count of groups and Docs on your site

==

xjamesb – That query should be run one time on each page load (assuming you are using no static caching). Docs itself doesn’t have anything directly to do with it, and in fact the query probably takes place well before Docs is even loaded (specifically, it happens the first time get_option() is called during a given pageload). If you’re having extreme performance issues related to this query, it could mean that a plugin is leaving large amounts of data in your wp_options table.

Hi,

Thanks for the info,

This was a few of the log entries that I was able to get from our IT team, there may be others… is there anything specific that I could look for that may be of help?

Here are the plugin/version details I forgot to include in my last email, a full list of plugins we use:

– WordPress 3.9

– Advanced Access Manager 2.6
– Akismet 3.0.0
– BackUpWordPress 2.6.2
– bbPress 2.5.3
– bbPress Enable TinyMCE Visual Tab 1.0.1
– Buddy Plugin 1.2 (associated with theme)
– BuddyPress 2.0.1
– BuddyPress Community Stats 0.5.1
– BuddyPress Docs 1.7.0
– BuddyPress Group Email Subscription 3.4
– BuddyPress Profile Visibility Manager 1.0.5
– Easy WP SMTP 1.07
– Email Login 4.6.3
– GB bbPress Attachments 2.1
– Google Analyticatoor 6.4.7.3
– Invite Anyone 1.2.1
– Jetpack by WordPress.com 2.9.3
– Prevent Password Reset 0.2.0
– Twitter Widget Pro 2.6.0
– WP Email Template PRO 1.1.3.1

Theme is:

– Buddy (by Ghostpool)

In terms of numbers we have 52 Groups, and a total of 103 buddy press docs, which can themselves contain between 2 and up to 30 items (we have a total of around 1500 media files).

Not all these groups are ‘active’. but a number of groups contain say, 10 docs each containing around 20 items.

Hope this helps.

LB.

littlebloke – I’ve done some additional investigation.

Based on the stack traces provided in the error messages above, it looks to me like AJAX requests (of the sort that happen while setting a featured image) are causing the front-end of the site to be load in the background. This is incorrect behavior: when making an AJAX request, WP should recognize it early on, and load only the portion of the application necessary to handle the request. Yet I see that your theme templates are being included. I’d guess that the strain of loading your site’s front-end over and over again during AJAX requests is bringing the server to its knees. I’d also guess that this is not strictly due to BuddyPress Docs; it just so happens that the Docs query is the straw that breaks the camel’s back.

I can’t reproduce this on my own installations, using setups similar to yours, so I’m unsure what to think. There are many things that could cause this sort of error. For example, if your AJAX request is going to an incorrect URL, it would trigger a 404 page that might exhibit the behavior you’ve described above. The first thing to check is your browser console during an AJAX request, to verify the correct address (https://example.com/wp-admin/admin-ajax.php). You may be getting a 404 (which would break AJAX functionality). From there, you may want to check your server access logs to see if you can verify that AJAX-style self-requests are proliferating. Finally, you’ll simply have to do some step-by-step debugging of the WP pageload process to get a sense of exactly what is causing these incorrect front-end pages to load in the background.

Good luck, and sorry I can’t be of more direct help. Please update this thread if you get closer to finding the culprit.

Hi,

Thanks that’s helpful… i’ll go away and do some investigation updating the thread further with any results.

Boone,

I would like use BuddyPress Docs again. Do you have an ETA for your next release?

Is is prudent for me to download the 1.7.x branch to which you link above and use that? Will the auto-updater work when you subsequently release a new version?

Many thanks,
James

> Do you have an ETA for your next release?

Not really. Sometime in the next week or two, probably.

> Is is prudent for me to download the 1.7.x branch to which you link above and use that? Will the auto-updater work when you subsequently release a new version?

Yes, and yes. This is what I would suggest.