Online Version unusable due to timeouts

Hi @zanduino

I hope you are doing well today.

I checked logs on our side and it seems the scan was stopped after 43min does too many server errors. The overall scan means that we encountered, too many server errors?and decided we’re not gonna progress any further. That’s generally an indication that the site is struggling to keep up with the traffic.

As a precaution, if we detect a site is struggling – as in, if it starts throwing us errors, we will bail and stop further crawling as to not additionally tax the site (as it clearly has bigger, more immediate issues than broken links).

This also includes things like e.g. request timeouts, which we will treat like server errors – if we detect too many of those, we will also bail out what happened in your case as well.

In either of those scenarios, there’s simply no good data to be had from scanning the site – something’s obviously wrong there, and BLC online engine is not able to finish that task to not harm your site.

How can I rerun a scan on just timeouts instead of having to do a complete new run?

I’m afraid this is not possible ATM.

Kind Regards,
Kris

According to BlueHost and the site statistics I see on the site, it is barely being exercised. This functionality is the only one that has timeouts, so I really don’t think that it is general web site issue. No other user or app has timeouts (plus I’m using litespeed caching).

What are your timeout settings for internal links? And can they be configured?

Hi @zanduino,

I checked the timeouts and if you check the existing scan results, you’ll notice except for 403, and 503 errors most of the URLs are reporting 25 as a status error in most of the site URLs.

The 25 as status would occur due to time out, and one of the common instances for a timeout would be if the page takes time to load.

If a page takes more than 10 seconds to load then BLC won’t be able to pick such page URLs.

If I try to manually load these reported site URLs, it does take around 10-15 seconds to load and could also notice the X-Qc-Cache has a status of “miss” for the header response.

Screenshot at 12:19:10.png

If I re-load the same page again then the cache kicks in and it changes the X-Qc-Cache status to “HIT” and the load time of the page also increases due to the cache.

I’m afraid, the issue noticed still seems more related to high load.

Could you please check and see whether you are able to notice the above behaviour, if yes, you’ll need to consider optimizing the website further.

Please do let us know how that goes.

Kind Regards,

Nithin

Hello Nitin,

when I check my pages from a browser in private mode to ensure I get uncached pages, I do not get lengthy response times. Usually 2 seconds but nothing over 10 seconds.

I just ran the litespeed crawler across the whole site for all possible client permutations (webp,jpg,mobile,tablet,PC and with guest/no guest mode) so that <span style=”text-decoration: underline;”>every single possible page is cached</span>. Using pages

Then I reran a scan using your online tool and it once again fails with “internal request timeout” on most pages.

When I use “pagespeed insights” to check on some of those pages, all of them are at, or under, 2.5 seconds.

Hi @zanduino,

Thanks for the follow-up. I checked your website via the Hub and I could notice there are only 34 URLs listed at the moment and the URLs which are listed aren’t from your website anymore.

Seems like the action of running the lite speed crawler across the site did work.

At the moment, the broken URLs listed are pointing to different website URLs listed on the relevant source page. I suppose these are backlinks.

Most of the reports in the BLC scan are now pointing to the status of the URLs having 403, 25 or 503 errors.

As mentioned in the previous response, the 25 errors as status would be due to timeouts when BLC takes time to crawl the site it throws such an error.

The 403 error as noticed would point to a firewall in their server or website blocking the BLC request and hence a 403 error.

The 503 error is server specific and when BLC tries to crawl the website is throws a 503 response and hence the issue.

Since they are pointing to a different site hosted externally I’m afraid there isn’t much that could be suggested other than marking such URLs as “Ignore Link”.

Please do check and let us know if you need any further assistance.

Kind Regards,

Nithin

I ran the crawler and that didn’t solve the problem. But I ran the WPMU online program twice, and the second time the internal timeouts were, for the most part, gone. The remaining error links are the same as those I get using the original local version.