LiteSpeed Cache Crawler – “No valid sitemap parsed for crawler” error

@daniellacatus I have tested on LTS enterprise, with XML Sitemap Generator for Google 4.1.21(default settings) and LSC 6.5.2 and I was not able to reproduce the issue(crawler worked and the sitemap was validated in LSC).
But, after I looked over the report and the data sent, I tested your sitemap link and is not being validated.

Tested the same sitemap with a development version(new code that will be released in the future) and it’s working.
The error will be going to be fixed in the next release.

We are sorry for the incovinience.

• This reply was modified 1 week, 4 days ago by litetim.

Thank you for your response and for testing with the XML Sitemap Generator and LSC.

I am interested in using LiteSpeed Cache Crawler to help preload the site. Currently, every time I publish or update an article, or when any plugin or theme update occurs, the first page load is significantly delayed—sometimes taking between 40 and up to 60 seconds. Given that our site is very active and frequently updated, such delays could impact traffic, as readers may encounter slow loading times.

• This reply was modified 1 week, 4 days ago by daniellacatus.

@daniellacatus can you make a test: disable Drop Domain from Sitemap ( https://docs.litespeedtech.com/lscache/lscwp/crawler/#drop-domain-from-sitemap )

Additionally, right after publishing an article, tools like Seobility, PageSpeed Insights, and GTmetrix often show errors like “This page is not accessible.” It takes several minutes for the page to become accessible. I am not sure what could be causing this issue.

Moreover, I’m using a premium plugin for social media distribution, which previously worked smoothly but now posts with substantial delays. For instance, articles published yesterday were shared on social media an hour later.

Any insights into these issues or configuration suggestions would be greatly appreciated.

• This reply was modified 1 week, 4 days ago by daniellacatus.

I have tested by disabling Drop Domain from Sitemap, as suggested, but unfortunately, it didn’t resolve the issue.

The delays are still present, both in page load times after updates and in external tools such as Seobility, PageSpeed Insights, and GTmetrix, which still show errors like “This page is not accessible” immediately after publishing an article. The social media distribution plugin is also experiencing delays.

Image – print

Not being able to run the crawler will make the page be inaccessible(if the server is overwhelmed) for the first load.

I am able to parse your sitemap on my server(with Drop Domain from Sitemap OFF, because of different domain).

I am still investigating the issue.

@daniellacatus please try to run this code on your root folder of server:
<?php
require( './wp-load.php' );
$response = wp_remote_get( 'https://*****/sitemap.xml' );
echo '<pre>'; var_dump($response); echo '</pre>';

Tell me the result, this tets if the sitemap is accessible from your own server.

The result

object(WP_Error)#3425 (3) {
  ["errors"]=>
  array(1) {
    ["http_request_failed"]=>
    array(1) {
      [0]=>
      string(59) "cURL error 28: Connection timed out after 5001 milliseconds"
    }
  }
  ["error_data"]=>
  array(0) {
  }
  ["additional_data":protected]=>
  array(0) {
  }
}

string(59) "cURL error 28: Connection timed out after 5001 milliseconds"

this looks like your server or something has blocked the request to sitemap

the code you used, was using wp_remote_get, which is a wordpress’s own function , nothing related to plugin , if wordpress itself failed get the sitemap , I’d suggest to check with your provider if something on your server.

——-

wait , 5 seconds might be too small, please try this instead

$response = wp_remote_get('https://*****/sitemap.xml', array(
    'timeout' => 60, // Timeout in seconds
));

for that line

• This reply was modified 1 week, 4 days ago by qtwrk.
• This reply was modified 1 week, 4 days ago by qtwrk.

I’ve continued troubleshooting and reached out to my hosting provider for additional assistance. They confirmed:

“We do not block the curl function in any way, and there are no server-level settings causing this timeout.”

Additionally, I have:

1. Temporarily deactivated all plugins, including the sitemap plugin, to rule out conflicts, but the issue remains.
2. Reviewed my .htaccess file and removed any potentially interfering rules, yet the error still persists.
3. Verified with the security plugin support team, who also confirmed that there are no settings in the plugin that would block the LiteSpeed crawler.

Given this, could you advise if any LiteSpeed Cache settings or further .htaccess adjustments might help resolve this issue?

• This reply was modified 1 week, 4 days ago by daniellacatus.

have you tried with 60 seconds timeout ? that should be a fairly reasonable number

I tried, the result is the same: ?No valid sitemap parsed for crawler.”

no , I mean that test script , where I asked you modify one line to higher timeout , namely

<?php
require( './wp-load.php' );
$response = wp_remote_get('https://*****/sitemap.xml', array(
    'timeout' => 60, // Timeout in seconds
));
echo '<pre>'; 
var_dump($response); 
echo '</pre>';

The resultat is:

object(WP_Error)#3416 (3) {
  ["errors"]=>
  array(1) {
    ["http_request_failed"]=>
    array(1) {
      [0]=>
      string(60) "cURL error 28: Connection timed out after 10001 milliseconds"
    }
  }
  ["error_data"]=>
  array(0) {
  }
  ["additional_data":protected]=>
  array(0) {
  }
}

wait, in the code, you put 60 , and it actually times out in 10 seconds ?