Crawler not caching pages
-
Hi have a similar problem to these closed topics.
https://www.ads-software.com/support/topic/crawler-exit-with-stopped_reset/
https://www.ads-software.com/support/topic/crawler-issues-blacklists-every-page-and-wont-crawl-stopped-reset/This morning I found all the pages in the blacklist. I removed them, tried to crawl, but crawler behaves strangely. Sometimes it just ends up with all the pages as grey dots. Sometimes it finds the pages (blue dot). But no pages are cached. The only way to cache a page is to visit it as guest. Sometimes when I click Manually run, the page just flickers, like a quick reload but nothing changes.
I have several client sites at this particular hosting company and all run Litespeed cache and working fine. But not this one.
I have enabled debug.log. IP’s and urls are obfuscated.
10/13/21 10:34:02.454 [99.99.99.999:55880 1 Yz1] e?’“ ------GET HTTP/1.1 (HTTPS) /wp-admin/admin.php 10/13/21 10:34:02.454 [99.99.99.999:55880 1 Yz1] Query String: page=litespeed-toolbox&LSCWP_CTRL=PURGE_EMPTYCACHE&LSCWP_NONCE=b1846333a0 10/13/21 10:34:02.454 [99.99.99.999:55880 1 Yz1] HTTP_REFERER: https://myownsite.se/wp-admin/admin.php?page=litespeed-toolbox 10/13/21 10:34:02.454 [99.99.99.999:55880 1 Yz1] Cookie _lscache_vary: 57d879048edace9989ace55b92584efb 10/13/21 10:34:02.454 [99.99.99.999:55880 1 Yz1] LSCACHE_VARY_VALUE: +webp 10/13/21 10:34:02.758 [99.99.99.999:55880 1 Yz1] [Ctrl] X Cache_control -> no Cache ( logged in user ) 10/13/21 10:34:02.759 [99.99.99.999:55880 1 Yz1] [Router] LSCWP_CTRL: PURGE_EMPTYCACHE 10/13/21 10:34:02.759 [99.99.99.999:55880 1 Yz1] [Router] LSCWP_CTRL verified: 'PURGE_EMPTYCACHE' 10/13/21 10:34:02.760 [99.99.99.999:55880 1 Yz1] [Purge] added * 10/13/21 10:34:02.760 [99.99.99.999:55880 1 Yz1] e??? Init 10/13/21 10:34:02.766 [99.99.99.999:55880 1 Yz1] X-LiteSpeed-Purge: public,* 10/13/21 10:34:02.771 [99.99.99.999:55880 1 Yz1] [Purge] added _MIN 10/13/21 10:34:02.772 [99.99.99.999:55880 1 Yz1] X-LiteSpeed-Purge: public,* 10/13/21 10:34:02.797 [99.99.99.999:55880 1 Yz1] [Object] flush! 10/13/21 10:34:02.798 [99.99.99.999:55880 1 Yz1] [Purge] Flushed object cache 10/13/21 10:34:02.798 [99.99.99.999:55880 1 Yz1] [Purge] Reset opcode cache 10/13/21 10:34:02.799 [99.99.99.999:55880 1 Yz1] [Purge] Purge all 10/13/21 10:34:02.817 [99.99.99.999:55880 1 Yz1] [Ctrl] 301 from https://myownsite.se/wp-admin/admin.php 10/13/21 10:34:02.817 [99.99.99.999:55880 1 Yz1] [Ctrl] 301 to https://myownsite.se/wp-admin/admin.php?page=litespeed-toolbox 10/13/21 10:34:02.817 [99.99.99.999:55880 1 Yz1] [Ctrl] X Cache_control -> no Cache ( 301 to same url ) 10/13/21 10:34:02.818 [99.99.99.999:55880 1 Yz1] [Ctrl] X Cache_control -> public forced ( Setting: / ) 10/13/21 10:34:02.818 [99.99.99.999:55880 1 Yz1] [Ctrl] Forced public cacheable due to setting: / 10/13/21 10:34:02.818 [99.99.99.999:55880 1 Yz1] [Ctrl] X Cache_control TTL -> 300 10/13/21 10:34:02.819 [99.99.99.999:55880 1 Yz1] [Router] get_role: administrator 10/13/21 10:34:02.819 [99.99.99.999:55880 1 Yz1] [Vary] set_cookie ---> admin_bar:1;logged-in:1;role:99 10/13/21 10:34:02.820 [99.99.99.999:55880 1 Yz1] e?’° X-LiteSpeed-Cache-Control: public,max-age=300 10/13/21 10:34:02.821 [99.99.99.999:55880 1 Yz1] e?’° X-LiteSpeed-Purge: public,* 10/13/21 10:34:02.821 [99.99.99.999:55880 1 Yz1] e?’° X-LiteSpeed-Tag: e49_default,e49_URL./wp-admin/admin.php/,e49_ 10/13/21 10:34:02.821 [99.99.99.999:55880 1 Yz1] End response -------------------------------------------------------------------------------- 10/13/21 10:34:03.375 [99.99.99.999:55880 1 wHu] e?’“ ------GET HTTP/1.1 (HTTPS) /wp-admin/admin.php 10/13/21 10:34:03.375 [99.99.99.999:55880 1 wHu] Query String: page=litespeed-toolbox 10/13/21 10:34:03.375 [99.99.99.999:55880 1 wHu] HTTP_REFERER: https://myownsite.se/wp-admin/admin.php?page=litespeed-toolbox 10/13/21 10:34:03.375 [99.99.99.999:55880 1 wHu] Cookie _lscache_vary: admin_bar:1;logged-in:1;role:99 10/13/21 10:34:03.375 [99.99.99.999:55880 1 wHu] LSCACHE_VARY_VALUE: +webp 10/13/21 10:34:04.487 [99.99.99.999:55880 1 wHu] [Ctrl] X Cache_control -> no Cache ( logged in user ) 10/13/21 10:34:04.802 [88.888.88.88:50490 1 9yy] e?’“ ------POST HTTP/1.1 (HTTPS) /wp-cron.php 10/13/21 10:34:04.802 [88.888.88.88:50490 1 9yy] Query String: doing_wp_cron=1634114044.7045419216156005859375 10/13/21 10:34:04.802 [88.888.88.88:50490 1 9yy] HTTP_REFERER: https://myownsite.se/wp-cron.php?doing_wp_cron=1634114044.7045419216156005859375 10/13/21 10:34:05.122 [88.888.88.88:50490 1 9yy] [Router] get_role: 10/13/21 10:34:05.127 [88.888.88.88:50490 1 9yy] [Optm] init 10/13/21 10:34:05.200 [88.888.88.88:50490 1 9yy] [Img_Optm] allowance_max 200 wet_limit 10/13/21 10:34:05.200 [88.888.88.88:50490 1 9yy] [Img_Optm] preparing images to push 10/13/21 10:34:05.202 [88.888.88.88:50490 1 9yy] [Img_Optm] need gather due to no new raw image found 10/13/21 10:34:05.203 [88.888.88.88:50490 1 9yy] [Img_Optm] gather_images bypass: no new image found 10/13/21 10:34:05.225 [88.888.88.88:50490 1 9yy] [Ctrl] X Cache_control -> public forced ( Setting: / ) 10/13/21 10:34:05.225 [88.888.88.88:50490 1 9yy] [Ctrl] Forced public cacheable due to setting: / 10/13/21 10:34:05.226 [88.888.88.88:50490 1 9yy] [Ctrl] X Cache_control TTL -> 300 10/13/21 10:34:05.226 [88.888.88.88:50490 1 9yy] [Router] get_role: 10/13/21 10:34:05.226 [88.888.88.88:50490 1 9yy] [Vary] role id: failed, guest 10/13/21 10:34:05.226 [88.888.88.88:50490 1 9yy] [Router] get_role: 10/13/21 10:34:05.226 [88.888.88.88:50490 1 9yy] [Router] get_role: 10/13/21 10:34:05.226 [88.888.88.88:50490 1 9yy] e?’° X-LiteSpeed-Cache-Control: public,max-age=300 10/13/21 10:34:05.227 [88.888.88.88:50490 1 9yy] e?’° X-LiteSpeed-Tag: e49_default,e49_URL./wp-cron.php/,e49_ 10/13/21 10:34:05.227 [88.888.88.88:50490 1 9yy] [Optm] bypass: Not frontend HTML type 10/13/21 10:34:05.227 [88.888.88.88:50490 1 9yy] End response -------------------------------------------------------------------------------- 10/13/21 10:34:05.333 [99.99.99.999:55880 1 wHu] [Ctrl] X Cache_control -> no Cache ( Admin page ) 10/13/21 10:34:05.466 [99.99.99.999:55880 1 wHu] [Ctrl] X Cache_control -> public forced ( Setting: / ) 10/13/21 10:34:05.466 [99.99.99.999:55880 1 wHu] [Ctrl] Forced public cacheable due to setting: / 10/13/21 10:34:05.466 [99.99.99.999:55880 1 wHu] [Ctrl] X Cache_control TTL -> 300 10/13/21 10:34:05.466 [99.99.99.999:55880 1 wHu] [Router] get_role: administrator 10/13/21 10:34:05.466 [99.99.99.999:55880 1 wHu] e?’° X-LiteSpeed-Cache-Control: public,max-age=300 10/13/21 10:34:05.467 [99.99.99.999:55880 1 wHu] e?’° X-LiteSpeed-Tag: e49_default,e49_URL./wp-admin/admin.php/,e49_ 10/13/21 10:34:05.467 [99.99.99.999:55880 1 wHu] End response -------------------------------------------------------------------------------- 10/13/21 10:34:05.706 [99.99.99.999:55880 1 wRm] e?’“ ------GET HTTP/1.1 (HTTPS) /wp-admin/admin-ajax.php 10/13/21 10:34:05.706 [99.99.99.999:55880 1 wRm] Query String: action=cff_ppca_token_check_flag 10/13/21 10:34:05.706 [99.99.99.999:55880 1 wRm] HTTP_REFERER: https://myownsite.se/wp-admin/admin.php?page=litespeed-toolbox 10/13/21 10:34:05.706 [99.99.99.999:55880 1 wRm] Cookie _lscache_vary: admin_bar:1;logged-in:1;role:99 10/13/21 10:34:05.999 [99.99.99.999:55880 1 wRm] [Ctrl] X Cache_control -> no Cache ( logged in user ) 10/13/21 10:34:06.135 [99.99.99.999:55880 1 wRm] [Ctrl] X Cache_control -> no Cache ( Admin page ) 10/13/21 10:34:06.143 [99.99.99.999:55880 1 wRm] [Ctrl] X Cache_control -> public forced ( Setting: / ) 10/13/21 10:34:06.143 [99.99.99.999:55880 1 wRm] [Ctrl] Forced public cacheable due to setting: / 10/13/21 10:34:06.143 [99.99.99.999:55880 1 wRm] [Ctrl] X Cache_control TTL -> 300 10/13/21 10:34:06.143 [99.99.99.999:55880 1 wRm] [Router] get_role: administrator 10/13/21 10:34:06.144 [99.99.99.999:55880 1 wRm] [Core] Silence Comment due to REST/AJAX 10/13/21 10:34:06.144 [99.99.99.999:55880 1 wRm] e?’° X-LiteSpeed-Cache-Control: public,max-age=300 10/13/21 10:34:06.144 [99.99.99.999:55880 1 wRm] e?’° X-LiteSpeed-Tag: e49_default,e49_URL./wp-admin/admin-ajax.php/,e49_ 10/13/21 10:34:06.144 [99.99.99.999:55880 1 wRm] End response --------------------------------------------------------------------------------
- The topic ‘Crawler not caching pages’ is closed to new replies.