Pages missing from sitemap crawler

  • Resolved jellejacob

    (@jellejacob)


    2 years, 8 months ago

    Hi,

    Seems like not al products are cached, even after sitemap refresh. Seems like the crawler is missing some pages.

    When searching the map it seems like not all pages are available?

    Thanks, Jelle

    The page I need help with: [log in to see the link]

Viewing 5 replies - 1 through 5 (of 5 total)
  • Plugin Support qtwrk

    (@qtwrk)

    Hi,

    Did you see these “missing” pages in your sitemap.xml ?

    best regards,

    Thread Starter jellejacob

    (@jellejacob)

    Hi,

    Thanks! Yes we did and it seems they are there. But for some reason some product pages showed up on the block list.

    Br, Jelle

    Plugin Support qtwrk

    (@qtwrk)

    what are the response header on these blacklisted URL ?

    Thread Starter jellejacob

    (@jellejacob)

    Hi, my hosting company solved the issue due to a block issue from the firewall if I have understand correctly. Thanks for your help!

    Thread Starter jellejacob

    (@jellejacob)

    Hi,

    It seems like the issue is here again. 244 pages from our sitemap appear on the blocklist, and we’re not sure why.

    Response header are indicating no-cache or litespeed cache > miss, but the litespeed_crawler database table is having a record in some cases? What should we do?

    Br, Jelle

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘Pages missing from sitemap crawler’ is closed to new replies.