crawler question

  • Resolved mcdeth

    (@mcdeth)


    4 months ago

    Hello. My crawlers complete their run within 30 minutes. On product pages, a custom function retrieves the latest xxx orders, processes them, and saves the output in transients. The first page load takes around 5 seconds (with a crawler timeout set at 20 seconds). However, I’ve noticed that less-visited pages, which haven’t yet been accessed by customers or spiders, load more slowly—even though they’re supposedly pre-cached by LiteSpeed.

    My question is: if page was crawled, shouldn’t it create transient? Let’s assume, to debug, a simple function that saves 1+1 equation into transients with unique key for each product.

    Additionally, in the attached image, crawlers are only enabled for WebP; however, there are cache misses on the disabled crawler (which was turned off months ago).

    View post on imgur.com

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Support qtwrk

    (@qtwrk)

    I am not sure what do you mean with transient , our cache doesn’t use transient at all

    if you disabled these crawlers , then browser with these condition will be missed, this is expected

    Thread Starter mcdeth

    (@mcdeth)

    Does the crawler trigger all page processes? And if one of these saves results to transients, will it be performed?

    Plugin Support qtwrk

    (@qtwrk)

    if you mean , does crawler go through all pages for caching , it does.

    I still don’t understand the quesiton about the transient…

Viewing 3 replies - 1 through 3 (of 3 total)
  • You must be logged in to reply to this topic.