Questions about Crawler ability.

  • Resolved monitor

    (@oomskaap)


    5 years, 10 months ago

    If my crawler crawled for over a week and several thousand posts are crawled, and I then “purge all cache” are those crawled pages then removed and obsolete?

    So it has to restart all over again? do I understand this correctly?

    Secondly, it would be AMAZING if you can include some post limit to the crawler.
    For example if we have 10 000 products, but we only want the last 1000 by latest date be crawled.

Viewing 1 replies (of 1 total)
  • Plugin Support qtwrk

    (@qtwrk)

    Hi,

    Yes , purge all cache will purge everything.

    That’s good suggestion , but for now , you can use a sitemap to tell crawler what to crawl as workaround.

    Best regards,

Viewing 1 replies (of 1 total)
  • The topic ‘Questions about Crawler ability.’ is closed to new replies.