Indexing stalled – cron job working

  • Resolved patrickhs

    (@patrickhs)


    1 year, 8 months ago

    Just looking for potential reasons that may be the cause of my indexing seeming to have stalled out even while the cron job is working properly. I seem to be stuck at 4251 of 4515 products and I can’t find anything to indicate why.

    Could this be an API issue or something else beyond my control? I’ve stopped the cron job, force restarted, and then confirmed it’s working again. It has worked intermittently but again stopped around 11 hours ago and has not continued to index my products. I checked my Pinecone instance as well and all seems fine there.

    Any advice would be appreciated.

Viewing 5 replies - 1 through 5 (of 5 total)
  • Plugin Author senols

    (@senols)

    Hi,

    one way to find out is to try instant embeddings. Can you go to your Products page and select couple of nonindexed product and hit instant embedding button to see what error you are getting?

    Thread Starter patrickhs

    (@patrickhs)

    Thank you, I did try that yesterday and I got a very brief error mentioning openai api but it was gone before I could capture it and I haven’t seen it again so I can’t give anything usable for it unfortunately.

    Just for grins I decided to click “Re-Index All” and I increased the cron job frequency since I have plenty of resources to spare and it seems to be working very well right now with no issues.

    On a side note – is there a limit to indexing speed beyond my own resource limitations? I’m not sure how fast I should push it but with the size of the index I am building I really don’t want to wait 3 more days for it to finish if I can help it.

    Thread Starter patrickhs

    (@patrickhs)

    One thing I wanted to mention just looking at my Pinecone metrics is that I see that my upsert requests dropped to zero even while the indexing showed as active on my Woo site. I am guessing that this is API related. This morning however, I am not having any issues and re-indexing about 15 items per minute with no problems.

    Thread Starter patrickhs

    (@patrickhs)

    Just wanted to update this because I believe I have it figured out now. It is actually indexing everything as far as products go. The reason for the discrepancy which makes it look like it has stopped indexing short of all products is that it is having an error on a certain number of products which lack a description for various legitimate reasons.

    The products lacking a description are being skipped. Rather than listing them as skipped or failed it just seems to omit them entirely from the count. It is only after I go back and manually try them using the instant embedding button that they are recorded as skipped and the products indexed count goes up accordingly.

    The reason I think this is what is happening is because the count of products missing a description, which I have also not tried manually with the instant embedding button, seems to remain consistent with the number left to index on the products indexed counter.

    So, the issue just seems to be in the count of products indexed which I can live with. For a free plugin this is hardly something I’d complain about.

    Plugin Author senols

    (@senols)

    Hi @patrickhs

    Thank you so much for all these detailed information. I really appreciate it ??

    I think I got your point and now I am going to try to replicate the issue myself. Probably will be fixing this soon.

    Thanks again ??

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘Indexing stalled – cron job working’ is closed to new replies.