Searchable Posts vs Posts

what is a registration ? i only use posts. only one type should exist

What I mean by “registration” is when using register_post_type()

By default and out of box, post, page and attachment are all public post types that aren’t excluded from search.

If you want ONLY the post post type to be enabled for both autocomplete, and instantsearch, we have filters available to set it as such, and automatically disregard any other post type from being indexed.

i only have one page on my site (everything else is a post), so that is not the reason the counts are different. and i still have more posts in wordpress than in the index.

• This reply was modified 3 months, 2 weeks ago by holisticremedysearch.

See my reply at https://www.ads-software.com/support/topic/number-of-posts-in-an-index/#post-17922311

For the sake of example, if you have 10 posts, and 1 page and those are the only post types getting indexed, but those 10 posts are extremely long, then your RECORD count is going to be more than your POST count, because of how Algolia splits content into smaller parts. It could end up being 4 records for every post, giving you 40 records, and 1 record for the page because that was just a quick biography about the site owner.

For what it’s worth, this isn’t something that WebDevStudios implemented ourselves when we forked Algolia’s original WordPress plugin. This is how Algolia implemented it themselves and we’ve been maintaining that workflow.

what do you mean split into smaller parts ? i don’t think anything i have is particularly long, not sure how to validate that anything is indeed long either. I just know everything after post 100 has just stopped indexing.

Just what it sounds like, honestly. To quote Algolia themselves a moment

To ensure good performance, Algolia limits the size of each record. Long content, like a detailed Wikipedia page, might be too big to fit into one of these records.

To work around this, divide long pages into smaller “chunks”. This not only helps you stay within the size limit but also makes your search more relevant. Break the page into sections or even paragraphs, and store each as a separate record.

Source: https://www.algolia.com/doc/guides/sending-and-managing-data/prepare-your-data/how-to/indexing-long-documents/

The splitting is a detail that the plugin already handles for you, and nothing that you need to do custom.

This act of splitting, creates more records than posts. Each “chunk” is its own record, but they all come from 1 post that got indexed. Thus 1 post may have 2 records, and so on.

I just know everything after post 100 has just stopped indexing.

Is this when trying to do bulk indexing? Any errors showing through?

i don’t think any of that long content applies to me. There is no error, just no new posts in the index.

Could be server errors as well like with exhausted memory as an example. However, I don’t have access to that information for your situation. If indexing is somehow stopping, then definitely something is probably happening. The question would be what.

Regarding the rest, not sure what to say at this point as I’ve explained as best i can how things work with both Algolia in general as well as how the plugin is interacting with things.

hmm maybe I should reinstall the plugin fresh ?

Admittedly nothing I can think of that doing that would solve. It’s all communication between your install and Algolia’s API and what, if anything, is causing an interruption to the processing of that content.

How many POSTS are you trying to get indexed overall? Note that I’m not asking about records, just the WordPress posts side.

right now we are at 153 and only 100 are showing up

If you know how, have you enabled any sort of WP/Error logging to see if something shows up in the logs for why it’s potentially failing at 100 exactly?

Also just in case, have you considered trying to remove ALL customizations that you’ve made, and let everything fall back to defaults, code wise, and see if it succeeds completely?

Alternatively, if you have a development copy of the website, trying to do things with only the Algolia plugin active is worth considering. We can help make sure that the development install gets its own index, so that it doesn’t “pollute” the live version.

i don’t know where any logging is, all i know about are the PHP logs from wordpress

yeah, why i wanted to revert to the clean slate. i can install on staging clean with new everything and see what is different, i can’t keep up with customizations anymore outside of the ones listed on the functions.php.

everything else is just referencing {{data.XXX}}

• This reply was modified 3 months, 1 week ago by holisticremedysearch.

i put together a staging site with a fresh install, separate API key, created 3 posts manually and all showed up instantly in the index.

i then uploaded 225 records via an import tool to create more posts and algolia took them all, so there must be something custom that has changed its ability to index automatically. this environment has no custom fields, so that is a consideration.

average record size is 568bytes STG versus 1KB in PROD

all i know about are the PHP logs from wordpress

Yes, that’s one of the potential sources. Turning on some of the PHP Constants from https://developer.www.ads-software.com/advanced-administration/debug/debug-wordpress/ if they haven’t been already, could provide some help here.

yeah, why i wanted to revert to the clean slate

Re-installing our plugin isn’t going to facilitate that, is what I was getting at on that front. The only “resets” that could be done in that vein would be removing any and all customizations from your work thus far, and just run the plugin as if you installed for the very first time, and set up your API credentials.

everything else is just referencing {{data.XXX}}

Since that’s templates, that don’t have any interaction with the indexing process, that’s fine.

Good to know that the test environment seems to have worked in full. That indicates that there must be something in the original spot that’s causing some interference or causing the indexing to stop and not able to resume.