Indexing shortcode content problems

I’m guessing it’s something in your setup, as Relevanssi expands all shortcodes by calling do_shortcode(). So, nothing unstandard in Relevanssi.

Are your shortcodes behaving? They should return their content, not echo it –?that’s something some shortcodes do wrong, and while it works in some situation, here it is bound to fail.

For the exact reasons of these results, I’d have to take a closer look at the code.

Hi Mikko,

Thanks so much for replying – sorry it’s taken me so long to get back to you.

My shortcodes are all behaving and returning the content properly.
Do you have any idea where to start / tips for debugging this?

Thanks!

Well, I’d start looking at the relevanssi_index_doc() function in lib/indexing.php – find the bit that says do_shortcode() there and see what’s happening to the shortcodes. What data Relevanssi actually sees there?

Some shortcodes can, for example, work so that they are not getting expanded in admin (WP Table Reloaded is one plugin that doesn’t register its shortcodes on admin side at all) where Relevanssi indexes posts. You’ll figure things like that out by checking the post content Relevanssi sees before indexing.

So, just add some var_dump(); and exit(); there and then try to update a post with a problematic shortcode and you’ll see what Relevanssi sees.

Thanks for that:

If I check Expand Shortcodes:
– dumping $contents just after $contents = do_shortcode($contents); – everything looks correct.
Self Closing shortcodes are expanded and the content shown. Enclosing shortcodes are expanded and the inserted + original content shown.
– dumping $contents at the end of the if (apply_filters('relevanssi_index_content', $ind... (just before closing bracket) shows the correct data too.

If I uncheck Expand Shortcodes:
– dumping $contents in the else { before strip_shortcodes – everything looks fine too.
Self Closing shortcodes are displayed unexpanded. Enlcosing shortcodes are displayed unexpanded with the other page contents.
– dumping after the strip_shortcodes does what it says on the tin, including stripping content within Enclosing shortcodes (I presume that is normal).
– dumping $contents at the end of the if (apply_filters('relevanssi_index_content', $ind... (just before closing bracket) shows the correct data too.

To summarise – with Expand Shortcode checked, by the end of the if (apply_filters('relevanssi_index_content', $index_content)) { statement the correct data is there, whether Expand Shortcodes is checked or not.

So somewhere after this point, pages with Self Closing shortcodes are getting stripped from the results (whether that have other content on them).

Does that make any sense to you? Could you point me to the next stop in the chain please?

If the pages and the content can be found in wp_relevanssi database and they are indexed correctly, then it’s time to look at search.

Print out the query that is given as a parameter for relevanssi_query_filter and run that on your database – does it return the pages that have self-closing shortcodes?

/*** I’ve included this top half just for my debugging process in case it helps anyone else ***/

I dumped $query on line 456 of search.php, just after the filters are added.

This gave me a query like SELECT *, title * 5 + content + comment * 0.75 + tag * 0.75 + link * 0 + author + category * 0.75 + excerpt + taxonomy + customfield + mysqlcolumn AS tf FROM MyTable_relevanssi WHERE term = 'neuberger' AND ((doc IN (SELECT DISTINCT(ID) FROM MyTable_posts WHERE post_type IN ('post','page','attachment','photo_archive','fast_facts'))) OR (doc = -1)) ORDER BY tf DESC LIMIT 500

The output matched the search results I see.

…

I then backtracked and searched the wp_relevanssi table – the pages there match the “faulty” results I am getting.

…

/*** **************************************************************************** ***/

Further backtracking – The problem is that the $post->ID is getting mangled.
My shortcodes are inserting sub-sections of content from a custom post type.
After $contents = do_shortcode($contents); in indexing.php the $post->ID value turns from the ID of the page displaying the content to the ID of the custom post type that is the source of the content. This seems to be the source of it all.

If I call the_ID() at the end of my shortcode, after the loop is ended and I’ve called wp_reset_query() it returns the correct page ID. Similarly if I call the_ID() in my page.php or loop-page.php

A hack around this is to add $myPostVar = $post->ID; after the if(bypassglobalpost) {...} else {...} and then change $post->ID to $myPostVar in the $wpdb->prepare statement on approx line 373.

Can you tell if this is caused by relevanssi or my code? I can’t see anything I am doing wrong in my shortcodes, but neither can I see why this would happen in relevanssi.

Thanks for all this help!

Hmm… hard to say. I do know Relevanssi changes the $post object to the $post currently being indexed in order to make shortcodes that depend on the $post->ID work better.

If you change

$contents = do_shortcode($contents);

in lib/indexing.php to

$temp_id = $post->ID;
$contents = do_shortcode($contents);
$post->ID = $temp_id;

does that solve the problem?

Hi Mikko,

Yes that works like a charm thanks!

Is there anything I can do to help work our whether this is specific to my site (and if so track it down), or whether this is something other users will have problems with?

Obviously if you can incorporate this into Relevanssi that would be great and save me having to re-edit the files after every new version, but I understand it wouldn’t be a priority if it is caused by a fault at my end.

Thanks again.

Oh, this is something I can implement in Relevanssi core, because it’s such a simple little safeguard against shortcodes that do unexpected things.

Great ??

I’ll mark this as resolved. Really appreciate you supporting even Free version users.

Had a somewhat similar problem in that search matches were being returned for posts/pages that didn’t contain the search item. Found that the $post global variable was being changed by do_shortcode($contents) in indexing.php and the solution was to save and restore the $post variable before and after the do_shortcode function (exactly as Mikko Saari did above). Was about to report this separately before I discovered this post. The problem exists in both versions 3.1.3 and 3.1.4 and may be a problem in earlier versions.

Basically if you have pages that include excerpts for other posts such as archive and category lists this could be a big problem.

Note generally these are pages you should be excluding from your search as the content is in the original post though this may not be an option for main pages and the like which may contain actual content.

Cheers
TheProfessor