Random noindex added in post

Hello!

Unfortunately, I’m not sure where the issue is. Could you please provide the URL of the page Google informed you about? I can then take a closer look ?? Thanks!

But after I disable noindex for attachments <meta name="robots" content="noindex" /> is gone for this page.

So I got more <meta name="robots" content="noindex" /> on page where it shouldn’t be. In options noindex enabled only for:
+ Apply noindex to every second or later page on the homepage?
+ Apply noindex to every second or later archive page?
+ Apply noindex to Date archives?
+ Apply noindex to Search pages?
+ Enable advanced query protection?
That was one of my most attractive page for google ??

• This reply was modified 3 years, 11 months ago by narwen.

Same problem here. The issue is that WordPress 5.7 introduced the wp_robots filter, as mentioned in this thread. I temporarily added the ‘remove_filter’ action and it removes what WordPress added. Hopefully the plugin devs will modify this in the next release.

More about WordPress wp_robots filter.

• This reply was modified 3 years, 11 months ago by boybawang.

Thanks! But I saw unexpected noindex on previous version too.

• This reply was modified 3 years, 11 months ago by narwen.

Hello!

Please disable advanced query protection, and see if that works. Be sure to clear your website’s caches after making changes!

You may also want to try disabling the archive pagination and search noindex types.
If that doesn’t work, another plugin or theme may also be messing with the queries. See if disabling plugins that are suspect works.

I’m not sure why this would invoke without provided a URL; otherwise, I could’ve given a straightforward answer. Nevertheless, I hope this helps!

Some report on my issue.
I’ve disabled advanced query protection and search page. But I still get randomly noindex in different pages. For example: https://7kingdoms.ru/story/speculative-game-of-thrones-cast-gallery/
Sometimes noindex disappears for itself, when I’ve for example update options, and I see noindex only via crawler monitoring of Yandex.Webmaster (which is more frequently and aggressive crawler than Google).

I did nothing with plugin but page in example do not contain <meta name="robots" content="noindex" /> just after <!-- The SEO Framework by Sybre Waaijer --> anymore. Could plugin change something by schedule? Or while posting new post?

Hello!

TSF does not change its values as long as the query and the page state are the same.

Please note that caching plugins may store pages with certain broken queries, where TSF may noindex it, whilst the caching plugin retrieves the same response for another not so broken query. You may find it beneficial to disable TSF’s Advanced Query Protection; although, I suggest tweaking the caching plugin better first.

Hello! I’ve already disabled Advanced Query Protection 2 weeks ago as you suggested and I bypass cache as logged in user while checking page for noindex meta. For now I disabled all noindex options and will check if I got any noindex pages in next week or two. Then I’ll start to enable it by one.

So. My tests are ended in failure. I liked plugin concept, so sad that I have to use something else.

Just reports at last. I have 1900 posts and last 3 months since I’ve used plugin noindex meta randomly appeared and disappeared on less than 10–15 posts. I have disabled all noindex options in plugin setting pages. I tried to enable forced index in posts settings itself. But problem still there:

<!-- The SEO Framework by Sybre Waaijer -->
<meta name="robots" content="noindex" />
<meta name="description" content="…" />
…
<!-- / The SEO Framework by Sybre Waaijer | 0.41ms meta | 1.78ms boot -->

Hi Narwen, we tried to replicate the issue but to no avail. There are a lot of variables that come into play in WordPress. We genuinely want to help you, but we arrived at a point where we need to see your website to move further. Note that we do not want admin on any other login-gated privileges, just URL we can access; ideally with URLs/pattern presenting the issues.

You can submit the URL privately, via our contact form. We do not have a history of ignoring problems of sweeping them under the rug, but at times, we need to see the URL and the environment to get a better idea what could be causing the issue.

I apologize for the problems caused, thank you for your understanding and your time.
Pierre.

Hello! I’ve wrote some detail via contact form week ago, hope it could clear something. Meanwhile I’ve moved on another plugin and do not have any unexpected noindex, so I think there are some tricky bug in how The SEO Framework add noindex. If I could help somehow to find this bug just let me know.

Hello!

I’ve received your email well and checked out the website.

This is undoubtedly a caching plugin issue, where it caches a late 404-response or comment-query response of the page, to both of which TSF applies the noindex. It is not a bug of TSF; rather, it’s a protective feature.

These queries can be evoked throughout the day by robots crawling your website. When the caches of your pages expire, a page request with such a query will cause the caching plugin to store a broken/incorrect version of the page. This version will then be presented from the cache when legitimate requests are made. It is why you “sometimes” get this result, because the caching plugin may store a correct version later. Although it may seem like TSF is behaving unpredictably, the cause lies elsewhere.

Unfortunately, I could not incontrovertibly verify this issue because you’ve switched SEO plugins before you sent the links I’ve asked you for three weeks prior. However, I highly doubt I’m mistaken.

There are 2 hard problems in computer science: cache (in)validation, naming things, and off-by-1 errors.

The only workaround is to tweak the caching plugin’s settings. It probably has to do with an option along the lines of “ignore query string…”. If no such option is presented, you may want to consider using a different caching plugin or reach out to the caching plugin authors and lead them to this topic.

There’s a good reason why neither www.ads-software.com nor WordPress.com (nor TheSEOFramework.com) uses static page caching, and this is exactly it: WordPress is a dynamic web app, not a static HTML generator. WordPress can present a page from an unlimited number of endpoints; TSF is there to steer robots in the right direction so that your visitors from search engines get the right page every time.

TSF has no prevailing bugs; it’s why I haven’t needed to update it since last year.

“But the other SEO plugin doesn’t have this issue,” you might say. That’s because TSF uses different logic: TSF relies fully on the Rewrite API and is taught to block known side-effects; other SEO plugins have adapted and modify that API instead. The benefit is that TSF has no bugs with WordPress when WordPress is used as it’s meant to be. The downside is that TSF profoundly reveals mistakes made by other plugins, and their users keep coming to me to help explain and fix that.

This is undoubtedly a caching plugin issue, where it caches a late 404-response or comment-query response of the page, to both of which TSF applies the noindex. It is not a bug of TSF; rather, it’s a protective feature.
…
It probably has to do with an option along the lines of “ignore query string…”. If no such option is presented, you may want to consider using a different caching plugin or reach out to the caching plugin authors and lead them to this topic.

I’m use nginx server FastCGI Cache. That isn’t wordpress plugin, this is part of server itself. It do not ignore query string in url. I expected that my envoirement not very common for wordpress sites, so I’ve checked pages as bypass cache user and yet I saw noindex on some pages. I’ve also used memcached for Object Cache and Cloudlare proxy. Could they affect on noindex? How I could check late 404-response myself on demo install?

you’ve switched SEO plugins before you sent the links I’ve asked you for three weeks prior

I’ve posted link to page with noindex in this thread a week before I switched plugin. I thought you don’t see any issue on page, now I see that you don’t noticed this post ??

• This reply was modified 3 years, 10 months ago by narwen.
• This reply was modified 3 years, 10 months ago by narwen.
• This reply was modified 3 years, 10 months ago by narwen.