Remove ‘X-Robots-Tag: noindex’

I’ve had to deactivate it. Silly to have 124 pages ‘noindex’. Still waiting for an answer, anyone..

This has been answered before https://www.ads-software.com/support/topic/xml-sitemap-x-robots-tag-noindex-follow/

Conclusion is the sitemap must have noindex header.
Your problem is something else, unrelated to that tag.

noindex means the sitemap itself shouldn’t show up on SERPs not that the pages on it shouldn’t. Google unable to fetch has nothing to do with the no-index tag.

It’s just telling you Why it won’t be indexed.

It’s already explained for sitemaps.

Other URLs which have ‘noindex’ are feeds. Yoast SEO sends ‘X-Robots-Tag’ because it’s XML content. I think that these pages don’t have content which should be part of SERP. More details – https://github.com/Yoast/wordpress-seo/issues/10262

This thread has been marked as resolved due to lack of activity.

You’re always welcome to re-open this topic. Please read this post before opening a new request.

Thanks for understanding!

@stodorovic No, that’s the problem. The presence of “noindex” header on sitemap nowadays causes Google Webmaster to fail to index the actual sitemap. I’ve been fighting it for forever – Google now refuses to accept my sitemaps, the tag needs to be removed.

@arcivanov Sitemap files aren’t web pages (HTML content) and it doesn’t make sense to show them into SERP. I didn’t notice any issue related to sitemaps (and X-Robots-Tag) on my websites.

You could see these discussions:

I’m sorry but it’s nonsense. The xml sitemap works as intended, the robots header is to keep the sitemap *itself* out of the search results.

Since this comes up from time to time — it’s fine to use the x-robots-tag HTTP header with “noindex” for XML sitemap files. They don’t need to be indexed to work as sitemap files, they’re more like robots.txt files (made for machines) than like HTML pages (made for indexing).

Since Yoast SEO 11.7 (just released):

• Remove the noindex from feeds as this causes issues for podcasts and other feeds.

Perhaps it will help you to resolve your issue,
I have just deleted some of lines from yoast plugins file

/wp-content/plugins/wordpress-seo/inc/sitemaps/class-sitemaps.php

I have removed 2 Lines from this file where ‘X-Robots-Tag’ found.

And Now sitemaps are being indexed… ??

@myalexu I have done the same and it had zero effect on whether Google Webmasters actually accepted the sitemap. Sad but true.

I repeat, “X-Robots-Tag: all” or removing “X-Robots-Tag” entirely had no effect whatsoever on whether GWM could fetch the sitemap.

It’s some internal mystery bug in GWM nobody cares to fix.

• This reply was modified 5 years, 2 months ago by arcivanov.

@arcivanov @myalexu

I’m not an SEO expert and I’m trying to fix the no index issue.

Once I request to index my page with google search console it appears! My web host is 1and1

“No: ‘noindex’ detected in ‘X-Robots-Tag’ http header”

Should I delete yoast and install another SEO plugin ? Can some assist me with a detailed answer.

THANKS GUYS !!!!

No need to delete Yoast, can you send your web link ?
Let’s take a look.

https://globalfootballfans.com/

@myalexu https://globalfootballfans.com/

@myalexu

https://globalfootballfans.com/

I am having the same issue.

https://www.explorehollywood.com/sitemap_index.xml

From Google Webmaster Tools

Availability: Excluded by ‘noindex’ tag
Indexing allowed? No: ‘noindex’ detected in ‘X-Robots-Tag’ http header

Micah

@myalexu @websperations hopefully myalexu can help us