Jili tongits download for pc.Claim Your Free 999 Pesos Bonus Today

Resolved bkwineper
(@bkwineper)

4 years, 1 month ago

I seem to have an issue with the Yoast XML Sitemap. It seems that it truncates some post URLs and only shows the first “category” slug of the post URL.

This seems to lead to many 404 hits from bots trying to access these non-existent URLs, presumably after having found them in the site map. These also show up as errors in the Google Search Console (the site map has been submitted to GSC):
“Submitted URL has crawl issue”
or
“Submitted URL not found (404)”

In one case, the (truncated) URL that is shown in the sitemap also causes an infinite redirect loop:
https://www.bkwine.com/reviews/wine-of-the-month/
(there’s no redirect on that URL so the loop must be created by some internal redirection function in WordPress)

There are numerous URLs like this:
https://www.bkwine.com/reviews/producer-recommendations/australia-wineries/
https://www.bkwine.com/sv/reportage/vintillverkning-vinodling/
https://www.bkwine.com/reviews/producer-recommendations/australia-wineries/
(For these three, and a few more, I have created a redirect to the real archive page so they no longer return a 404.)

These URLs don’t really exist on the site. They look like category archives (the slugs are all archives) but a correct category archive URL should have /category/ in it.

Looking at the site map it seems quite obvious that this is post URLs that have been truncated so that the actual post slug has been chopped off, thus showing URLs that certainly should not be in the sitemap.

I have tested this with all other plugins disabled – except Yoast SEO – and regenerating the sitemap (turning it off, clearing cache, turning it on) and the resulting sitemap still has these errors.

You can see an example, showing mostly URLs ending in the /news/ category slug, but also the /reviews/producer-recommendations/australia-wineries/ slug, in this screenshot of the XML sitemap page 4:
https://www.bkwine.com/wp-content/uploads/2020/10/sitemap.jpg

You can of course access the sitemap on https://www.bkwine.com/sitemap_index.xml

You can see that all those /news/ URLs relates to different pages since they have varying number of images.

Most of these errors seem to be on page 4 of the sitemap (it’s hard to know if it is all). Looking at the last-modified date, they are all last modified on almost the same minute. One possibility is that this could have to do with an import of posts from an older blog (on Blogger) that might have been done at that time.

But that does not explain why the Yoast site map shows a truncated URL instead of the full URL.

It is certainly concerning since it reduces the indexation of the site and also causes Search Console errors and lots of 404s.

The page I need help with: [log in to see the link]

Viewing 4 replies - 1 through 4 (of 4 total)

Pcosta88
(@pcosta88)

4 years, 1 month ago

Hi,

To clarify, for some URLs the sitemap is showing them without the /category/ attached to it. For example, on https://www.bkwine.com/post-sitemap4.xml we can see the URL is: https://www.bkwine.com/reviews/producer-recommendations/australia-wineries/.

However, when we click on it the /category/ part is generated: https://www.bkwine.com/category/reviews/producer-recommendations/australia-wineries/.

You are expecting that /category/ URL to appear as part of the URL in the sitemap?

Thread Starter bkwineper
(@bkwineper)

4 years, 1 month ago

Sorry, I was perhaps not clear. And thank you for asking the question…:

For:
https://www.bkwine.com/reviews/producer-recommendations/australia-wineries/
I have created a redirect in order to avoid 404s to
https://www.bkwine.com/category/reviews/producer-recommendations/australia-wineries/

The same for the other ones in the list.

Before I created that redirect https://www.bkwine.com/reviews/producer-recommendations/australia-wineries/ gave a 404.

The https://www.bkwine.com/reviews/producer-recommendations/australia-wineries/ that is shown in the sitemap should not exist. It should have a final part that comes from the post title, see screen shot:
https://www.bkwine.com/wp-content/uploads/2020/10/australia-winery-archive.jpg

However, I now discover that this post DOES NOT HAVE that final part of the URL:
https://www.bkwine.com/wp-content/uploads/2020/10/mclare-vale-post-edit.jpg

As you can see in that screen shot, the permalink just ends with //. The title between the // is missing.

So, it seems (so far) that the problem is not with Yoast sitemap but with post URLs missing the unique last part of the URL. It seems very likely that this is an error that happened when importing posts from the old blog.

So now I have a real headache. How do I find those posts that have URLs that end in //. (It’s not like I can search the posts for “//”…)

Anyway, it seems this is not your problem, at least for the moment. Unless one should consider it an issue that the sitemap discards one of the final “/”.

I will have to try and figure out a way to find all those // posts.

Plugin Support Michael Ti?a
(@mikes41720)

4 years, 1 month ago

Hi,

We’re not sure if this is an issue with the Yoast SEO plugin but rather with the permalink settings that you’ve set when you imported posts from your old blog.

Just to clarify, also, with the Yoast SEO plugin, can you clarify if you’ve removed the categories prefix in the SEO > Search Appearance > Taxonomies section? You can learn more about it here.

Also, so you’re expecting the final link to be something like:

https://www.bkwine.com/category/reviews/producer-recommendations/australia-wineries/generational-farming-at-fox-creek-wines-mclaren-vale/

Is this correct?

Thread Starter bkwineper
(@bkwineper)

4 years, 1 month ago

Hi Michael,

After much investigation, I have now found what is the problem.

It is not caused by Yoast SEO.

The issue was that when a large number of posts were imported from an old Blogger blog something must have gone wrong, without any error being obvious or reported by the importer.

The result was that some of the imported posts had an empty post_name in the WP post table. “post_name” simply failed to register in the table and was left empty. (You can see that here: https://www.bkwine.com/wp-content/uploads/2020/10/mclare-vale-post-edit.jpg)

Therefore, as an example, a post that should have been assigned the permalink:
https://www.bkwine.com/reviews/producer-recommendations/australia-wineries/generational-farming-at-fox-creek-wines-mclaren-vale/
was registered with an empty post_name and thus became:
https://www.bkwine.com/reviews/producer-recommendations/australia-wineries//
(nothing between the two ending // and the double / being automatically reduced to a single one when browsing).

This “incomplete” URL was listed in the Yoast sitemap and, of course, caused 404s and Google Search Console errors. The URL did not exist (=404) and the sitemap listed incorrect URLs (=GSC error).

I have now manually corrected all these posts (some 70 posts) and made sure they have post_name. The sitemap is now regenerated and correct. Well, it always was “correct” although the URLs were not.

Sorry to have bothered you with something that was not a Yoast issue, but your questions here certainly helped me find the error, so many thanks for that!

Viewing 4 replies - 1 through 4 (of 4 total)

The topic ‘Sitemap truncates URLs and shows URLs that don’t exist’ is closed to new replies.