jacraver

@johnny5
If the change we implemented was to reset WP after the category is ignored.

This seems like you can enter anything in the category sections. Then when you click enter it essentially ignores the category and resets WP and then find the normal path through redirection plugin.

How would WordPress be implementing this redirect by itself?

Hey John (@johnny5) ,

Alright so I think that worked to solve the 404 issue of random URLs not returning a 404, but now we are noticing some other interesting issues more to do with the categories and subcategories section of the URL.

Take this URL for instance: https://rebeta1-attorney-at-law-magazine.pantheonsite.io/legal-marketing/networking/best-legal-conferences-to-attend

You can replace either of the two category sections “legal-marketing” or “networking” with random characters or add random characters to those sections and it will redirect you right back to the article almost as if it doesn’t even matter what the categories are.
Ex: https://rebeta1-attorney-at-law-magazine.pantheonsite.io/legal-marketing/ashdhdfsjfd/best-legal-conferences-to-attend, https://rebeta1-attorney-at-law-magazine.pantheonsite.io/ahsdsdfhdfd/networking/best-legal-conferences-to-attend or https://rebeta1-attorney-at-law-magazine.pantheonsite.io/legal-marketing/networkingashdjdskfdg/best-legal-conferences-to-attend

However if you add anything in the actual article title section not the category, this will 404: https://rebeta1-attorney-at-law-magazine.pantheonsite.io/legal-marketing/networking/best-legal-conferences-to-ahsdsfdjfhjdsattend

Do you know why when replacing the section of the categories and subcategories it is not returning a 404?

• This reply was modified 6 months, 1 week ago by jacraver.

Hey John,

So we implemented that on this test site: https://rebeta1-attorney-at-law-magazine.pantheonsite.io/public-articles/personal-injury/medical-malpractice/ajax/spa/spa/page_action/page_view_timing/page_view_event/aggregate/page/2

and that is now returning a 404 as well as all the other links that were indexing on Google’s side. We think that fixes the issue. We have not implemented it on the live site yet.

Does it look like that to you from the link above?

Thank you very much for taking the time to dive into this.

@cyber49 yeah, I imagine removing %postname% from the redirection migration tab would essentially break all the redirects themselves. So it will return a 404 because the redirect would be removed entirely.

@dzynit if yours matched, I would see how that would cause some issues. However, did you remove the tag entirely from the redirection migration tab, or did you make them match correctly?

For instance we are using the redirect migration function to redirect all of our old post structure, https://attorneyatlawmagazine.com/%postname% to our new post structure, https://attorneyatlawmagazine.com/%category%/%postname%.

I do believe the break is within the permalink migration functionality, but I don’t think removing the %postname% is my answer. Mine don’t match like you are saying either @dzynit so I am not sure that is the solution for our problem either.

Let’s keep putting our heads together though to see if we can figure this out. We have had this issue for well over a year.

Alright, I will try to keep the rest of this convo not in Google doc. Was using it for picture purposes.

So we understand that you think it might be another plugin/systems but we have done some diagnosing to pretty much assure it is Redirection for the 404 issue.

Here is an example of the site that only has Redirection enabled: https://redirectplu-attorney-at-law-magazine.pantheonsite.io/. When we go to one of the example URLs that should 404 it does not: https://redirectplu-attorney-at-law-magazine.pantheonsite.io/public-articles/personal-injury/medical-malpractice/ajax/spa/spa/page_action/page_view_timing/page_view_event/aggregate/page/2

Here is an example of site that has no plugins enabled: https://stripedwpon-attorney-at-law-magazine.pantheonsite.io/. When we go to the same example URL that should 404 it does: https://stripedwpon-attorney-at-law-magazine.pantheonsite.io/public-articles/personal-injury/medical-malpractice/ajax/spa/spa/page_action/page_view_timing/page_view_event/aggregate/page/2

Live site that has all plugin enables: https://attorneyatlawmagazine.com/.
Specific URL Example: https://attorneyatlawmagazine.com/public-articles/personal-injury/medical-malpractice/ajax/spa/spa/page_action/page_view_timing/page_view_event/aggregate/page/2

When we are on a dev environment like the first two examples, the issue still exists. This dev enviroment does not involve Pantheon/Varnish, Cloudflare and we killed all plugins including RankMath. Right when the Redirection plugin is enabled it causes the links not to 404.

One extra item we found out was it seems like the only links that are affected by the 404 issue are the URLs that involve subcategories. If the URL does not have a subcategory in it, it will 404 correctly.

What can cause this if everything besides Redirection is disabled?

Hey John,

I responded to your latest feedback here with the screenshots and exports at the bottom of the original document here: https://docs.google.com/document/d/1WcIPGwCoJD42tw_7wD8p_alhYu53ITxK6o2cS64v11A/edit?usp=sharing

Let me know if this helps at all to recreate the issue or diagnose further.

Hey John,

Thanks for the quick response. It looks like in your first line of the reply you say that our issue is “- any request to a URL that doesn’t exist returns a 404” however that is not the issue. The issue is “Appending any path to any post URL results in a 200 response instead of 404” There is something broken in the plugin that is causing everything with a redirect to result in a 200 once any text is added to the end of the URL or in the middle of the URL. This leads visitors to a page that really doesn’t exist and is an overview of all our recent articles (but not the 404 page).

(Note: Cloudflare and Varnish are not at play, because we have our test site reproduced on a dev environment that they cannot effect)

The redirects that are implemented work. However when you take any URL and append anything to the end or in the middle of the URL, it doesn’t lead to a 404, it results in a 200. This looks to happen on any URL that we have implemented through Redirection which is essentially almost all of our posts, unless they are old posts that we changed the URL and didn’t implement the redirect. 

The main reason we are using the Redirection plugin was because of the permalink structure feature to add the categories of the article into each URL.

One of the redirects in this is
https://attorneyatlawmagazine.com/unveiling-the-mystery-a-deep-dive-into-expert-tire-failure-analysis 

—> https://attorneyatlawmagazine.com/from-the-expert/unveiling-the-mystery-a-deep-dive-into-expert-tire-failure-analysis

The redirect works however append anything to the end of the URL like /ahsjdhs:
https://attorneyatlawmagazine.com/from-the-expert/unveiling-the-mystery-a-deep-dive-into-expert-tire-failure-analysis/ahsjdhs – this doen’t 404, it leads to a page with an overview of the recent articles that doesn’t even exist. 

Or enter the random appended path in the middle of the article and it returns the 404: https://attorneyatlawmagazine.com/from-the-expert/unveiling-the/ahsjdhs-mystery-a-deep-dive-into-expert-tire-failure-analysis

Somehow the hundreds of thousands of URLs that are being indexed and picked up in Google Search Console all don’t exist but the error of the URLs with appended paths not going to a 404 is somehow causing Google to pick them up:

• https://attorneyatlawmagazine.com/public-articles/personal-injury/medical-malpractice/jserrors/metrics/session_trace/jserrors/spa/metrics/aggregate/page/498 (Some URLs show items that make no sense like this one)
• https://attorneyatlawmagazine.com/legal/opinion/u-s-bop-the-least-accountable-agency’A=0%2Fpage%2F2%2Fpage%2F2%2Fpage%2F1263%2Fpage%2F1271%2Fpage%2F2%2Fpage%2F1270%2Fpage%2F1253%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F1253%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F2%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F2%2Fpage%2F2%2Fpage%2F1252%2Fpage%2F1252%2Fpage%2F1252%2Fpage%2F1252%2Fpage%2F1252%2Fpage%2F2%2Fpage%2F2%2Fpage%2F1252%2Fpage%2F2%2Fpage%2F1252%2Fpage%2F2%2Fpage%2F2%2Fpage%2F1252%2Fpage%2F1252%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F1253%2Fpage%2F1253%2Fpage%2F1253%2Fpage%2F1253%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F2%2Fpage%2F1253%2Fpage%2F1253%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F1253%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F2%2Fpage%2F2%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F2%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F1253%2Fpage%2F2%2Fpage%2F1254%2Fpage%2F2%2Fpage%2F505%2Fpage%2F2%2Fpage%2F2%2Fpage%2F2%2Fpage%2F498%2Fpage%2F498%2Fpage%2F498%2Fpage%2F2%2Fpage%2F498%2Fpage%2F498%2Fpage%2F2%2Fpage%2F498%2Fpage%2F2%2Fpage%2F498%2Fpage%2F2%2Fpage%2F498%2Fpage%2F499%2Fpage%2F499%2Fpage%2F499%2Fpage%2F2%2Fpage%2F2%2Fpage%2F2%2Fpage%2F499%2Fpage%2F2%2Fpage%2F2%2Fpage%2F2%2Fpage%2F499%2Fpage%2F498%2Fpage%2F2%2Fpage%2F498%2Fpage%2F498%2Fpage%2F2%2Fpage%2F2%2Fpage%2F2/page/2 (Some get insanely long)
• https://attorneyatlawmagazine.com/talk-of-the-town/minnesota-news/winthrop-weinstines-david-tanabe-selected-for-the-2022-blackshear-presidential-fellowship/page/497 (Some show paginations that have never existed like this page 497 of an article that is one page long. Additionally we removed this article a while ago but items still show up for older article like this that have been drafted in our system or removed completely)