How to Deal with Millions of Records in 404 Redirection Log

Have you configured your expiry options?

https://redirection.me/support/options/

why am I still getting so many 404s

Please see https://redirection.me/support/tracking-404-errors/

however, that was more than a year or so ago, why am I still getting so many 404s

Google and other search engines won’t forget your old URLs for many, many years. I have a 12 yr old website which to this day still receives occasional probes from Googlebot after a shop page I briefly tested in the site over 11 yrs ago!

You can try to create a redirect matching the old URLs with a “Do nothing” action. That will prevent the 404s from these URLs from being added to the log.

Or you can do this with a match for any URL and Googlebot’s User Agent. Then monitor Google’s 404s on Google Search Console.

Thanks for the posts and advice both of you! @johnny5 I did take a look at the expiry options. I have Redirect Logs set to “1 week” and 404 Logs set to “1 day”. And I just jumped over to see what’s in the 404s log and there are already 36,000 entries. It doesn’t seem to be exactly 1 day though, as I see records in with a date stamp of today on the 27th and the oldest on the 25th. But nothing before then. However, I haven’t cleared this table in the past week, so it does appear to be working, just not exactly one day (not sure how it calculates).

Anyway, what’s strange is, I didn’t even touch these settings when I went to clear out the 404 logs when it had 3M records. So I would assume it was already set with 1 day expiry. So very strange, maybe some how my site got 3M hits within a 24 hr period to the dead links. I wouldn’t think so, but not sure what else would have caused it.

Ill monitor the 404s log now to see if it sticks to clearing out the logs automatically within a 1-2 day delay.

If you need to clear the redirection or the 404 logs out the WPS Cleaner plugin has an option for the redirection plugin for it. You can then purge both logs if needed.

I am having the same issue.

I deleted all the logs, set the expiry date for both logs and redirects to 1 day. Today is the 4th afternoon and I still see logs from the 2nd in the morning.

It seems that expiry date is being ignored.

The log is cleared using WP Cron, and will be slowly cleared so as not to affect performance of your site.

Please start a new thread instead of posting on a 5 month old thread.