Hi, I found a really good solution to this. Basically, when Googlebot encounters a file that isn’t there anymore it receives a 404 response. I found out this doesn’t actually mean the file is gone for certain, and so Googlebot will continue trying for that file over time.
The best way to deal with files/pages you remove is to return a 410 response, which means that it’s gone for good and so stop trying to access it.
There’s an easy way to deal with this by adding the following to your 404.php file:
header($_SERVER["SERVER_PROTOCOL"] . " 410 Gone");
If you don’t have a 404.php you can create one by making a .php file containing the following:
<!DOCTYPE html>
<?php
header($_SERVER["SERVER_PROTOCOL"] . " 410 Gone");
header("Cache-Control: max-age=2592000");
?>
<div id="main-content">
<div class="container">
<div id="content-area" class="clearfix">
<h1><?php esc_html_e('Page Not Found'); ?></h1>
</div>
</div>
</div>
That creates a very basic 404 page that returns a 410 response. I kept it basic so it doesn’t use much server resources to load.
You can then test this by accessing a url that doesn’t exist on your site, and then looking at the network tab with Chrome developer tools. You will see the 410 response.
Got this info from here (see last response)
https://stackoverflow.com/questions/41997261/wordpress-change-all-404-to-410-error-code
Hope this helps you and anyone else with Autoptimize causing 404 errors!
