• Resolved harish_das

    (@harish_das)


    Hi
    I have been getting crawl errors for posts which no longer exist on my blog. It comes as not avaialable. And it come from many links. So error list keeps on increasing. I use Google XML sitemaps to generate sitemaps. I deleted my old sitemap and created a new one manually but still no success. Do I need to delete any other files whenever I delete or modify any post?
    Thanks

Viewing 6 replies - 1 through 6 (of 6 total)
  • Once Google knows about a page it will periodically check to see that it is still there (that’s what the crawling is all about).

    It will get the message eventually that it doesn’t exist, but it does usually takes a couple of weeks!

    Patience is not just a card game.

    Peter

    It will take a long time, more than likely many months. We still have old links to some files appear in Google after a long time, and the google bot visits a few times a day.

    We wrote a small php script that when an (old) post id was attempted to be accessed, it did a 301 (permanent redirect) to the new link. That way, it caters for google and all visitors, as you may have links from other sites.

    jpomerantz

    (@jpomerantz)

    Pete – can you share the php? I am having a similar issue with non-existing pages that are months old.

    Thanks

    Jay
    https://www.theparentschool.net

    pete_398

    (@pete_398)

    Jay, the following is not the entire script, but a small section of code. You would normally place this in your ‘404’ (not found) file.

    <?php
    
    $http_host = $_SERVER["HTTP_HOST"];
    $request_uri = $_SERVER['REQUEST_URI'];
    
    //For posts/pages which no longer exist, setup array to point to new url
    $old_url = 'https://' . $http_host . strtolower($request_uri);
    
    $urls = array(
       'https://example.com/wordpress/?p=1' => 'https://example.com/?p=1',
       'https://example.com/wordpress/?p=13' => 'https://example.com/?p=3',
       'https://example.com/wordpress/?p=18' => 'https://example.com/?p=4',
       'https://example.com/wordpress/?p=22' => 'https://example.com/?p=5',
       'https://example.com/wordpress/?p=29' => 'https://example.com/?p=6',
       'https://example.com/wordpress/?p=34' => 'https://example.com/?p=7',
       'https://example.com/wordpress/?p=39' => 'https://example.com/?p=8',
       'https://example.com/wordpress/?p=43' => 'https://example.com/?p=9'
       );
    
    if (array_key_exists($old_url, $urls))
    {
    	header( "HTTP/1.1 301 Moved Permanently" );
    	header("Location: $urls[$old_url]"); /* Redirect browser */
    	/* Make sure that code below does not get executed when we redirect. */
    	exit();
    }
    
    //Other code below ...
    
    ?>
    jpomerantz

    (@jpomerantz)

    Thanks Pete – am I correct to assume that what you provided will be what I need – I just need to change the page names and redirects?

    pete_398

    (@pete_398)

    Each site will be different. I had a load of other code that was pretty much relevant to only that site, so didn’t post it.

    I’m pretty sure that that code will work okay, but I haven’t tested it. Best to place it in a php file, modify it with ‘echo’ statements to display those variables, then you will be confident it is doing what you want it to do.

    You may need to make sure the echos are before e redirect I think, and also before a ‘header’ from memory.

    If it doesn’t work, post here, and myself or someone else will be able to help hopefully.

Viewing 6 replies - 1 through 6 (of 6 total)
  • The topic ‘crawl error for posts which no longer exist’ is closed to new replies.