Googlebot and RSS

  • I’ll just throw this out there… In looking through the server logs, I notice that the googlebot is constantly looking for index.rdf, rss.xml and atom.xml, so it appears Google is trying to index feeds.
    I know the WP feed is accessed from wp-rss.php, etc. I also notice that Yahoo Feed Seeker chews up the php files no problem. Is there a way to feed googlebot the files it’s looking for? Or should I even be concerned?
    This is the first week of my first install of WP, and I must say I’m impressed. As far as the spiders go, I had just installed WP and within a half an hour, the place was crawling with them. Geez, I didn’t even have any content on there yet…

Viewing 7 replies - 1 through 7 (of 7 total)

  • Google is trying to grab everything, even if it’s not there. I wouldn’t worry ??

    I think it will still look for those files, no matter what you do, and besides, like podz said, it grabs wp-rss2.php and the others anyway if they are linked-to from your index page.
    You could set up rewrite rules to redirect requests to the right files, and if you use the 301 redirect, google should update the files it looks for since 301 is a “permanently moved” response.

    I had the same thing in my logs a few weeks ago. What I did was redirect all the files that google was looking for to wp-rss2.php

    Thread Starter gaszer

    (@gaszer)

    Thanks for the quick responses. Pardon my ignorance, but how would I go about setting up the redirect requests?

    Put this in your .htaccess file.
    Redirect permanent /blog/index.rdf https://www.yourdomain.com/blog/wp-rss2.php
    One entry like that for each of the files you want to redirect.

    gaszer, wp-rss2.php is the rss2 file generated by wordpress. for atom, there is wp-atom.php and wp-rss.php is the RSS 1.0 file. wp-rdf.php is the rdf file.
    So you will have to do a bunch of redirect rules, each of which resembles the one described by Joshua, one for each file that google is looking for.

    Thread Starter gaszer

    (@gaszer)

    Gracias!

Viewing 7 replies - 1 through 7 (of 7 total)
  • The topic ‘Googlebot and RSS’ is closed to new replies.