robots file – google sitemaps, all in one seo pack disallow

  • I have a standard vanilla installation of wordpress as well as ” in in one seo pack” and google sitemaps

    i have serached the forums and its “virtual” if your have google sitemaps installed, however the the default seems to be ” block everything from being indexed . i check my robots.txt:

    User-agent: *
    Disallow:

    privacy setting is “public” and allow indexing etc.
    Am I missing something obvious or do i just edit the robots.txt and “allow”

Viewing 4 replies - 1 through 4 (of 4 total)
  • Thread Starter highrankings

    (@highrankings)

    SLight mistake in the version of wordpress, its 2.92

    Moderator James Huff

    (@macmanx)

    That’s actually correct. In this case, the robots.txt file is disallowing nothing.

    Thread Starter highrankings

    (@highrankings)

    so how do i “allow” if the default is disallow when google sitemaps is installed.

    Quote: The virtual robots.txt generated by WordPress is used. A real robots.txt file must NOT exist in the blog directory!

    so wordpress owners always have to edit robots.txtafter google plugin is installed??

    Moderator James Huff

    (@macmanx)

    so how do i “allow” if the default is disallow when google sitemaps is installed.

    You really don’t need to. Try to think of Disallow: and Allow: with the word “only” after each. Disallow: is “Disallow only:” and in your case that would be “Disallow only: Nothing.” Allow: would be “Allow only:” and you don’t want to get caught in that mess. Stick with disallowing what you don’t want to allow, the bot will automatically assume that the rest is allowed.

    so wordpress owners always have to edit robots.txt after google plugin is installed??

    No, you don’t need to use that option. All that option does is insert the following line into the virtual robots.txt file:

    Sitemap: https://www.yourdomain.com/sitemap.xml.gz

    You can create your own physical robots.txt file (this will deactivate the virtual file) and add that line yourself.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘robots file – google sitemaps, all in one seo pack disallow’ is closed to new replies.

Tags