robots.txt problems

  • Hello. I installed a plugin for my site that creates a site map and a robots.txt file. The sitemap checks out well, but when I use my google webmaster tools to check the robots.txt file, in six out of the eight lines it says: Syntax not understood.

    I was wondering if anyone might be able to help me with my robots.txt file. I have tried using various ones offered in the codex documentation pages in the SEO section on robots.txt, but none of them check out well with google. Furthermore, I’m just shooting in the dark, as I have no idea what I’m doing to my site with these files.

    Any help would be much appreciated.

Viewing 4 replies - 1 through 4 (of 4 total)

  • https://www.angerwise.com/robots.txt

    {\rtf1\ansi\ansicpg1252\cocoartf949\cocoasubrtf350
    {\fonttbl\f0\fswiss\fcharset0 Helvetica;}
    {\colortbl;\red255\green255\blue255;}
    \margl1440\margr1440\vieww9000\viewh8400\viewkind0
    \pard\tx566\tx1133\tx1700\tx2267\tx2834\tx3401\tx3968\tx4535\tx5102\tx5669\tx6236\tx6803\ql\qnatural\pardirnatural

    \f0\fs24 \cf0 User-agent: *\
    Disallow:}

    do NOT edit that file in MS Word. Do NOT save that file as an .rtf in Wordpad and then rename it. An .rtf (rich text format) file gets MS specific code added to it.

    Use notepad, or some other plain text editor.

    Thread Starter fifthhouse

    (@fifthhouse)

    HI there and thank you for your help.

    I’m not sure what I’m to do with this. I see that this is my robots.txt file, but when I check it with google sitemaster tools, I get the message as described above.

    Do you know how I need to edit the file to eliminate the errors?

    Moderator Ipstenu (Mika Epstein)

    (@ipstenu)

    ?????? Advisor and Activist

    Make a new Robots.txt file, in NOTEPAD (not word or wordpad or anything like that).

    No one can tell you what to set your robots.txt to since … well we don’t know what you want it as. What plugin did you install?

    As an example, here’s a working robots:

    User-Agent: *
Disallow: /cgi_bin/

# BEGIN XML-SITEMAP-PLUGIN
Sitemap: https://domain.com/blog/sitemap.xml.gz
# END XML-SITEMAP-PLUGIN
    Thread Starter fifthhouse

    (@fifthhouse)

    The plugin I installed is the DagonDesign site map generator.

    It creates a robots.txt file automatically.

    I don’t know what I want in the file, but I have read numerous times that you need one in order to make sure google doesn’t somehow mistakenly ignore your site.

    So I think I just want a basic robots.txt file, without specic things that it stipulates.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘robots.txt problems’ is closed to new replies.