Discourage search engines from indexing this site option and robots.txt

  • Resolved kenneth.von.rauch

    (@kennethvonrauch)


    11 years, 5 months ago

    Hey guys,

    I am just wondering how the “Discourage search engines from indexing this site’ feature works. Here’s what I mean. I was under the impression that if you tick the “Discourage search engines from indexing this site” option (in Settings -> Reading>, it’ll add the following scripting in robots.txt:

    Disallow: /*

    so that all my site is blocked for search engines. Problem is, once I enable that feature, it does not change my robots.txt file. Now here’s my questions. Which file or way that feature uses to discourage search engines from indexing my site?

Viewing 6 replies - 1 through 6 (of 6 total)

  • once I enable that feature, it does not change my robots.txt file.

    WordPress only has a virtual robots.txt, and it drops out when you have an actual one. So, I think you will have to remove your own file to get that feature to work.

    Thread Starter kenneth.von.rauch

    (@kennethvonrauch)

    Thanks for your reply. Well, I get that part., but the question remains,” If I tick the “Discourage search engines from indexing this site” option, how does search engine knowI did it? I mean which file is used to communicate to the search engine?

    @kenneth, I took a look at this option in WP 3.6.1.

    It adds the following to the webpages:

    <meta name='robots' content='noindex,nofollow' />

    Thread Starter kenneth.von.rauch

    (@kennethvonrauch)

    You made my day, Lorenzo. ?? That’s really how it works. Thanks a lot!!!

    When I use the same feature, it adds

    <meta name=’robots’ content=’noindex,follow’ />.

    Any idea how I can change follow to nofollow?

    raven

    (@ravengildea)

    @ardinent

    You can edit the meta tag by hand in the Dashboard:

    Appearance > Editor
    On the right, select the Header file (header.php)

    Find the tag in the code, make your changes, and click the blue Update File button.

    Note that if you are using a theme you downloaded (rather than a custom or child theme you built yourself) any changes you make to the theme template files will be overridden when you update the theme.

    To avoid this problem, you can make a child theme and make your changes to header.php there.

Viewing 6 replies - 1 through 6 (of 6 total)
  • The topic ‘Discourage search engines from indexing this site option and robots.txt’ is closed to new replies.