robots.txt

  • hi there
    im using robots.txt and i use this code

    User-agent: *
    Allow: /
    Disallow: /*?s=
    Disallow: /wp-admin/*
    Disallow: /wp-content/*
    Disallow: /wp-includes/*
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes/*
    Disallow: /trackback
    Disallow: /comments
    Disallow: /category/
    Disallow: */trackback
    Disallow: */comments
    Disallow: /*.js$
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.cgi$
    Disallow: /*.html$
    Disallow: /*.htm$
    Disallow: /*.doc$
    Disallow: /*.flv$
    Disallow: /*.swf$
    Disallow: /*.txt$
    Disallow: /*.rar$
    Disallow: /*.zip$
    Disallow: /page/*
    Disallow: /page/*/
    Disallow: /pages/*

    it is work or i must add more ?

    i want just posts index in google

Viewing 5 replies - 1 through 5 (of 5 total)

  • Why not remove the Allow in robots.txt then just put in the header of
    Posts.php:
    <meta name=”robots” content=”index,follow”>
    Pages.php
    <meta name=”robots” content=”noindex,nofollow”>

    Thread Starter rody

    (@rody)

    can you be more specific

    Actually if you put in the Header.php for your theme in place of the original section for the robots enter this:

    <?php if(is_single()) { ?>
    <meta name=”robots” content=”index,follow” />
    <?php }
    else {
    <meta name=”robots” content=”noindex,nofollow” />
    };?>

    That should make the search engines only index and follow Single Post pages Everthing else will get ignored.

    Thread Starter rody

    (@rody)

    oh

    that is good

    where i must add this code ?

    Header.php.

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘robots.txt’ is closed to new replies.