search engines, permissions, and robots.txt

  • This is the first time I have used php for a web site.

    First of all, I have set the permissions on the php files to 604, and things seem to be working as expected. Will this cause problems for search engine spiders, like googlebot?

    For search engines, what happens if I disallow access to the php files so that they will not be spidered. Does this prevent search engines from picking up the content? Is disallowing access to php files a good idea? A bad idea? Clearly I have no idea!

    Is there a good tutorial on this somewhere

Viewing 3 replies - 1 through 3 (of 3 total)

  • If you want search engines to catalog your site, thereby including your site in search results, you should not block the search engine spiders. Your permissions should not cause a problem. If a human being can see them, so can googlebot.

    Googlebot and some other spiders access my index.php page, but they never seem to crawl into the rest of my pages. Is there something special I should add to the robots.txt file or some special tags I should add to my wordpress header template?

    site:
    https://fitness.draikes.com

    dnraikes, if googlebot knows where your site is, and it sounds like it does, it will eventually get to all of your other pages.

    Do not disallow access to .php files. There is no reason to, and it will keep googlebot and other spiders from moving around — WP is built in PHP, your pages ALL end in .php ??

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘search engines, permissions, and robots.txt’ is closed to new replies.