help .htaccess

Hi mavachrt.

When you say you don’t know the source of the robots.txt file, do you mean you don’t know the url of your robots.txt file? ie… https://www.yoursite.com/robots.txt? If so, you can access your robots.txt file via ftp to make edits. Your robots.txt file can be found at this address, https://dellinspirons.com/robots.txt

May I ask what changes you are planning to make to your robots.txt file and why. I ask because a lot of users lay around with this file because they read an article online and wind up doing more harm then good.

thank you joedooley,
I want to know how to use the wordpress source code need to add to the robots.txt file or not? because this is my first time using wordpress.

With the robots.txt file the main purpose is to exclude the spiders from accessing those areas. This can be a fine line because you don’t want to block them from indexing your content which will hurt your SEO ratings.

To play it safe you could add something like I pasted below. I just updated my robots.txt file the other day and there are no blockages with this code.

The example the WordPress codex provides uses this code:

Sitemap: https://www.example.com/sitemap.xml

# global
User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /trackback/
Disallow: /comments/
Disallow: */trackback/
Disallow: */comments/
Disallow: wp-login.php
Disallow: wp-signup.php

If you are looking for an easy way to edit your robots.txt file you can download the WordPress SEO plugin by Yoast which I couldn’t recommend more because of the ease of use, how well the plugin is built and the effectiveness. After handling SEO on my own before which took up way too much of my time and now using the Yoast SEO plugin found here I usually have my Blog posts indexed within two hours as long as they are 400 word plus articles and my pages are SEO optimized.

# Disallowed and allowed directories and files

# Define website sitemaps
Sitemap: https://www.developingdesigns.com/post-sitemap.xml
Sitemap: https://www.developingdesigns.com/page-sitemap.xml
Sitemap: https://www.developingdesigns.com/attachment-sitemap.xml
Sitemap: https://www.developingdesigns.com/category-sitemap.xml
Sitemap: https://www.developingdesigns.com/post_tag-sitemap.xml
Sitemap: https://www.developingdesigns.com/author-sitemap.xml

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-content/
Disallow: /wp-includes/
Disallow: /comment-page-
Disallow: /trackback/
Disallow: /xmlrpc.php
Disallow: /feed/
Allow: /wp-content/uploads/

The link explaining the robots.txt file in the WordPress codex can be found here.

information is useful to me, thank you for sharing.