BIG robots.txt problem :( !
-
Hi ya guys!
I will work on making this descriptive so my chances of getting it resolved are higher.
I’ve been trying to resolve this for hours now –
* I attempted to submit my sitemap to Google – however they came back and said it is being blocked by my robots.txt. Sadly, I even didn’t know what was that robots.txt until last night. So… I searched for my website in google.com to see my web-site there: and what?! It’s said: For this result, no description available because of robots.txt on the site – find out more.
* I called up my host (SUPERHOSTING) many times and told them this. They had guys looking at it for about 20-30 minutes. The only thing they could derive from this issue is that I have some plug-in. I deleted it. Again – the same. (one thing more: I didn’t have any plug-in before that…and it was the same: NO SEO …)
* I created a robots.txt in my domain.com\cpanel saying the following:
# robots.txt generated at https://www.mcanerin.com
User-agent: *
Disallow:
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Sitemap: https://www.pathofwarrior.com/sitemap_index.xmlAnd in Google Webmasters tool: the same problem:
Google tried to crawl these URLs in the last 90 days, but was blocked by robots.txt.
Googlebot is blocked by https://……example….and so on :(((
*BUT when I go to “Sitemap” in google webmasters, it says:
An error occurred while trying to access your Sitemap. Please make sure that it meets our guidelines and can be accessed from the location you provided and then resubmit.BUT WHEN I TEST IT ON: BLOCKED URL ADDRESS (in google webmaster tool) it says the following:
Analysis of robots.txt
value
Row 7: Sitemap: https://www…../sitemap_index.xml
result
Found/Discovered a valid reference to the Sitemap************************
So…….. what to do,really :/ ?
Remember, I do not have a visible robots.txt file anywhere on my server. I’ve went through every folder… and I created one in cpanel…but it doesn’t really work…
Can someone please help?
Thanks in advance,
Bobb.
- The topic ‘BIG robots.txt problem :( !’ is closed to new replies.