WordPress Site is not getting crawled

you have to be patient. google is a busy guy and it can take up to weeks/months till it comes to your website.

Analytics has little to do with Google indexing a site, and it’s best to choose one or the other when it comes to www or not.

In your Google Webmaster account (login not through GoDaddy but directly to your Google account) analyze your robots.txt file (though it looks fine from here), check for crawl errors, make a sitemap with WordPress ? Google XML Sitemaps ? WordPress Plugins and submit that to Google, too.

How long has the site been up? Got any links to your site yet?

I’m guessing the answer to the first question is not very long at all, if the date on this post is anything to go by:
https://thegolfstudent.com/43/hello-domain

So yes, just be patient and things will start happening for you.

Couple of things to do while you wait though…

It’s a minor point, but in your robots.txt you’ve got:

User-agent: *
Allow: /

There isn’t an “allow” in the robots exclusion protocol. As, well, it’s an exclusion protocol after all. ??

To be fair, I would think that the major search engines would still understand what you’ve got just fine – but you might as well just delete your robots.txt file entirely if ‘allow everything’ is what you’re trying to say. More info on robots.txt here:
https://www.robotstxt.org/robotstxt.html

You’ve also got a nofollow on some of your internal links (ie the ‘recent comments’, which I’d be inclined to remove if it were my site. If a search engine looks at a page and finds two links to the destination page, one with a nofollow and one without, what happens? I’d guess they’d err on the side of caution and treat it as a nofollow.

Just my 2p anyway.

Thanks for your help guys, I am raking my head with this thing. Other people’s perspective and helps is surely appreciated.

Still don’t know why its giving me errors in the webmaster crawl feedback though. Seems like most other things are okay. Bugging me, haha!

Hey Mendelyano,

I need this help too!! Where do I find/check the robot.txt file (is it in Filezilla, WordPress, or my Bluehost account?? and the no follow stuff- where do I check that?

how should these be set to get maximum SEO

thanks in advance!
Dave

Unsure, this is why I asked it here. Hopefully we can resolve together

Weird thing is that all these forum postings are getting crawled and indexed in google but not the site!

davidblawn: read Search Engine Optimization for WordPress ? WordPress Codex

Mendelyano: obviously, the WordPress forums are much more extensive, content-rich, linked to and with other sites and updated constantly, and thus deemed important by the Google bot than your own website(s).

Yes, hehe I need just a portion of the magic juju on my site now. I will read the link you gave, thank you for the input

I read the link but I am still finding some issues. I notice that on the google webmasters account area I am having errors being crawled and indexed. The diagnostics tool identifies crawl errors in link that I dont even see any longer. The issue was that when I first imported the blogger site to WP the permalinks were incorrect so they were fixed. I am thinking that this is when Google tried to crawl my site and had errors.

I had the blogger site redirecting traffic to thegolfstudent.com but I believe it was causing a poor redirect so I have since turned it off so the blogger site is up and fully viewable again.

I was wondering, I only had about 20 posts prior to the importing. Should I just delete the pictures and simply report them into WP? I don’t see any other links from WP to Blogger which is why I am having so much trouble understanding where the problem are.

Thanks