Robots.txt is dissallowing

I see several (and rather common) misunderstandings here.

I did a site audit from a online business and they stated that no pages are indexed at all?

If you paid for this “audit”, ask for a refund immediately as the tool you used or the person who did the audit for you has no clue what they’re doing. If it’s a free tool, stop using it immediately, as you can’t rely on its recommendations.

Google is indexing your site. This means Google is not “disallowed” or blocked from visiting your site. Rather, they’re able to visit your site and have found and added over 350 pages on your site to the search engine.

You can confirm the above from your Google Search Console dashboard, and also from this public search Google site:feedmekate.com

I was trying to research it and I typed my site followed by robots.txt and it is showing this:

From what I read online, this is not good. Could this be the reason why all of a sudden I am getting no traffic and also, very little impressions by google for my website?

It would be the reason and a major problem if your entire site was being disallowed for all bots or even Google alone. But that’s NOT what’s happening here.

I see three Disallow statements in your robots.txt.

The first two, Disallow: /wp-json/ and Disallow: /?rest_route= tells all bots to NOT visit URLs on your site beginning (respectively) with https://feedmekate.com/wp-json/ and https://feedmekate.com/?rest_route=.

These two URL paths are only used internally by WordPress for some technical job. No public URL on your site will have these paths in their addresses, so it’s a good practice to disallow search engines from following and indexing these paths.

The third Disallow rule tells one specific user-agent or bot called "AdsBot" to not crawl ANY pages on your site. I’m not 100% sure who owns this user agent, but the important thing here is that this bot being disallowed is NOT the same as the normal Google, Bing or any search engine’s bot.

1) In short, no, Robots.txt is NOT disallowing Google or any search engine from crawling and indexing your site. Use your Google Search Console to confirm whether Google’s various bots can visit your site or not.

2) From the outside, I can see Google is clearly indexing your site, with over 350 pages indexed already.

3) Your problem seems to be that of poor ranking, not crawling or indexing.

Good luck!

Hi There,

Thank you so much for clarifying that for me as I had no idea.

I understand my website is relatively new and poor ranking is likely, but what I can’t quite get my head around is that for a short period of time, the recipes that I published (once understanding keyword research and digging deep) I had many that were up there at the top.
Now, I can type that same exact recipe name and goggle only shows approx 89 searches relating to that, and my recipe does not even show whatsoever on any page at all.

Only 9 pages of search results come up (with that recipe) and in that, there is only a few handfuls of actual recipes with that name. Other ones they show are mostly unrelated.
i.e – I published a recipe for ‘Thermomix Cauliflower Mash’ – upon doing keyword research with Semrush it was very easy to rank for (a diff score of only 6), around 560 searches per month (Worldwide) and 360 of them in Australia (where I am).
I own a Thermomix and love it so did a recipe for it. It was showing in the recipe carousel and approx 4th on the first page. Now it doesn’t show at all, even if i go to the end of the total search results.
This is just one example. It has happened to every recipe bar 1.
Im just lost and don’t understand why google would show it then totally not show it…..
Sorry for rambling on, Im just so new to all this.
I have written my pages well for the best SEO and have given a lot of information, etc to try and rank and it was working but not now.
If anyone has any ideas that would be great.
I am working on trying to get some backlinks but that is not easy either when no body can even find you…..LOL
Thanks heaps
Kate