Hey @pglad,
Thank you for your feedback. You mentioned a few interesting things so I’ll respond to them below:
1. We actually don’t generate the robots.txt file ourselves, but hook into the one that WordPress Core generates and let users control it. In the past we did generate it ourselves though.
I’m not sure where you saw that message but I’m assuming that you saw it in the results of the SEO Analyzer. That probably means our remote analysis server wasn’t able to read your robots.txt properly. This could happen for any number of reasons. If you were to post the URL of your website, I’d be more than happy to take a look and see what’s going on.
It sometimes occurs that our server is blocked by a third-party security plugin or service (like Cloudflare and Incapsula) or by a hosting provider. In those cases, our server sees a “Blocked” or “Forbidden Access” page that is generated by those services. Because they are typically served as a 200 OK response and not a 403 Forbidden response, our server isn’t able to distinguish those from your actual website and bases the results of that invalid page.
I can’t say that is the case here without knowing the URL of your website, but it’s possible.
2. Same as the above. If our server is blocked, it would explain why it’s not finding those headers.
With regard to our TruSEO content analysis:
3. I’d love to know what the content of that article is so that I can check it out and run some tests on it. If there truly are no consecutive sentences, then that is a bug we definitely want to fix.
4. The fact that you’re seeing different results (22% vs 9%) is quite normal since every service has its own ways to identify passive voice, transition words, power words, etc. There’s no standardized set or approach for these so the results will always differ between products.
Content analysis is quite hard and it’s something that we’re continuously trying to improve. Your feedback above is helpful and we will definitely take it into account. As I said, I’m more than happy to look into all of these issues for you if you’re able to share the URL of your website and your latest post with us.
Cheers!