theselecter

Excellent! Thank you! ??

Thanks for the quick response Lisa!

On the Crawler File page I need to enabled the Crawler Cron and generate the Crawler File for this to work properly. Is that correct?

Thanks,
Dave

Sybre,

Thanks for the link to the article. It helped clear this up some.

I think where I was getting a little confused was that there are 2 robots. One is the robots.txt and the other is Meta Robots file. It does look like you can use both but need to be careful of conflicts.

Even though the NOINDEX page showed up in webmaster tools as Allowed for some reason, when I do a google search for info:https://mysite.com/page it does not show up. Also, only the robots.txt file showed up in webmaster tools. So I deleted that one and elected to go with the Meta Robots method for now.

So as far as I can tell, all is working well. ??

Thanks again,
Dave