@nikkoboy – Interesting, but typically if the content can be indexed by google, its already public and not protected in the first place. To be clear what I mean, google caches copies of the pages and serves them directly to users in some circumstances. Think about amp and the View Cached option that google offers, both are copies google keeps and serves from their end. Any user viewing from those sources or similar, or even browsing from facebook app browser would all encounter the full content just as the bot did.
In my honest opinion, have public content & private content. The private content should be expected to be private at all times by its nature. If you want google to scan it, it should likely be in the public category.
This could maybe be done, but I’m not convinced of the overall benefit outweighing making a feature like this work flawlessly with all the various hosting/caching/optimization setups we would have to work around.
On our radar now though, so I guess we will try to keep an open mind for the future.
