Google indexed hundreds of /go/ pages

If you do not want the search engines to index your URLs use your robots.txt to tell the SE not to index them.

Anything that is publicly accessible on your website will be indexed by search engine unless you tell them not to.

You also have to allow the search engines some time to read your robots.txt & update their listings.

Hi Ron,
here’s the foreword of my previous post, since I think you missed it:

Hi there,

1) I have set a disallow /go/ in my robots.txt
2) Using Yoast’s plugin I have set “no-index” for Simple Urls

I’ve already done the things you say (before publishing a single URL), but google has indexed them anyways.

You may need to add rel=”noindex” to the links in your content. nofollow tells the bot not to follow them vs not to index them.

You can’t noindex a link. That would have to be defined on-page, in the meta data.

You can remove the /go/directory for 90 days via webmaster tools. You will have to keep doing this every 90 days in order to keep the links out of google’s index.

I have the exact same setup and submit URL’s to remove to Google weekly manually:
https://www.google.com/webmasters/tools/removals?pli=1

It would same me some time if we could prevent indexing ??

Has anybody determined whether this keeps the links out? As Neo7 suggests it’s not possible to keep links out.

Adding rel=”noindex” to existing links would then be a hell of a job but could probably be done with a regex. From then on it would be great if Simple URLs would offer those URLs via the interface.

Hey mate,
I’m not John Mu, but as far as I know submitting URLs removal weekly is not a good thing to do. That should be a tool to be used every now and again.

I just got rid of this plugin and set up 301 redirects through a 3rd domain. It makes it much easier, cleaner and faster. Plus I don’t have to deal with the hormonal changes in the plugin developers moods and their willingness to fix problems.

Good luck

Thanks, I might rethink this setup indeed ?? SimpleURLs integrates nicely in WordPress, counts clicks in WordPress and Google Analytics. Migrating by creating a .htaccess shouldn’t be too hard. But your 3rd domain URLs are still indexed?

Ok so this what I’m doing and it works like a charm.

Get a new short domain, that looks like you have your own private URL shortening service. I got some .cc domains

Let’s say you get mack.cc

Now on mack.cc install:
https://www.ads-software.com/plugins/redirection/

Then upload an excel file with all the redirections following the instruction you’ll find on the plugin site or somewhere else.

For example:
mack.cc/1 —redirects to–> affiliate link 1
mack.cc/2 —–> affiliate link 2

Upload a robots.txt to block the entire mack.cc website from being indexed.

On mack.cc, on the redirection plugin setting you’ll find statistics telling you how many times your redirect links have been clicked.

Plus on your website if you’re using Jetpack stats all your mack.cc clicks will be tracked as external clicks and you can easily check how many times they have been clicked.

All your redirection are on mack.cc, so that lightens up your main site. All you have to do is to nofollow your mack.cc links on your main site because you don’t want to send “link juice” to an empty noindexed site.

That’s it.

Hope that helps.

It does, thanks!

Regarding the redirection plugin I heard it grows your database quickly because of all the 404’s that are generated by sites/bots. Any experience with that?

I’m not sure what you mean about growing my database,
could you please expand?

Thanks

I haven’t experiences it myself of course but heard/read somewhere (long time ago) every 404 hit creates a new database record causing at least the database to grow very fast and maybe even slow down WordPress.

Hmmm,
I don’t know what you mean by “database record”, where’s this database? I’m not sure what you’re referring to.

As I heard the redirection plugin adds records to the WordPress database on each hit.