SEMrush “integration”

+1

We understand you are concerned about the new integration with the SEMrush. You identified multiple queries that you want us to address and we’ll address them in order:

What information does this “integration” collect and transmit?
We track if the integration is enabled, if a connection with SEMrush is established (auth token), and the current country database that’s used for querying SEMrush. No personal data is transmitted.

Does it transmit information to SEMrush or other third parties if I have NOT created a SEMrush account?
While the SEMrush integration is disabled or not authenticated, it doesn’t send any information to SEMrush.

Related to both; we care about privacy very much. We’ve written about that here and here and here for example.

WHY is this feature turned on by default in 15.1?
The SEMrush integration is enabled by default because we are convinced it’s a useful integration and our users will find it very helpful.

@mazedulislamkhan:

The SEMrush integration is enabled by default because we are convinced it’s a useful integration and our users will find it very helpful.

You could not be more mistaken. Yoast’s recent determination to reinvent the wheel (in answer to questions I don’t think any of your users were actually asking) and the rush to roll out changes like this without any consideration suggests a deeply disturbing “We know best” attitude and a blithe disdain for the problems you may be creating for users.

@ate-up-with-motor thank you for your feedback. I also truly appreciate the time you’ve taken to ask your questions here.

At the same time, it makes me sad to learn that you think we suffer from a ‘we know best’ attitude. We’re actively listening to our users and what challenges they have when doing SEO with WordPress. One of the things we’ve heard a lot is that keyword research is hard. And we agree. It is hard. So we found a way to make it a bit easier, which is the integration with SEMrush.

Lastly, you mention you feel we’re blind to problems we may be creating for our users. Since SOLVING problems for our users is our goal, it seems like we’re doing something completely wrong in your eyes. Given that you post this in a thread about the SEMrush integration, my guess is that you feel the SEMrush integration is creating a problem for you. Can you please share with me, either here, in the WP Slack or via a DM on twitter (https://twitter.com/TacoVerdo/) what problem the SEMrush integration caused?

Thanks a million!

@tacoverdo: The problems it has caused me — and I sincerely hope you won’t think I’m being flip here, because I’m completely serious — has been to waste my time.

Website owners now live in an extremely complex and contentious regulatory environment. Lawmakers and regulators are writing new “privacy” rules almost every day that even quite small websites are expected to abide by. Those rules include an extremely expansive view of what constitutes “personal information,” what constitutes “sharing” that information, and what may be considered “selling” such information. The penalties for violating those rules even accidentally can be extremely stiff.

When I add a new piece of software or a new plugin, or if some existing product issues a substantial update, this means I have to audit it to figure out what information it might be collecting or sharing and with whom. I had never heard of SEMrush, but I had to find their website, review their privacy policy, and make updates to MY privacy policies EVEN THOUGH I HAVE NO INTENTION OF USING THEIR SERVICES. That took a great deal of my time and energy that I could ill afford.

Furthermore, I had not forgotten the Ryte integration issue last year, where I was never able to get a coherent answer from either Yoast or Ryte about what information they might be collecting. That integration too was automatically enabled. It did not leave me with confidence in Yoast’s good faith.

What I really don’t understand is why you would enable it by default. If you wanted to include it as an option because you genuinely think it’s useful, fine — default the option to “OFF” and let users decide for themselves. Since the Yoast SEO plugin already has a notification system, and since if I understand correctly the SEMrush service requires creating an account, the net result would have been the same, except that you would not have consumed more than an hour of my time and left me scrambling to put out a potential fire.

I want to add here that I’m not comfortable having the plugin communicating with Yoast, either. I prefer to limit my websites’ communications with outside servers for privacy and security reasons, and Yoast has driven home the importance of doing so. I’m still furious that the plugin incorporates a blog feed widget that automatically communicates with Yoast servers even when hidden and can’t be fully disabled through the plugin; I finally had to add a custom function to stop it from loading. It does not make me trust Yoast’s privacy practices or good intentions.

Again, I have no objection to making this functionality and these options available to users who want them, but features that communicate with external servers — including Yoast’s! — need to be opt-in.

Thank you so much for your elaborate answer. I now totally get where you’re coming from and why this caused so much frustration.

I’m not here to make promises I can’t keep, so I can’t and won’t tell you that we’ll change our whole approach to integrations built into Yoast SEO. Both because I don’t know and because it’s not my decision to make.

What I see in your story that I probably can influence is that we should make sure you don’t have to dive into the 3rd party’s website to find their privacy policy, if we already know that we’re not sending over any data unless you manually authorize the integration.

Finally, what I really want to stress is that we do care a lot about privacy. We have never, and will never, store our user’s data if we don’t have to.

All of that said, I will provide a summary of our conversation here to my colleagues. And again, thank you for taking an extra hour to help me understand what caused your frustration with this release. I appreciate it a lot!

Dear Community,

For those who don’t want SEMRUSH (and other bad bots) to track and extract info from your website, please add the following to your robots.txt file:

# Bad Bots
User-agent: semrushbot #website crawler
Disallow: /
User-agent: zoominfo #website crawler
Disallow: /
User-agent: woorank #website crawler
Disallow: /

Above should block the “bad bots” even if the Yoast SEMRUSH option is turned on.

For additional information on how to block SEMRUSH, please visit their website at https://www.semrush.com/bot/

Cheers!

@jetxpert Thanks!