Tips to deal with Website scrapers

  • I’ve been doing all sorts of things to deal with website scrapers, specifically one site that has become quite successful with just my content. At this point these scrapers do send traffic my way, so I’m hesitant to go with the dmca stuff to try to shut them down, so I just want to screw with them to either make their life harder but mostly to send more traffic my way. I’ve gone the route of putting stuff in my RSS feed footer, but whatever scraping software they use will edit that out once they have caught onto it(usually a day or two).Its a pain to change this every day. I’m thinking about putting my site name in the title, but I can’t figure out how to do this and I only want it in my rss feed. Anyone else have any more tips?

Viewing 1 replies (of 1 total)
  • Moderator James Huff

    (@macmanx)

    First and foremost, you need to “claim” you feed (which it sounds like you’ve already done) by installing something like Better Feed. Better Feed allows you to create a custom footer for your feed that appears below every post. You can fill the custom footer with things like a copyright notice, a link back to your blog, and even a digital fingerprint to track content theft.

    Next, install Bad Behavior to keep most content scraping bots from even visiting your blog.

    After that it, it’s up to you if you want to block bots either via user-agent or IP with your .htaccess file. I strongly recommend against IP blocking as most spam bots and content scraping bots use dynamically generated or spoofed IPs, so you’ll only wind up blocking an unsuspecting “victim”.

    AntiLeech still exists, and according to the creation date on the downloaded file, it was probably last updated 12/28/09, but whether or not it supports the latest versions of WordPress is unclear.

    If you find that your content has been stolen, this article provides tons of helpful tips.

Viewing 1 replies (of 1 total)
  • The topic ‘Tips to deal with Website scrapers’ is closed to new replies.