So, you spend hours of your own time, or hundreds of your own dollars creating content for your new website.
And what happens next? Some guy, or gal, with an automatic scraper program comes along and takes your content (time and money) for his scraper site.
You get penalized for duplicate content and make less of a return on your investment. And the scraper? What does he care? He’s got no time or money invested in his junk sites.
There really hasn’t been a lot you could do about these sites. Often they’re not even on domains registered with real information, so you can’t even find out who to send the copyright infringement notices to…
But, it appears someone has had more than enough of people stealing his stuff.
Check out AntiCrawl.com. From this site you can download a FREE script to keep the crawlers at bay while still allowing search engine robots and good traffic.
You will get a very simple script to add to your sites to:
1. Stop the scrapers right in their tracks by specifying the number of pages allowed to download per second.
2. Set a daily limit on the number of pages to be downloaded. Set it to 10 or 100 per day – the lazy bastards won’t bother coming back.
3. Let Google, MSN and Yahoo!’s Slurp index your site freely. (The REAL ones).