Blocking lechers and site rippers with mod_rewrite and php, MySQL

[remdex]

Hi,

Just wanned share my experience fighting against site rippers and lechers. I won't paste link to my gallery, just share some solutions protecting your own gallery's from rippers and traffic thief's.

First part solutions consists of main mod_rewrite rules:
(Dont forget replace <variable> with your site variables )

Code Select Expand


RewriteEngine On

#Disable HotLinking
RewriteCond %{REQUEST_FILENAME} .*jpg$|.*bmp$|.*jpeg$|.*gif$|.*png$ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !<your_site>\.org [NC]
RewriteCond %{HTTP_REFERER} !<friend_site>\.lt [NC]
RewriteCond %{HTTP_REFERER} !youtube\.com [NC]
RewriteCond %{HTTP_REFERER} !google\. [NC]
RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]
RewriteRule ^.+ - [F]

#Disable direct access to fullsize script
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{QUERY_STRING} fullsize=1$
RewriteRule ^.+ - [F]

#Image can be accesed only from fullsize script.
RewriteCond %{REQUEST_FILENAME} .*jpg$|.*bmp$|.*jpeg$|.*gif$|.*png$ [NC]
RewriteCond %{REQUEST_FILENAME} albums [NC]
RewriteCond %{REQUEST_FILENAME} !thumb [NC]
RewriteCond %{REQUEST_FILENAME} !normal [NC]
RewriteCond %{REQUEST_FILENAME} !userpics [NC]
RewriteCond %{HTTP_REFERER} !google\. [NC]
RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]
RewriteCond %{HTTP_USER_AGENT} !google [NC]
RewriteCond %{HTTP_REFERER} !fullsize=1
RewriteRule ^.+ - [F]

#Tricky rule Witch i will explain later
#In gennaraly it passes all request to fullsize images to php script where is implemented anti leeching logic.
#Comment out this rule if you wont use script fullsize.php script.
RewriteCond %{REQUEST_FILENAME} .*jpg$|.*bmp$|.*jpeg$|.*gif$|.*png$ [NC]
RewriteCond %{REQUEST_FILENAME} albums [NC]
RewriteCond %{REQUEST_FILENAME} !thumb [NC]
RewriteCond %{REQUEST_FILENAME} !normal [NC]
RewriteCond %{REQUEST_FILENAME} !resample [NC]
RewriteCond %{REQUEST_FILENAME} !userpics [NC]
RewriteCond %{HTTP_REFERER} !google\. [NC]
RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]
RewriteCond %{HTTP_USER_AGENT} !google [NC]
#RewriteCond %{HTTP_REFERER} fullsize=1
#RewriteRule ^.+ - [F]
RewriteRule ^(.*)$ fullsize.php?path=%{REQUEST_FILENAME}


#block site rippers agents
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
    RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
    RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
    RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
    RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
    RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
    RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
    RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
    RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Jyxobot [OR]
    RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
    RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
    RewriteCond %{HTTP_USER_AGENT} Twiceler [OR]
    RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
    RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Mail.Ru [OR]
    RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
    RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
    RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} \'Mozilla [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
    RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
    RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
    RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
    RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
    RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
    RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
    RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
    RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
    RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
    RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Twiceler [OR]
    RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
    RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]
    RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
    RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
    RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
    RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
    RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Yeti [OR]
    RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
    RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
    RewriteCond %{HTTP_USER_AGENT} ^wget [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
    RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
    RewriteCond %{HTTP_USER_AGENT} ^Zeus
    RewriteRule ^.* - [F,L]



fullsize.php script code (put it in root directory):

Code Select Expand


<?php
$time_window_seconds=35;
$max_hits_per_ip_per_window=2;

define('IN_COPPERMINE', true);
require('include/init.inc.php');
cpg_db_query("INSERT INTO ipleechcheck (ip,hittime) VALUES ('{$_SERVER['REMOTE_ADDR']}','".time()."')");
cpg_db_query("DELETE FROM ipleechcheck WHERE hittime < ".(time()-$time_window_seconds)." ");
$result = cpg_db_query("SELECT count(*) as total FROM ipleechcheck WHERE ip = '{$_SERVER['REMOTE_ADDR']}' ");

        if (mysql_num_rows($result) === 1) {
                $row = mysql_fetch_assoc($result);
                $IPCount = $row['total'];
        }

 if( $IPCount > $max_hits_per_ip_per_window ) 
{

header('HTTP/1.1 503 Service Temporarily Unavailable');
header('Status: 503 Service Temporarily Unavailable');
header('Retry-After: 3600');
header('X-Powered-By:');
echo '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Temporarily Unavailable</title>
</head><body>
<h1>Service Temporarily Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
</body></html>';
exit;

} else {
$fileName = $_GET['path'];
if (file_exists($fileName))
{
$extension = end(explode('.',$fileName));

switch (strtolower($extension)) {
   case 'png':
    header('Content-type: image/png');
    echo file_get_contents($fileName);
   break;
  
   case 'jpg':
   case 'jpeg':
   header('Content-type: image/jpeg');
    echo file_get_contents($fileName);  
   break;
  
   case 'bmp':
   header('Content-type: image/bmpg');
    echo file_get_contents($fileName);
   break;
  
   default:
   break;
   }
}

}
?>


In this script you can configure two parameters:
1. $time_window_seconds=35; //Perriod witch apliers rule
2. $max_hits_per_ip_per_window=2; //Number of visits

In generally this scripts checks that user cannot hit full size image, not script (mod_rewrite comes in help here ), twice in 35 seconds.
So normal users should not see any different unless they will very quickly view full size images. And of course you can replace 503 error with image Where is written you have reached bandwidth limit in some seconds etc. Or just show that he was temporary blocked.

MySQL table structure looks like

Code Select Expand


CREATE TABLE IF NOT EXISTS `ipleechcheck` (
`id` bigint(20) NOT NULL auto_increment,
`ip` varchar(255) NOT NULL,
`hittime` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;


Waiting for comments

P.s sorry for my crappy English

[Nibbler]

Thanks for your contribution. Few minor things:

$_GET['path'] looks like it needs to be sanitized.
You have a typo here : Content-type: image/bmpg
The rewrite code needs to be modified if you use non-default names for the albums or userpics directory or thumb_ or normal_ prefixes.
The mysql table could use some optimizations eg. field types and keys and make it a memory table. You don't need that auto_increment field at all.

Good work

[TigerClaw]

Good work!

Some user agents don't have ^ before the name like:

RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} \'Mozilla [NC,OR]

I think it's an error, isn't it?

[dke]

very nice, will be testing this out!! thanks.

[paquets]

This mod works very nicely. Is there any way that the error message or the same protection be working on the " normal_" file as well? Right now, I allow people to search my cpg installation, so they can see thumbnails but when they click on it, it brings them to a page that promps them to login or register.

Withthis corrent mod unregistered viewers cannot see the full size but they can see the normal size...

Think this would be possible to achieve?

[armus]

I bugged with it too and in some cases hotlink disable has not worked at cpanel but my hosting tech just disabled via cpanel admin and palced these rewrite rules in the .htaccess

Probably you are using cPanel, cna use the fallow simply, i would like to share

RewriteEngine on


RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://youralloweddomain.com/.*$      [NC]
RewriteCond %{HTTP_REFERER} !^http://www.youralloweddomain.com$      [NC]

RewriteRule .*\.(gif|jpg|jpeg|png)$ http://www.yourdomain.com/ [R,NC]