Htaccess – Block Evil Robots, Site Rippers, and Offline Browsers

 

Eliminate some of the unwanted scum from your userspace by injecting this handy block of code.

After such, any listed agents will be denied access and receive an error message instead. Please advise that there are much more comprehensive lists available this example has been truncated for business purposes. Note: DO NOT include the “[OR]” on the very last RewriteCond or your server will crash, delivering “500 Errors” to all page requests.


# deny access to evil robots site rippers offline browsers and other nasty scum
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} ^Anarchie [OR]
RewriteCond %{HTTP_USER_AGENT} ^ASPSeek [OR]
RewriteCond %{HTTP_USER_AGENT} ^attach [OR]
RewriteCond %{HTTP_USER_AGENT} ^autoemailspider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xenu [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus.*Webster [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.* - [F,L]

Or, instead of delivering a friendly error message (i.e., the last line), send these bad boys to the hellish website of your choice by replacing the RewriteRule in the last line with one of the following two examples:


# send em to a hellish website of your choice
RewriteRule ^.*$ http://www.hellsite.com [R,L]

Or, to send em to a virtual blackhole of fake email addresses:

# send em to a virtual blackhole of fake email addresses
RewriteRule ^.*$ http://s2.spampoison.com [R,L]

You may also include specific referrers to your blacklist by using HTTP_REFERER. Here, we use the infamously scummy domain, “asae.org” as our blocked example, and we use “yourdomain” as your domain (the domain to which you are blocking asae.org):

RewriteCond %{HTTP_REFERER} ^http://www.asae.org$
RewriteRule !^http://[^/.]\.yourdomain\.com.* - [F,L]

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

*