) Due to frustration from this previous thread: http://www.lunarforums.com/lunarpages_web_hosting_email_pc_security/autobanning_ips_that_scan_for_nonexistent_contentt43805.0.html;msg310478#msg310478 - I created this thread and the Auto IP Ban Script.

Many of us web site/domain account holders have been plagued by the dreaded nuisance of site rippers, site scanners, unruly search engine spiders and rogue search engine bots. This is clearly evident whenever you view your error logs and notice a really long list of errors pertaining to files or folders that do not reside on your web account. And it leaves you wondering, why are they trying to find that particular file or folder? This form of site sniffing is due to someone or some program automatically running a script that looks for certain files or folders. What they are looking for are security holes in your web account for files that can be exploited. Although there is the ever present danger of exploits and the constant need for security; many of these exploits go unfixed or unnoticed until it is much too late for the account owner. In some cases certain search engines and or robots will crawl your pages and more than often will not respect or obey the parameters set forth in the robots.txt file that you have created and or configured. Some of these spiders/bots will ignore the robots.txt and then traverse wherever they like about your web account folders. This in turn will list or collect everything in your web account regardless of what you have blocked or have not properly hidden. In response to this annoying practice of sniffing or scanning a web site; I attempted to seek assistance from local hosting providers and other resources. To no avail, not a single person or company had a sure fire (and or free) way to fully combat this problem. I was told to either just ignore the scanning or "manually edit" my .htaccess file each and every single time there was a violation. Confounded by the lack of assistance and the need for being cheap; I set forth to learn more PHP coding (yes, I took the time to read) and came up with a simple but highly effective script that dealt with the problem immediately. Within the first 24 hours of implementing my newly created script, I had effectively and permanently banned five IP addresses that were scanning my web sites. After the first 5 seconds of them scanning my site and hitting the Auto IP Ban Script - THEY WERE STOPPED COLD! So, here is the script. As you can see it is small, plain, loosely written and has much potential. Try it out. It shouldn't be too hard to get it to work on your web site. It only took me less than a day to write and perfect. Have fun and I hope this works for you rather than the - "Just deal with it" or "Ignore it" responses you will normally get. Download the ZIP file: http://www.fbi-team.org/_pub/auto_ip_ban.zip

Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) (Installation steps [Testing Phase] - Please read carefully) 1. Copy below code into a file called: mytrap.php (Use an HTML/PHP editor or Notepad. Be sure to remove the trailing spaces if you copy from this forum) Code:
<?php // * Auto IP Ban Script - v1.0 * // * Author: Inque187 * // * Date: (c) Oct 5, 2007 * // * Works with PHP 4.0+ & 5.0+ * // * DISCLAIMER: Author of this script is not and will not be held liable and or responsibile for any configuration error s, loss of data, interruption of * // * service, and or any other means of misuse by neglect pertaining to this script that affects any internet account. Us er of this script takes sole * // * responsibility for any resulting problems that arise due to usage of this script in part or in whole and shall hold indemnity against the author. * $ipad = $_SERVER['REMOTE_ADDR']; // Collects the user/visitor IP address. $ban = "Deny from $ipad\n"; // What will be written to the .htaccess file if IP needs to be banned. $file = "htaccess.txt"; // Change to > .htaccess <- once thoroughly tested. Should be placed in the root directory. $search = file_get_contents($file); // Prepare the .htaccess file by gathering entire contents. $check = strpos($search, $ipad); // Checks the .htaccess file if the current user IP address string does e xist. // This next part of the script checks to see if the IP is already banned or not. // If the IP does not already exist; it will write the ban line to the .htaccess file, display the message, and then emai l you a copy. // If the IP is already listed in the .htaccess file; the script ends with only a displayed message. if ($check === FALSE) { $open = @fopen($file, "a"); // Open the .htaccess file and get ready for writing only. $write = @fputs($open, $ban); // Write the banned IP line to the .htaccess file. (Example: Deny from 12.34.56.7 89) // Email a copy of ban and info to your admin account (or other email address). // Make sure you change the email address. @mail('BlOcK_IpS@YoUr_WeB_sItE.cOm','Banned IP '.$_SERVER['REMOTE_ADDR'].'',' Banned IP: '.$_SERVER['REMOTE_ADDR'].' Request URI: '.$_SERVER['REQUEST_URI'].' User Agent: '.$_SERVER['HTTP_USER_AGENT']); // IP address is not banned - so there is a need to write to .htaccess file.

// Display the error message to the user. (You may change to read what you want). echo '<html><head><title>IP Address '.$ipad.' - Blocked or Banned! </title></head><body bgcolor="#FF000000" text="#FFFFFF" oncontextmenu="return false;"><center><font face="Verdana, Arial" ><h1>THANK YOU - DON\'T COME AGAIN!</h1><b>IP Address '.$ipad.' Has Been Blocked or Banned!<br />Contact the web admin if this ban is by mistake.<p />Have a nice day!</b></font></center></body></html>'; // Close the .htaccess file - all done. @fclose($open); } else { // IP address is already banned - no need to rewrite to .htaccess file. // Display the error message to the user. (You may change to read what you want). echo '<html><head><title>IP Address '.$ipad.' - Blocked or Banned! </title></head><body bgcolor="#FF000000" text="#FFFFFF" oncontextmenu="return false;"><center><font face="Verdana, Arial" ><h1>THANK YOU - DON\'T COME AGAIN!</h1><b>IP Address '.$ipad.' Has Been Blocked or Banned!<br />Contact the web admin if this ban is by mistake.<p />Have a nice day!</b></font></center></body></html>'; } // End of File/Script; exit; ?>

2. Save mytrap.php file and then FTP upload to the root folder of your web account. (You can also use this same file in any subdirectories designated as 'Add-On Domains') 3. For testing purposes only - create an empty text file called htaccess.txt and then FTP upload it to the same root folder of your web account where the mytrap.php file was uploaded to. (This way while testing the Auto IP Ban script, you will not overwrite or destroy your existing original .htaccess file) Quote *** WARNING *** - the following step is for those who have some knowledge in editing the .htaccess file. It is highly recommended that you make a copy of your original .htaccess file and keep in a safe place in the event that an error occurs. - If you are well versed in manually editing the .htaccess file - proceed with caution. - If you are not well versed in manually editing the .htaccess file - STOP RIGHT NOW! - Please consult the latest documentation regarding the .htaccess file: http://httpd.apache.org/docs/2.0/howto/htaccess.html 4a. Open your original .htaccess file in a text editor and add the following lines of code: (It is recommended that you put these lines of code at the top of the .htaccess file) Code:

Options +SymLinksifOwnerMatch <FilesMatch 403.shtml> Order Allow,Deny Allow From All </FilesMatch>

4b. Then add these lines at the bottom of the .htaccess file - be sure to leave an extra empty line at the bottom too! Code:
RewriteEngine On RewriteRule ^cgi-bin /mytrap.php [NC] RewriteRule ^cgi-bin/(.*)$ /mytrap.php [NC]

(Be sure that you are not using this directory/folder "CGI-BIN"- Ensure that you have added the "Options +SymLinksifOwnerMatch" line of code to the .htaccess file or you will ban/block your own web site IP address and or will receive an "Error 500 Page" when accessing your site. If you are in fact using the "CGI-BIN" folder, edit this to reflect a folder not in your root folder. (Example: /home/account/public_html/fumuda) 5. Save .htaccess file and then FTP upload to the root folder of your web account. 6. Open a web browser and enter in the address bar: http://www.YoUr_WeB_sItE.cOm/cgi-bin (You can use a trailing slash too - Change "YoUr_WeB_sItE.cOm" to your own web domain) 7. You should be presented with a red background screen with white colored text stating that your IP address has been banned/blocked. 8. Go check your email that you changed in the mytrap.php script and you should have a message from your hosted server with your own IP address (that has been banned), the file/folder you tried to access and the user agent string of your browser. 9a. Download/Browse to: http://www.YoUr_WeB_sItE.cOm/htaccess.txt 10a. Open the htaccess.txt file in Notepad and you should see the line: Deny from 12.34.56.789 (with your IP address instead) 11a. If you see the red message screen, have received the email with the ban info, and verified the "deny" text was written to the htaccess.txt file EVERYTHING WORKED PERFECTLY!

How do I unban myself? [Testing Phase] Technically you haven't banned yourself from your own web site. At least not yet. Remember the htaccess.txt file? This was used only for testing purposes. Luckily it wasn't the real .htaccess file or you wouldn't be able to access your web site via a web browser right now. So, with the testing complete; there is no need to unban yourself.

Setting Auto IP Ban for [Live Banning] A. Follow the instructions in step #1 above but edit the $file value from htaccess.txt to .htaccess (Make sure the double quotes stay around the value and also no extension on the end) B. Skip step #3 altogether - Testing is no longer needed C. Continue the rest of the steps until you come to steps #9a, #10a, and #11a - skip those three steps and replace with the below steps #9b, #10b, 11b 9b. Now type your web URL in the browser address bar: http://www.YoUr_WeB_sItE.cOm 10b. You should be presented with an "Error 403 - Forbidden Access" page. 11b. Download your: .htaccess file from your root folder - open it in a text editor and look at the last line of the file - You should see the line: Deny from 12.34.56.789 (with your IP address instead) . D. If you see the red message screen, have received the email with the ban info, and verified the "deny" text was written to the .htaccess file, and most of all, CAN'T ACCESS YOUR WEB SITE - EVERYTHING WORKED PERFECTLY!

How do I unban myself? [Live Banning] So now you can't access your web site. The Auto IP Ban Script worked perfectly but now you want to access your web site again. The fix is simple. Download your: .htaccess file from your root folder - open it in a text editor and delete the entire line that contains your IP address from the file and then FTP upload the file back to your web account. Then refresh or reload your web page. You should now be able to access and see your web site again.

Adding Extras (Blocking access to other folders and files) With the working functionality of the Auto IP Ban Script, a web owner has the potential flexibility to practically block anything that does not exist in their web directory folders. As it may sound simple, this is the hard work that needs to be done to make the Auto IP Ban Script really effective. If you want to add more folders or files that these miscreants have been looking sniffing or scanning for; the first thing you need to do is locate what they have already be trying to find. This information can be found in your web account logs. Depending on the frequency of the violations, you may have a huge amount of information or a very little amount to sift through. The best way to find these violations is to use a log viewer. I recommend: http://www.kiwisyslog.com/kiwi-log-viewer-overview/ - You can sift through large filesize logs easy and filter to highlight specific things you need to find. Your best bet is to look for a cluster of lines by the same IP address that is attempting to access non-existent files or folders. They are usually grouped together in a span of about 1-15 minutes. Once you have found all the violations and recorded them on paper or a text file, you will need to add them to the .htaccess file. As with in Step 4b. you can add extra lines with the corresponding violations you have found. Just add a new line and change the folder name or file name to match your recordings. If you want to block everything inside a folder, use the character set /(.*)$ after each folder name. This will block everything inside that folder. It works also for different versions of the folder whether it be plural or numbered variations. Example Additions: Code:
RewriteEngine On RewriteRule ^cgi-bin /mytrap.php [NC] RewriteRule ^cgi-bin/(.*)$ /mytrap.php [NC] # Newly Added Folders RewriteRule ^administrator /mytrap.php [NC] RewriteRule ^horde /mytrap.php [NC] RewriteRule ^phpmyadmin /mytrap.php [NC] # Newly Added Files RewriteRule ^cgi-bin/formemail.cgi /mytrap.php [NC] RewriteRule ^hidden/login.php /mytrap.php [NC] RewriteRule ^README /mytrap.php [NC]

(You can add as many as you like! Just make sure you are not using the folder or files you add or everyone that visits them will be banned from your web site)

Bot and Spider Trapping (The Tangled Web You Weave) The Auto IP Ban Script has an extended feature to lure and trap search engine spiders and robots that ignore or disobey your configuration settings in the "robots.txt" file. If a search engine spider or robot so happens to ignore your "robots.txt" file; you can catch them in the act and effectively ban them permanently from your web site. The setup is really easy. 1. Install, edit, configure and verify that Auto IP Ban Script is working properly. 2. Create or edit your robots.txt file. 3. Add the following lines to your robots.txt file Code:
Disallow: mytrap.php Disallow: /mytrap.php

4. Save and FTP upload your robots.txt file to your root folder of your web account. 5. You can then choose to add a "blind" link at the top of your web page just below the <body> tag so that search engine spiders and robots will pick it up. If they ignore the robots.txt file - they will be banned. Code:
<a href="http://www.YoUr_WeB_sItE.cOm/mytrap.php" title=""><img src="pixel.gif" alt="" height="1" width="1" /></a>

Make sure the image file you use is put in the proper folder according to the SRC info. Also, it is best to use an image that has a transparent background so it will not be visible to any regular visitor. We don't want to ban our honest visitors, now do we? (Check with the more common respectful search engines [Google, Yahoo, etc] to see if your web site may be penalized for using "blind" links. Some search engines will mark/grade/rate your web site lower if you have these types of hidden links in your web code) 6. Sit back and watch the fun of spiders and robots being banned!

Maintenance Thereafter [Long Term Effects] You ask what are the drawbacks to using this script? Aside from unbanning regular nosy visitors, there is only one required maintenance feature. Depending on the frequency of sniffing and scanning attacks on your web site; you will have to periodically download and check your .htaccess file for excessive amounts of banned IP addresses. If you have a lot of scanning going on your web pages, your .htaccess can grow to a very large filesize in a few weeks. Moderate your .htaccess file and ascertain if the Auto IP Ban Script is right for your web site. If you are getting sniffed and scanned on a regular basis and it is becoming a serious issue, I highly recommend that you purchase some software that will handle the demand for better security.

Closing By no means is the Auto IP Ban Script supposed to be a fix-all to stopping web site scanning and disobedient search engines. Auto IP Ban Script is a primary step in an attempt to curb or discourage web site sniffing/scanning for non-existent files and or folders. Thus it will save precious bandwidth usage and processing time. Not to mention, it will reduce the size of your log files and give you some piece of mind in regards to web site security. If you have any questions about Auto IP Ban Script; please post here or email me at: inque187@yahoo.com Enjoy!

Re: Auto IP Ban Script (Stop Rogue Scanning and Spider/Bot Trapper) Reply #1 on: October 05, 2007, 06:50:37 PM Very neat - please keep us up to date on it's progress! Logged
New to Web Site Hosting? Check Out the Lunarpages Blog Hosting Guide!

Follow us @lunarpages on Twitter!

Important Threads: Read This Before Posting! | Lunarforums Rules! | Mitch's Link of the Day! Also, be sure to check out and subscribe to the Lunartics Blog and the Lunarpages Newsletter !

Need Web Hosting Help? Check out the Lunarpages Web Hosting Wiki. It has tons of tips, tutorials and resources!

Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #3 on: October 23, 2007, 11:39:34 AM

Had to register to thank the inventor! Have this installed today and it looks great! I have some questions about .htaccess and how to get a "hit" on a certain request that i know only a hacker would hit. I host some small sites with a couple of hundred visitors per day. On them i run a opensource php image gallery called 4images which seems to be popular trying to hack. People are going at it a couple of times per week according to the access log. I get around computers well in general and has managed to get this going on a VPS, but my php and .htaccess skills are limited so i might as well ask here... 1. I have "visitors" in the log looking like this /index.php?template=../../../etc/passwd%00 or they try upload a php script cloaked as a .jpg and then try to call it /index.php?template=../../data/tmp_media/hackerscript.jpg%00 (The jpg hack is patched a long time ago but its still anoying) Is it possible to trigger on the ../../ (or perhaps %00) part in .htaccess? 2. Would it be possible to add a feature of generally allowed admin IPs? For example, i know which IP-addresses that would access the /admin folder of the gallery software. With this feature, anyone but the allowed IPs will be blocked. That way if a previously unknown hack to the admin tools where found it is prevented with this. Again, super script!

/Thomas Gothenburg, Sweden Logged

While reading over this humongous nugget of information: http://www.lunarforums.com/lunarpages_web_hosting_email_pc_security/htaccess_everything_you_need_to_knowt43944.0.html I found a better way to setup scanners, sniffers, and bad bots. With a simple rework of the character string, one could lay just about any type of trap for any file, folder and or string that these miscreants are looking for. Again there are two things to consider: 1. Ensure that you have no files or folders using these character strings or you will ban innocent visitors and even yourself from your web site. If you do, just remove the banned IP and the offending lines from your .htaccess file and re-upload to your FTP account. 2. Compile a list of things that these scanners, sniffers, and bad bots are looking for by sifting through your log files. Now, for the good stuff. With simple reading and understanding of how things in the .htaccess world work; I came up with some new character strings to get hits. Before we were using just this set of strings in the .htaccess file: Code:
RewriteEngine On RewriteRule ^cgi-bin /mytrap.php [NC] RewriteRule ^cgi-bin/(.*)$ /mytrap.php [NC]

But now we can make it much simpler by checking for specific things in a requested string query. So we can use these sets of strings in the .htaccess file instead: Code:
RewriteEngine On # Checks for a specific file name RewriteRule ^prx.php$ /mytrap.php [NC,L] # Checks for character string queries inside a file name and or folder name RewriteRule ^(/*.*)?cgi(.*/*)?$ /mytrap.php [NC,L]

Inque187 Anti-Cheat & Beta Team Spacescooter Operator Offline Posts: 39

Sic Vis Pacem, Para Bellum!

Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #4 on: October 24, 2007, 05:15:24 PM Inque187 Anti-Cheat & Beta Team Spacescooter Operator Offline

nvm Last Edit: October 24, 2007, 05:47:40 PM by Inque187 Logged

Posts: 39

Sic Vis Pacem, Para Bellum!

Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #5 on: October 24, 2007, 05:26:58 PM

thomasj Newbie Offline Posts: 4 Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #6 on: October 25, 2007, 01:17:27 AM

Thanks for your reply! Unfortunately it does not work in my case. I might have a clue on why... As you see in my previous post the log shows this entry "http://www.mysite.com/index.php? template=../../../../../etc/passwd%00" i would like to trigger on the ../../ or passwd. BUT those are not a part of the catalog structure or filename in Apaches point of view but a parameter to php since its after the ? right? The other question, about authorized IP:s is that doable? (perhaps some sort of IF in the mytrap.php? "IF ip equals 123.12.12.12 then OK" /Thomas Logged

Inque187 Anti-Cheat & Beta Team Spacescooter Operator Offline Posts: 39

I'll look into that suggestion but wouldn't that defeat the main purpose of keeping certain unauthorized sniffers/snoopers away? If the IP address does not generate a block, they should be good to go without having to add an extra "IF" statement. Logged

Sic Vis Pacem, Para Bellum!

Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #7 on: November 20, 2007, 01:38:07 PM thomasj Newbie Offline Posts: 4 Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #8 on: November 20, 2007, 02:26:06 PM

Quote from: Inque187 on November 20, 2007, 01:38:07 PM I'll look into that suggestion but wouldn't that defeat the main purpose of keeping certain unauthorized sniffers/snoopers away? If the IP address does not generate a block, they should be good to go without having to add an extra "IF" statement. Well perhaps. Its another way of using this script. I know the specific IPs (home computer and my computer at work) that will visit the www.site.com/admin/ folder. I would add the /admin/ folder as "forbidden RewriteRule /admin /mytrap.php". Since i know which IPs will visit /admin, all others that try to run anything in there is banned. For example, if there is a newly discovered bug in the script, the first thing a hacker would do is to gain admin access and head for the admin tools. With this he is busted before he can say php. Could you confirm my other suspicion about "../../../../etc/passwd%00" not being a part of the url from a Apache point of view an therefor not working? Thanks anyway! /Thomas Logged

quattrone Newbie Offline Posts: 1 Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #9 on: April 02, 2011, 07:06:03 PM rjs Newbie Offline Posts: 1 Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #10 on: November 13, 2011, 01:53:25 PM

I too felt compelled to register just to say thanx to Inque187. I have been plagued by snifbots for quite awhile now, banning IPs by hand every once in awhile when the outrage welled inside me, but that was for sure a losing battle. It just now occurred to me that perhaps there was a way to automatically do that when certain non existent folders or pages were pinged. And there arent a lot if any solutions out there -- just this one, and it works great. Thanx so much!! Logged

Well, I too felt I should register to say its a great idea.!! Many thanks to Inque187 for this concept. rj Logged

dgswilson Newbie Offline Posts: 1 Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #11 on: December 02, 2011, 05:55:27 AM tazilon Newbie Offline Posts: 1 Re: Auto IP Ban Script (Stop Rogue Scanning and Trap Bad Spiders/Bots) Reply #12 on: October 21, 2012, 07:10:05 AM

very nice, thanks I have a perl logger that records IP, UA etc. So how do I tell script to not write ban? I'm pretty sure I know where to do this but I thought I'd just ask. Also I guess I'll ask if there is a way to output to a different location than .htaccess - Like (log.txt)? Like $file = "log"; and then replace ban with (?) write something Logged

I realize that this thread is 5 years old, but I just found it, implemented the script, and it works VERY WELL! Thank you VERY MUCH! I've been lokking for something like this for a long time... thanks again!