The script has auto blacklisted over 500 ip addresses in the past 30 or so hours that have tried accessing pages that end in .php. That seems like an awful lot of spammer bots. Let me know if you know of anyone else that may have gotten blacklisted on accident.

I also updated the robots.txt file which the search engine bots like google and yahoo should reread shortly. I put in some exclusions for some of the directories and configurations to stop search engine bots for some of the other lesser used sites from scanning the site. I increased the search bot scan delay between each page from 0 to 5 seconds which should help.

I never realized how much of that site traffic is actually made up from search engine bots crawling the entire site. I would say it's probably at least 50% of the connections.

Last edited Wed, 04 Dec 2013, 5:55am by scott