To our sites each day come tens of thousands of visitors, many of them identifiable but also come various non-identifiable crawlers, spam bots and robots. I.e. we have tens of thousands of ip addresses every day which can and will be analysed for block spam bots and other threats.
Our unique protection system allow easy identification of the real visitors ip with any bots/crawlers ip addresses and other threats include unknown crawlers/bots masking themselves as normal users.
The operating principle of our protection system is very simple: We allow any user to browse 10 pages on any of our websites, and after system start to display free registration form on the website and to block all pages on website (i.e. visitor will see only free registration form on all pages). We ask visitors to freely register on site themselves or to revisit us after to 24 hours to browse other pages.
Crawlers/Bots ignore such instructions and they would try to continue crawling the website. Normal users can click links on the website also but he/she will see all the time registered page and after they have seen 10-20 pages they will stop. But Bots would continue crawler on the website even despite those displays. We count the use of the site. And after 100 requests to see different site pages with registration form and every time is displayed a registration form we can be 100% sure the visitor is a BOT, even if it is masking itself as a normal user.
If user agent line has no any identification of that bot and is masking itself as a normal user we will automatically add its IP to our blacklist. For example Google Bot have user agent:
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html and not hide it.
Therefore our Blacklist of IPs addresses consists only of unknown crawlers/spam bots masking themselves as normal users and our Blacklist has no search crawlers like Google Bot, Yahoo Bot, Bing Bot and others which nobody has tried to mask as a normal real user.
We have not found on the internet a service of similar kind which creates a IP Addresses Blacklist of unknown dangerous crawlers / spam bots in real time so we hope you will found ours useful.
All unknown crawlers give more load on site server/ possible in trying to extract email addresses etc. from your website for creating spam or finding backdoors on your website for hack which have no benefits.
-----
You can add results from our blacklist/web bots databases to your own site free of charge. That will be usefull for your users. You need to place a link on myip.ms website on a page where you agree you will publish your ip check results.
You need to send requests to page http://blacklist.myip.ms/ ..ip.. (blacklist) / http://bot.myip.ms/ ..ip.. (web bots) and in the text source result page you will find a hidden field: blacklist_test_results with value of result which you can afterwards use on your website.
------
We invite you to download our up-date fresh Blacklist Ip addresses in your website firewall so that you can block those ips from your website.
Live Blacklist IP Addresses (25,000+ ip, October 2024)
