Explore
- About Us
- Sitemap
Block Spy Bots
Protect Authority Blog Network (ABN) From Rev Engineering
Block All The above spy Bots. Spyder Spanker is a good Option. But Excess use of Spyder Spanker can be a foot print.
- Moz (rogerbot)
- Majestic SEO (MJ12bot)
- SEO Spyglass
- Ahrefs - AhrefsBot
- Raven Tools
- Open Site Explorer (Moz)
- Majestic SEO
- GgigaBlast (gigablast.com)
.HTACCESS CODE
Example
RewriteEngine On
RewriteBase /
SetEnvIfNoCase User-Agent .*rogerbot.* bad_bot
SetEnvIfNoCase User-Agent .*exabot.* bad_bot
SetEnvIfNoCase User-Agent .*mj12bot.* bad_bot
SetEnvIfNoCase User-Agent .*dotbot.* bad_bot
SetEnvIfNoCase User-Agent .*gigabot.* bad_bot
SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_bot
SetEnvIfNoCase User-Agent .*sitebot.* bad_bot
<Limit GET POST HEAD>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>
ROBOTS.TXT
Example
robots.txt file code ( in robots.txt which should block everything except Google, Yahoo and Bing.)
Like:
User-agent: Googlebot
Disallow:
User-agent: msnbot
Disallow:
User-agent: Slurp
Disallow:
User-agent: Teoma
Disallow:
User-agent: rogerbot
Disallow: /
User-agent: exabot
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: gigabot
Disallow: /
User-agent: AhrefsBot
Disallow: /
User-agent: *
Disallow: /