Randem Systems Support Board

Thanks, I made the correction

On second thought on the Iran IP addresses you should only put 94.101.140. and 94.101.141. in the IP blacklist. 94.101. may be too wide a range to block.

Thank you

Sometime you need more than one blacklist item for the bots can spoof just about anything. You need to add semrush to the Spider blacklist. The names are not case sensitive. For the Iran blocking of IP's use 94.101. in the IP blacklist. As far as China, that is a large area to block. You would need several IP ranges to block that (113. , 114. 115. 116.) or you may find a domain to block which would be easier (EX. something.ch, where .ch would be the ext from china).

More questions:

We are seeing additional bots and unwanted foreign searches:
crawl11.bl.semrush.com  (semrush.com was in the Domain Blacklist, but not blocked)  are the names case sensitive?

China Telecom Anhui - 113.203.192.0 - 116.201.255.255 (is there an easy to block this IP range?)

Iran - 94.101.140.0 - 94.101.141.255 (Same question as above)

I made the changes.

Have you taken GoggleBot out of the Whitelists? The Whitelists override the Blacklists.
To Block Google:

Remove GoogleBot from ANY Whitelist.
Place GoggleBot, Google in the Spider BlackList.
Place GoogleBot.com, google.com in the Domain Blacklist
Place 66.249. in the IP Blacklist.

You may not need all of the above to accomplish the task.

Good Morning,
Spiders listed in the "Whos on line"
crawl-66-249-66-159.googlebot.com
crawl-66-249-66-128.googlebot.com

I have:
Blacklist Domain: googlebot.com
Blacklist Spider: googlebot

Thank you

You only need one entry googlebot.com for all those entries... No need for the rest.

Hi,
I added googlebot to the spider list.
The Domain list shows:
crawl-66-249-73-82.googlebot.com   1   
136   crawl-66-249-73-84.googlebot.com   1   
119   crawl-66-249-66-128.googlebot.com   1   
120   crawl-66-249-66-130.googlebot.com   1   
121   crawl-66-249-66-159.googlebot.com   1   
122   crawl-66-249-66-214.googlebot.com   1   
144   crawl-66-249-66-87.googlebot.com   1   
117   crawl-66-249-69-210.googlebot.com   1   
118   crawl-66-249-69-212.googlebot.com   1   
124   crawl-66-249-69-214.googlebot.com   1   
128   crawl-66-249-69-50.googlebot.com   1   
134   crawl-66-249-73-86.googlebot.com   1   
143   crawl-66-249-75-116.googlebot.com   1   
123   crawl-66-249-79-20.googlebot.com   1   
78   googlebot.com

sonnyh,

Add gogglebot.com to the Domain Blacklist and googlebot to the Spider Blacklist.

Thank you.
I misunderstood, thought that was the Bot name.

Hi sonnyh,

There are 3 blacklist, Domain, IP and Spiders. crawl-66-249-69-212.googlebot.com would need to be added to the Domain blacklist for it to be blocked. Adding it to any other blacklist would not get the desired effect.

Hi,
I have previously added crawl-66-249-69-212.googlebot.com to the spider blacklist.
I noticed that is was in the "Whos Onlibne" list this morning.

Is there anything else that needs to be done after something is added to the blacklist?