Blocking petalbot

lewis

Has anyone successfully blocked petalbot at the firewall level?

They are aggressively indexing many sites behind the firewall. It would be very efficient to block them at the firewall level.

johnpoz

@lewis said in Blocking petalbot:

petalbot

Once you validate an IP they are coming from, for example I found this IP

;; QUESTION SECTION:
;251.160.119.114.in-addr.arpa.  IN      PTR

;; ANSWER SECTION:
251.160.119.114.in-addr.arpa. 86400 IN  PTR     petalbot-114-119-160-251.aspiegel.com.

You could then block the whole range that IP falls into

user@i9-win:~$ whois 114.119.160.251
% [whois.apnic.net]
% Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html

% Information related to '114.119.128.0 - 114.119.191.255'

% Abuse contact for '114.119.128.0 - 114.119.191.255' is 'hws_security@huawei.com'

inetnum:        114.119.128.0 - 114.119.191.255

Or you could even block the whole ASN or the bigger IP block.

% Information related to '114.119.128.0/18AS136907'

route:          114.119.128.0/18
origin:         AS136907

I doubt this would cause you problem - but keep in mind blocking that whole range or asn could prevent legit traffic - if there is any that would come from that IP range, which I doubt ;)

lewis

@johnpoz Hi, thank you for your help.

I tried this before but petalbot is still getting in.
I put the rule at the top of the Rules, below private and bogon rules I have in there.

I figured I've got something missing which is why I wondered how others might be doing it as I've never used blocking before.

johnpoz

@lewis well your block there isn't getting any hits.. see the 0/0 B means that rule never evaluated... So your blocklist isn't matching, or you have a rule maybe in floating allowing that.

That IP was just one I found when searching for petablock - not something I saw on my firewall.. They could be using a different range - you would want to look on say on your webservers log for the IPs hitting it that are the petalbot - or look in your state table for stuff you want to block.. And then make sure you kill any existing states as well for those IPs.

lewis

Yes, I looked at the logs and they seem to be using the 114.119.0.0 network.

I accidentally used your entry instead of a /16 but it's not blocking and I can now see traffic on the rule.

johnpoz

@lewis said in Blocking petalbot:

but it's not blocking and I can now see traffic on the rule.

Not possible - but what is possible is say ip .x has a state, and ip .y in that range does not - .x would be still be allowed via the state, while .y would be blocked and show has hit to the rule.

Make sure you kill all states that have IPs in that network block.. Or for good measure kill them all ;)

States are evaluated before rules, so anything that currently has a state would continue to be allowed.

lewis

I reset the states and still see 114.119.x.x hitting sites.
I looked online before posting and docs I found seem to be how I've got this set up. Not sure what I'm missing.

That said, looking at states and filtering for 114.119, there aren't any states.

johnpoz

@lewis well then the traffic is not coming through pfsense maybe.. Here is the thing if your block rule trigger, pfsense would not let it through and also block it - how would that be possible.

there aren't any states.

Then it didn't go through pfsense - if pfsense allowed traffic it would have to create a state..

lewis

LOL, as usual, it was something simple. You kept saying 'nope' but I kept seeing the bot.

I realized just now that I set that up on another firewall and wasn't watching the servers on the same network.

So far, no more petalbots so I'll add this to the other firewalls too.

Thank you very much for helping me.

johnpoz

@lewis no problem - glad you got it sorted..