Stopping heavy junk traffic (or ddos)

fireix

Since Friday, I have had thousands on thousands of IP-addresses connecting to the frontside of a customers wordpress website and I struggle hard to stop it. I also recieve ICMP-pings from same IPs.

The amount of traffic is no problem for pfSense or our network, but the dedicated server we now upgraded it to struggles a bit under the load once wordpress-page is up (replacing it with static php-file/html-file is no problem, the load is no problem - but the attacking traffic is still hitting it).

This is a pretty low volume website (about 5-10 visitors there at any given time), but starting Friday, it just blew up with 1000 visits a second or so. It is just a simple informative wordpress-page for a non-controversial registered stock company and it is developed/maintained by professional developer (not us).

All IPs in the logs (both seen by pfsense and by cPanel) belong to ZSCALER, INC in the US or ZSCALER in the Netherlands and 99% of all traffic is from them. I have reached out to their abuse and their noc-contact stated in the whois, but not a single respons. I even used their contact page.

Once I manage to block one range, they open a completely new range. Is there any way I can block all IPs belonging to this company? For instance, I can block 165.225.192.0/18 in Configserver, cPanel and pfSense, but they just fire up a new block in next second.

NetRange: 165.225.192.0 - 165.225.255.255
CIDR: 165.225.192.0/18
NetName: ZSCAL

Gertjan

@fireix

You have a firewall, right ? ;)
On the WAN interface of your pfSense, block that network.

And true, the same stupid https requests come back, using other source IP addresses. This goes on and on forever. When your site becomes 'known' on the Internet, even more of these requests pop up.
Your site is WordPress based and that is easily detachable. Robot script file will 'try' forever.

This why 'fail2ban' has been invented.
fail2ban uses filters that analyze the web server's log files, and if a filter matches, the IP is blocked for a while - you can chose the block time.
The firewall should be on the same device as the web server.
fail2ban is a Debian/FreeBSD/etc package (not Windows).
Here you can see how many IP's are blocked on my server.
I also filter fake mail server traffic, etc. It works completely automatically.

This is also the moment you start to understand that a public accessible server doesn't belong on a 'home' setup. You need at least a small inexpensive VPS, so up-link and down-link speed isn't an issue anymore.

johnpoz

@fireix ZScaler is huge cloud provider and proxy provider.

Blocking them could impact normal customers.

You could go about it like @Gertjan says and play wack-a-mole..

Or vs just blocking the specific netblock they are using, block the whole ASN..

So for example the range you gave, via whois

NetRange:       165.225.192.0 - 165.225.192.255
OriginAS:       AS62044

So you could block that whole AS, which would be more of their netblocks..

https://hackertarget.com/as-ip-lookup/

Pfblocker can be very useful in loading lists from ASNs and then blocking all of those networks.

Now such a huge company like zscaler is going to have multiple ASNs - most likely based upon geo location, etc. I believe that one is the EMEA one.. So you may need to have multiple ASNs blocked. But again they are huge proxy provider, and legit customers could be flowing through them, they are big in web filtering, etc. So blocking them could block legit users.

edit:

it just blew up with 1000 visits a second or so. It is just a simple informative wordpress-page for a non-controversial registered stock company

You sure you just didn't get listed on something like slashdot or hackernew, reddit there are many websites that if your site gets mentioned could create huge amount of traffic.. Which is legit traffic, but your site just not capable of that level of traffic..

https://en.wikipedia.org/wiki/Slashdot_effect

fireix

@johnpoz I have looked at he refereers in the visitor log (always blank in the refer-field) and they all have the same browser-version/string in the requests. Also, they never visit any other pages than the frontpage (/). I used the visitor-log tools in cPanel to watch this.

I have blocked ranges up to /16 and /17, but thanks for the tip of the ASN blocking - that can come in handy :)

As you see under, this is just the 443-traffic that pfSense now is blocking good, in that second it was 142, but it is often around 300 and up to 1000. From the same IPs, it is even more ICMP entries in the protocol field.

@Gertjan Thanks :) I'm using Configserver that have this functionality, but the way they hit the frontpage with different IPs, it doesn't seem to be very effective. I have played with the ddos-parameters, but the only thing that have worked so far is to exclude their network ranges as they appear.

fireix

@fireix My problem is not upling/downlink speed, we have tons of that as we are located in two of the biggest data centers in my country with redundancy/connections everywhere and 10 Gbps network speed. This is a dedicated server, I wouldn't imagine this running on a small VPS.. So no home-seup here :)

johnpoz

@fireix your other option to just blocking IP, is if you can find a common aspect of the bad traffic that you can trigger on and filter it that way using IPS..

You could also look to limiting max connections from host or connection rates in your rule. To keep any IPs that are sending many connections at a fast rate limited. Which should still allow legit traffic to get through.

edit:
Where your hosting this at? Its possible they offer some sort dos/ddos protection - even if at a cost, and optional to turn on.

example:
https://www.ovh.com/world/anti-ddos/
https://www.leaseweb.com/cyber-security/ddos-ip-protection

Any actual DC should offer such services.. Vs attempting to do it on your own firewall..

They have to have something, or attack to 1 customer could take down all of their customers, etc. Once it becomes volumetric to the point it loads up their overall connections.

Our DC we use Netscout, called Arbor before.. We have multiple 10ge connections into the DC.. But these days its not hard to take out a 10ge pipe.. Its not unheard of to see TBps type attacks.. So taking down a 10ge pipe is a mere raindrop in the hurricane ;)

fireix

@johnpoz When I visit any of the IPs in the browser, I actually get redirected to this portal login: https://gateway.zscaler.net/auD?origurl=http%3A%2F%2F165%2e225%2e34%2e59%2f&_ordtok=qSZ3WV37BqjPsMTPk6W7v4L5DP

It seems like the dos-attack is to small to be noticed, but it is big enough that a heavy wordpress site would go down. So not sure if they would manage to stop it. If it was really an attack, they should be able to do it better than this...

Wonder if it could be some kind of reflection attack or similar...

A Former User

@fireix, Anyway, how did they stop the attack?

fireix

@silence They? :) I added one and one network range each time a new IP appeared and this managed to stop the attack already at pfSense. Since the traffic then didn't go to the heavy wordpress site, that solved the issue.