State Table DoS - SYN_SENT connections all over the place!?! Help me!!!

zimmy6996 · Oct 15, 2019, 3:11 PM

Good morning community ... I'm running in to a problem the last 30 days or so, and need to help to solve it. For about the past 3 weeks, I've seen this trend where i'm getting whacked on the state table with stuck connections. Basically, it appears that its some sort of NMAP scanning, or some now script kiddie tool that is making waves on the internet. For some reason, it has really started to cause headaches for me.

First, off, let me show you my state table counts ...

As you can see, sometime around the 26th of september, this started to rear it's head. And the past week has got out of hand.

Yes the 700K state spike yesterday, I definitely was able to see it's was 3 hosts that were just hammering away at me. What is happening is it leaves all these sessions in a SYN_SENT state. The sessions in the state table all relate to ports where we have valid services open. We blocked the traffic upstream, and it stopped.

Is there something I can be doing on the firewall side to handle this? Im guessing im not the only one to see this situation happening. I run a 8GB RAM pfsense firewall so the max table is about 800,000, but when this thing pushes in to 7K range, it starts going sideways ...

KOM · Oct 15, 2019, 3:13 PM

Snort or Suricata can protect against SYN floods and other similar DoS attacks.

zimmy6996 · Oct 15, 2019, 3:20 PM

Example of the types of connections that come in. There will be loads of them, coming from a handful of IP's. It's clearly a port scanning issue. Its just there is something new about the latest scripts running, because it's causing the connections to hang.

johnpoz · Oct 15, 2019, 4:27 PM

There is a currently been a flood of syn traffic, I am seeing large spike in such traffic - many coming from HK, but have seen others from Malta, etc.

Yeah open services would always be open to such attacks. As mentioned IPS could help in preventing this from being forwarded in to your end device.

Or can just block it with firewall, I blocked all access to 443 from anything other then US currently

edit: BTW... I just looked and yesterday in that 24 hours period had 54k hits to 443 that were denied because IP was not US, and another 44k to ports 8080 from all sources.

zimmy6996 · Oct 29, 2019, 11:58 PM

I'm going CRAZY here! I don't understand why this has never been a problem until about 30 days ago ... Here is a perfect example of what is happening ....

WAN tcp 47.74.56.139:57289 -> 74.112.73.42:443 SYN_SENT:ESTABLISHED 1 / 4 40 B / 176 B
LAN2 tcp 47.74.56.139:57289 -> 74.112.73.42:443 ESTABLISHED:SYN_SENT 1 / 4 40 B / 176 B
WAN tcp 47.74.56.139:55885 -> 74.112.73.105:587 TIME_WAIT:TIME_WAIT 1 / 4 40 B / 172 B
LAN2 tcp 47.74.56.139:55885 -> 74.112.73.105:587 TIME_WAIT:TIME_WAIT 1 / 4 40 B / 172 B
WAN tcp 47.74.56.139:58651 -> 64.4.202.128:25 SYN_SENT:ESTABLISHED 1 / 5 40 B / 220 B
LAN tcp 47.74.56.139:58651 -> 64.4.202.128:25 ESTABLISHED:SYN_SENT 1 / 5 40 B / 220 B
WAN tcp 47.74.56.139:45114 -> 64.4.202.81:25 SYN_SENT:ESTABLISHED 1 / 5 40 B / 220 B
LAN tcp 47.74.56.139:45114 -> 64.4.202.81:25 ESTABLISHED:SYN_SENT 1 / 5 40 B / 220 B
WAN tcp 47.74.56.139:40592 -> 74.112.73.81:80 SYN_SENT:ESTABLISHED 1 / 6 40 B / 264 B
LAN2 tcp 47.74.56.139:40592 -> 74.112.73.81:80 ESTABLISHED:SYN_SENT 1 / 6 40 B / 264 B
WAN tcp 47.74.56.139:41011 -> 64.4.202.62:587 TIME_WAIT:TIME_WAIT 1 / 4 40 B / 172 B
LAN tcp 47.74.56.139:41011 -> 64.4.202.62:587 TIME_WAIT:TIME_WAIT 1 / 4 40 B / 172 B
WAN tcp 47.74.56.139:63295 -> 74.112.73.48:22 SYN_SENT:ESTABLISHED 1 / 5 40 B / 220 B
LAN2 tcp 47.74.56.139:63295 -> 74.112.73.48:22 ESTABLISHED:SYN_SENT 1 / 5 40 B / 220 B
WAN tcp 47.74.56.139:36741 -> 64.4.202.94:80 CLOSED:SYN_SENT 1 / 0 40 B / 0 B
LAN tcp 47.74.56.139:36741 -> 64.4.202.94:80 SYN_SENT:CLOSED 1 / 0 40 B / 0 B
WAN tcp 47.74.56.139:33678 -> 74.112.73.14:80 TIME_WAIT:TIME_WAIT 1 / 4 40 B / 172 B
WAN tcp 47.74.56.139:58930 -> 74.112.73.41:443 SYN_SENT:ESTABLISHED 1 / 3 40 B / 132 B
LAN2 tcp 47.74.56.139:33678 -> 74.112.73.14:80 TIME_WAIT:TIME_WAIT 1 / 4 40 B / 172 B
LAN2 tcp 47.74.56.139:58930 -> 74.112.73.41:443 ESTABLISHED:SYN_SENT 1 / 3 40 B / 132 B
WAN tcp 47.74.56.139:43105 -> 74.112.73.205:443 SYN_SENT:ESTABLISHED 1 / 6 40 B / 264 B
LAN2 tcp 47.74.56.139:43105 -> 74.112.73.205:443 ESTABLISHED:SYN_SENT 1 / 6 40 B / 264 B
WAN tcp 47.74.56.139:52604 -> 74.112.73.201:587 SYN_SENT:ESTABLISHED 1 / 6 40 B / 264 B
WAN tcp 47.74.56.139:58313 -> 74.112.73.205:443 SYN_SENT:ESTABLISHED 1 / 6 40 B / 264 B
LAN2 tcp 47.74.56.139:52604 -> 74.112.73.201:587 ESTABLISHED:SYN_SENT 1 / 6 40 B / 264 B
LAN2 tcp 47.74.56.139:58313 -> 74.112.73.205:443 ESTABLISHED:SYN_SENT 1 / 6 40 B / 264 B
WAN tcp 47.74.56.139:41633 -> 74.112.73.50:80 SYN_SENT:ESTABLISHED 1 / 6 40 B / 264 B
LAN2 tcp 47.74.56.139:41633 -> 74.112.73.50:80 ESTABLISHED:SYN_SENT 1 / 6 40 B / 264 B
WAN tcp 47.74.56.139:63062 -> 64.4.202.81:25 SYN_SENT:ESTABLISHED 1 / 3 40 B / 132 B
LAN tcp 47.74.56.139:63062 -> 64.4.202.81:25 ESTABLISHED:SYN_SENT 1 / 3 40 B / 132 B
WAN tcp 47.74.56.139:33238 -> 74.112.73.201:80 SYN_SENT:ESTABLISHED 1 / 6 40 B / 264 B
LAN2 tcp 47.74.56.139:33238 -> 74.112.73.201:80 ESTABLISHED:SYN_SENT 1 / 6 40 B / 264 B
WAN tcp 47.74.56.139:44409 -> 64.4.202.94:80 SYN_SENT:ESTABLISHED 1 / 6 40 B / 264 B
LAN tcp 47.74.56.139:44409 -> 64.4.202.94:80 ESTABLISHED:SYN_SENT 1 / 6 40 B / 264 B

Now this is just an example of 1 IP that is doing it to me. Sometimes it is a single IP that does it, sometimes it will be a random blast from a /24 network. Either way, it eats up 50K-500K session states. At times, it can really blast the system.

Nothing has changed other than i think an upgrade from 2.4.4-p1 to 2.4.4-p3. I don't know if the pfSense version has something to do with it, or if it's something that has happened on the internet. Just crazy that all these sessions get stuck like this.

Can anyone help?!?!??!

johnpoz · Oct 30, 2019, 12:19 AM

Help with what the pesky flies? Block them if they are doing a non flooding syn/state attack..

We have all been seeing an uptick in such traffic - see my pesky fly thread ;)

Where you run into problems is if volumetric type attack - nothing you can do about that but get with your upstream provider.

It has nothing to do with pfsense.. I show that 47.74 56 guy is from japan.. Do you need to allow connections from JP? If not block the whole freaking country ;)