DDoS pfSense dies on XSYN and OVH scripts.
-
I am, by the way, running stateless as a test since it doesnt have any impact running with SYN cookies enabled :)
-
After re-reading the code this morning and looking at the packet dump, it seems "tot_len" is being set and used as the IP+TCP header size.
iph->tot_len = sizeof(struct iphdr) + sizeof(struct tcphdr);
No fragmented data wielding SYN packets here.
-
I've stopped a ddos on my network setting advanced rules options to limit connections per second and total states per ip.
Then I've reduced rule blocked time.My link dropped from 100% to 30% during attack.
-
In this case, total states per IP won't help because they're spoofed.
I don't see what's so special about these packets. They're all the standard 60 length IP+TCP, and 40 length IP. Both IP and TCP checksums are valid, IP has no flags, TCP's only flag is SYN, random source ports, random Seq numbers. I must be missing something, but it looks like a normal SYN flood, but it isn't.
-
Send me a PM with your ext. IP and pfsense behind looking at port80.
Then I will flood it for 180 secs and then you will see how it responds.
It just goes offline instantly…
-
I don't know, sounds like a sure-fire way to get packet-loss on my quality graph. Almost up to a week of 0 packets lost and a 1.2ms avg ping. Maybe in a few days, once I take a picture :-)
I don't mean 0.0% packet-loss, I mean 0 packets in actual numbers. I already have 0.0%, but there's some red on that quality graph.
-
Chris would get back to us with testing IP but not heard anything yet.
-
Look at the ping to the LAN side of pfSense…
http://youtu.be/HoGQ_2sg0J0
LAN goes offline and tries to keep going. On the test server I see maybe 25mbit of traffic and nothing that renderes it useless at all.
Pf just dies completely. With this I can take down any site running pfSense if I want to.
If you run Windows FW on the server with no pfSense infront, no issues.
-
FYI, your public IP is shown on your interfaces widget.
Another FYI, I get a nice stable ping to your gateway. Nice upstream :-)
Ping statistics for 80.x.x.x:
Packets: Sent = 149, Received = 149, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 138ms, Maximum = 141ms, Average = 139ms4 15 ms 15 ms 16 ms tengigabitethernet4-4.ar7.chi1.gblx.net [67.17.213.117]
5 108 ms 107 ms 108 ms 4.68.110.157
6 103 ms 103 ms 103 ms ae-116-3502.edge3.London1.Level3.net [4.69.166.134]
7 103 ms 103 ms 103 ms ae-116-3502.edge3.London1.Level3.net [4.69.166.134]
8 138 ms 138 ms 138 ms tdcdenmark-level3-xe.london1.Level3.net [4.68.63.90]
9 139 ms 140 ms 140 ms ae1-0.taanqe10.dk.ip.tdc.net [83.88.22.247]
10 140 ms 140 ms 141 ms cpe.ae11-388.taanqe10.dk.customer.tdc.net [62.243.131.198]
11 139 ms 139 ms 140 ms 80.x.x.x -
:D Its my home network so I dont care about that.
Just removed Squid to see if it handles traffic better…
Edit: It didnt.... :(
-
And the forum goes down at once as well.
Its the engine of PfSense thats the issue here. There is core functionality hit here and nothing done in the gui or elsewhere can prevent it.
-
So what's up here? Anyone tested this on FreeBSD?
-
Spoofed packet attacks may be used to overload the kernel route cache. A
spoofed packet attack uses random source IPs to cause the kernel to generate
a temporary cached route in the route table, Route cache is an extraneous
caching layer mapping interfaces to routes to IPs and saves a lookup to the
Forward Information Base (FIB); a routing table within the network stack. The
IPv4 routing cache was intended to eliminate a FIB lookup and increase
performance. While a good idea in principle, unfortunately it provided a very
small performance boost in less than 10% of connections and opens up the
possibility of a DoS vector. Setting rtexpire and rtminexpire to two(2)
seconds should be sufficient to protect the route table from attack.
http://www.es.freebsd.org/doc/handbook/securing-freebsd.html
net.inet.ip.rtexpire=2 # (default 3600)
net.inet.ip.rtminexpire=2 # (default 10 )
#net.inet.ip.rtmaxcache=128 # (default 128 )Anybody has any comments on this because it seems to be deep within the routing stack that this occurs.
-
Just out of curiosity, why would you want to store individual IP addresses in a routing table? Isn't that the whole point subnet masks and routing tables?
-
I dont know… its nowhere to be found in pfSense so I added it manually to get rid of it...
-
So, is there progress being made in coming up with a set of safe defaults that mitigate this attack in 2.2.1?
-
So the change helped?
It sounds like the best thing might be to completely disable. Since that probably can't happen, I wonder if there are values smaller than 2 seconds that may be better. I could see low end boxes being much more sensitive to this issue. A lot of packets can come in a 2 second window with more and more people getting 100Mb+ connections.
-
It didnt help. It takes this forum and store.netgate.com down as well easily.
Throughput needs only to be about 20mbit before it dies and cant handle the traffic.
Its no issue if you use windows firewall as the frontend and the webserver itself can easily handle the traffic both regarding backlog and overall traffic and packets.
Its pfSense related and take it down instantly.
-
-
I havent tested it on FreeBSD.
So I cant relate to that. You are more than welcome to provide me with a FreeBSD target on PM, so we can test.
Its pfSense related and take it down instantly.
So it does NOT happen on FreeBSD?
-
Its pfSense related and take it down instantly.
So it does NOT happen on FreeBSD?
I tried it on a clean freeBSD 10.1
- it was much better than pfsense, not saying that is was 100% up, it had some packetloss as well, but no more then pfsense which instantly or mostly get 90-100% packetloss.
It was without any tuning as well on freebsd 10.1
- it was much better than pfsense, not saying that is was 100% up, it had some packetloss as well, but no more then pfsense which instantly or mostly get 90-100% packetloss.
-
"but no more then pfsense which instantly or mostly get 90-100% packetloss"
So was it less or more. Same? how much less or more?
-
"but no more then pfsense which instantly or mostly get 90-100% packetloss"
So was it less or more. Same? how much less or more?
It really depend on the attack method. SYN-ACK or SYN-FIN, packet size etc.
But after over 100 test i would still say pfsense could have done it better. It is not handling SYN request correctly. I don't have the skills to fix it or go deeper into it.
Result:
FreeBSD 10.1 = every 7-8th ping = packetloss (avg packetloss 10-20%)
PFsense = every 1-2nd ping packetloss (avg packetloss 80-90%)So there is a notable difference clearly. PFsense was running stateful. Stateless helped a little bit.
-
Anybody with serious freeBSD skills wanting to help us test this??
Money could be involved :D
-
I wonder if getting someone from the FreeBSD forums may be useful at this point.
-
We have had ZERO response from the pfSense guys. This is quite disturbing since we can take down any site protected by pfSense as it is.
Right now its better to run without pf at all and rely on windows Firewall on VM's and let pf handle the routing. Only way to survive the attacks as it is.
Thinking og getting my old ISA2006 online again to test and see how it behaves.
-
A little more…
http://youtu.be/boa7bbeKRG0
Now we can limit the states that is created but basic routing is not working....still.
-
youtu huh?
-
Video is private :
-
Better??
-
Have you tried PFSense 2.2? I know FreeBSD 10.1 was tested and PFSense 2.1.5 was tested, but those are two quite different versions of FreeBSD.
I could let you try it against my box, but only a few quick tests, wife likes to watch Netflix :-)
-
We have had ZERO response from the pfSense guys. This is quite disturbing since we can take down any site protected by pfSense as it is.
Right now its better to run without pf at all and rely on windows Firewall on VM's and let pf handle the routing. Only way to survive the attacks as it is.
Thinking og getting my old ISA2006 online again to test and see how it behaves.
You didn't try, now did you? Did you send a message to coreteam? To Chris, or me, or…?
No, you just randomly attacked the store and forum.
-
WTF???!!!
Lowprofile is the one having this dialog with Chris and we are working hard on trying to solve this!?
Spent most of friday evening in a datacenter discussing options with Lowprofile in here…
So you are fucking acusing me of taking the store and forum offline...
You take those words right back or you will hear from a lawyer ....THATS NOT OK TO INSINUATE THAT AT ALL!!
-
WTF???!!!
Lowprofile is the one having this dialog with Chris and we are working hard on trying to solve this!?
Spent most of friday evening in a datacenter discussing options with Lowprofile in here…
So you are fucking acusing me of taking the store and forum offline...
You take those words right back or you will hear from a lawyer ....THATS NOT OK TO INSINUATE THAT AT ALL!!
You said you did right here:
https://forum.pfsense.org/index.php?topic=88694.msg491103#msg491103"It didnt help. It takes this forum and store.netgate.com down as well easily."
It didnt help. It takes this forum and store.netgate.com down as well easily.
Throughput needs only to be about 20mbit before it dies and cant handle the traffic.
Its no issue if you use windows firewall as the frontend and the webserver itself can easily handle the traffic both regarding backlog and overall traffic and packets.
Its pfSense related and take it down instantly.
-
Its pfSense related and take it down instantly.
So it does NOT happen on FreeBSD?
Now that I've got the "script" (it's C code) that Supermule posted compiling, we'll look at it.
Reporting this to Chris in private is so seriously Not how this is done.
There was no email to security@
There was no email to coreteam@ -
popcorn
-
Dude….pls. READ what it says....
IT takes the forum and the store offline. IT didnt say I did it.... but suggests that YOU are vulnerable as well ....
So I inform you and then its my fault and me beeing behind it??
I am only reporting whats found among people I talk to IRL frequently. They test like mad people at the moment to come up with whats wrong with the software and I only report it when and IF we find something....
-
I havent reported anything to Chris. Lowprofile is the one handling that and the one in touch with Chris.
He has reported our findings or no findings… He is the one and NOT me....
So then you wouldnt find any email from me to the adresses you wrote because I didnt send one!
Its THAT obvious...
@gonzopancho:
Its pfSense related and take it down instantly.
So it does NOT happen on FreeBSD?
Now that I've got the "script" (it's C code) that Supermule posted compiling, we'll look at it.
Reporting this to Chris in private is so seriously Not how this is done.
There was no email to security@
There was no email to coreteam@ -
This is exactly why I never joke about farting in a crowded elevator to my GF.
I'll mention it in jest. Someone will actually do it and suddenly…. I'm the bad guy... :-\ -
Dude….pls. READ what it says....
IT takes the forum and the store offline. IT didnt say I did it.... but suggests that YOU are vulnerable as well ....
So I inform you and then its my fault and me beeing behind it??
I am only reporting whats found among people I talk to IRL frequently. They test like mad people at the moment to come up with whats wrong with the software and I only report it when and IF we find something....
Reporting issues with the software is fine. Reporting issues with the software in such a way that someone can reproduce them is even better (so your 'script' is actually useful).
Attacking other people's infrastructure (which you reported having done) is not fine.
And, frankly, you DID NOT INFORM ME.
Having a private conversation with Chris (and at this point I don't care if it was you, or Lowprofile , or someone else) and having a discussion in the forum where you report that "no response from the pfsense guys"
We have had ZERO response from the pfSense guys. This is quite disturbing since we can take down any site protected by pfSense as it is.
Right now its better to run without pf at all and rely on windows Firewall on VM's and let pf handle the routing. Only way to survive the attacks as it is.
Thinking og getting my old ISA2006 online again to test and see how it behaves.
is not responsible, or friendly, or even … professional.