UDP DDoS protection with pfSense
-
I wonder how efficient NAT is. NAT could be quite efficient if integrated into the states.
Ingress
WAN - SPI - NAT - Core logic - LAN
Egress
WAN - Core Logic - NAT - SPI - LANAssuming this modular setup, which is great of proper layering, but depending on how transparent the layering, the NAT may need to do its own lookup after the SPI has already done a lookup.
If the NAT was integrated into the SPI
Ingress
WAN - SPI/NAT - Core logic - LAN
Egress
WAN - Core Logic - SPI/NAT - LANNow you only have one lookup and all of the NAT state is stored along with the firewall state. Assuming it isn't already similar to this.
I also read that the NAT is single threaded, but which reduces the usefulness of the firewall being threaded because traffic from the firewall gets shoved through the NAT anyway, probably making things worse than just single theaded. Again, I make a lot of assumption, many of which are probably wrong because PFSense seems to have great performance as long as I don't get a lot of new states being created.
Either way, I can't wait for the line-rate stuff. There's going to be so much change that it's not very useful discussing the current system. Worry about performance tuning after 3.0 :p
-
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
-
Inbound traffic on WAN: for it to pass, it needs to be IP of the pfSense WAN port (or WAN broadcast) and the state table needs to have an entry with the source port matching the dest port of the inbound traffic, no? Lower layers of the stack should be dropping packets not destined for the WAN (assuming not promsicuous mode), so would a first level check of "does this dest port match any source port in the state table" work? I think the big writer of a state table is the outbound path, updates on a single entry on the inbound (I'm assuming very simple case here of no open ports on WAN), so if the state table were also indexed by source port it may be possible to do the lookup (read) lockless. If no match on ports, don't bother doing anything else, just drop the packet. If there's a port match, then pass it into the NAT/redir logic and onward.
Of course a lot of this depends on the definition of "handling" the line rate traffic. Simply not crashing the box (implies all you are doing is dropping packets) or actually getting legitimate work done?
Disclaimer: the code may already be doing this, I don't know. Just random thoughts on what actually happens in IPV4 NAT.
-
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
Line rate talk was packets per second, but did not mention the rate of new connections. Every packet already needs to look up in the state table, so reading from the state table will handle millions of packets per second. The next question is how quickly can you write to the state table.
Even if the state table fills up quickly, you can continue reading from it just fine. In theory, existing connections should continue to work uninhibited.
Just a random idea. They could reserve N number of states in the state table for connection going out. This way if someone SYN floods you on your WAN, you can still make some connections out even if incoming WAN connections in are rejected because of a full table.
-
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
Line rate talk was packets per second, but did not mention the rate of new connections. Every packet already needs to look up in the state table, so reading from the state table will handle millions of packets per second. The next question is how quickly can you write to the state table.
Even if the state table fills up quickly, you can continue reading from it just fine. In theory, existing connections should continue to work uninhibited.
Just a random idea. They could reserve N number of states in the state table for connection going out. This way if someone SYN floods you on your WAN, you can still make some connections out even if incoming WAN connections in are rejected because of a full table.
Hmm. Watermarking basically. "I can handle X conn/sec max, start doing RED at %y(X) so I have bandwidth for legitimate traffic". Interesting.
That ties into "handling" the load; be able to drop everything without crashing or actually do useful work.
-
@mer:
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
Line rate talk was packets per second, but did not mention the rate of new connections. Every packet already needs to look up in the state table, so reading from the state table will handle millions of packets per second. The next question is how quickly can you write to the state table.
Even if the state table fills up quickly, you can continue reading from it just fine. In theory, existing connections should continue to work uninhibited.
Just a random idea. They could reserve N number of states in the state table for connection going out. This way if someone SYN floods you on your WAN, you can still make some connections out even if incoming WAN connections in are rejected because of a full table.
Hmm. Watermarking basically. "I can handle X conn/sec max, start doing RED at %y(X) so I have bandwidth for legitimate traffic". Interesting.
That ties into "handling" the load; be able to drop everything without crashing or actually do useful work.
Not drop packets, drop new connection attempts, or do something with them to not allow new connections eat up all of the CPU.
-
Alright, my first post and well it will be in DDOS because this shit is eating my brains out. Simple thing and remember this folks pFsense is for home user grade not business like providing filtering methods for this type of attack.
Something that can take a DDOS attack and let's say not dying every time… a test setup will go with something like this:
FreeBSD with kernel optimization and NETMAP some intel 540 adaptor(I don't like them) and a sensor that will detect the attack and apply rules to your little box. Intel know about the ntuple but is limited for filtering like this, proto,ports,IP nothing more than this so hmm we can kick out some DNS,NTP amplification, and it will do it at 10G line rate but that's all. Now for the next part of my funny lab.
Anyway now I'm playing with Some 2xE5-1620 netmap and Chelsio T5, and boy I tell you this card is nice. What it can do more than those Intel well same thing but It does have ASIC hmm here is the pretty part where you can get and some filtering, you can program those ASIC to take the real beating, and over that you can redirect on fly some part's of the traffic to you machine where you can filter with ipfw and from here the game begins.
Those big solution that cost over 200k this is the method that use, and of course a big army of devs and HW that will make you drool like a fool. In near future even those solution won't help anymore because the problem begins with the ISP that allow this type of traffic to go from their network.
Some resolution would be if the traffic can be cut from the roots, but our big carrier like Telia and NTT they like you to buy more and more and more 10-40-100G. This are my 2 cent. -
@Mup:
Simple thing and remember this folks pFsense is for home user grade not business like providing filtering methods for this type of attack.
That's bullshit unless you're also going to say a Cisco ASA is for home user grade not business. The hardware appliances we sell stand up to DDoS as well as ASAs and comparable commercial firewalls costing several times as much, in some cases over 10 times as much money for the same performance in new connections/sec handling. You don't put a firewall in front of things that are subject to resource exhaustion attacks.
@Mup:
Anyway now I'm playing with Some 2xE5-1620 netmap and Chelsio T5, and boy I tell you this card is nice. What it can do more than those Intel well same thing but It does have ASIC hmm here is the pretty part where you can get and some filtering, you can program those ASIC to take the real beating, and over that you can redirect on fly some part's of the traffic to you machine where you can filter with ipfw and from here the game begins.
There are some interesting possibilities with the hardware filtering there for sure, let's just say we're aware. ;) That's the type of thing that can stand up to extreme abuse.
-
That's bullshit unless you're also going to say a Cisco ASA is for home user grade not business. The hardware appliances we sell stand up to DDoS as well as ASAs and comparable commercial firewalls costing several times as much, in some cases over 10 times as much money for the same performance in new connections/sec handling. You don't put a firewall in front of things that are subject to resource exhaustion attacks.
The HW appliance that you sell, alright let's talk a little about it, do you filter something with the Chelsio card with that can go at line rate ? If I do a ddos on it with fragmented packets and a little spoofed you think that can withstand ?
Don't bullshit next time, it say 8 mil conn right ? That will saturate fast, or it will crash for sure at 64 bytes with 1-2 mil of con… You say about the user grade business please remake your equipment with 2 power source after that you can go after other prizes. "CMB" I'm not here to kick about this solution that is not enough prof. For this I wanted to go on next thing, did you play with the Chelsio card, do you want to share more information ?
-
Alright, my first post and well it will be in DDOS because this shit is eating my brains out. Simple thing and remember this folks pFsense is for home user grade not business like providing filtering methods for this type of attack.
No one, absolutely no one is pressing you to use pfSense at your business point of presence.
If you find that this is not the real solution for business you could go with other solutions elsewhere.
There are enough out of them and also OpenSource likes pfSense is bases on.
OpenBSD & BGPDD
OpenBSD & Quagga
DD-WRT or OpenWRT
MikroTik RouterOS
fli4l & Eisfair
Endian
ZeroShell
OPNSense
Sophos UTM
Untangle UTM
ClearOS GatewayIts all free of charge and also able to use for home and for business, ok some must be paid then.
Something that can take a DDOS attack and let's say not dying every time… a test setup will go with something like this:
Why? Because you say? And why FreeBSD? Because pfSense is based on? I really think pfSense is really
far away from the original FreeBSD, ok it is based on, but after so many changes it would be not the same.And why this development peoples should do DDoS tests? Because you say or plain want this?
FreeBSD with kernel optimization and NETMAP some intel 540 adaptor(I don't like them) and a sensor that will detect the attack and apply rules to your little box.
And where we will see it then in pfSense? Never? And why then taking such a construct to test it out?
Intel know about the ntuple but is limited for filtering like this, proto,ports,IP nothing more than this so hmm we can kick out some DNS,NTP amplification, and it will do it at 10G line rate but that's all. Now for the next part of my funny lab.
It is really funny, but not more.
Anyway now I'm playing with Some 2xE5-1620 netmap and Chelsio T5, and boy I tell you this card is nice. What it can do more than those Intel well same thing but It does have ASIC hmm here is the pretty part where you can get and some filtering, you can program those ASIC to take the real beating, and over that you can redirect on fly some part's of the traffic to you machine where you can filter with ipfw and from here the game begins.
One of the Chelsio adapters are able to full offload the NAT process, and many other things also,
so that means this ASIC/FPGA is used to offload some tasks from the pfSense, and if this is to
much money and pfsense is only for home usage you can freely use the Innominate mGuard SD PCIe
or mGuard SD PCI card as well to off load the NAT process totally. And by the way if the pfSense is only for home, who should buy
a Chelsio 10 GBit/s server adapter for ~800 €? Home users?Those big solution that cost over 200k this is the method that use, and of course a big army of devs and HW that will make you drool like a fool. In near future even those solution won't help anymore because the problem begins with the ISP that allow this type of traffic to go from their network.
Bigger ISPs are often more expensive but do this like no other one to protect their clients from
this attacks or offering much better to pay for services to get rid of this hassle. But an ISP is
not an likes the other ISP.Some resolution would be if the traffic can be cut from the roots, but our big carrier like Telia and NTT they like you to buy more and more and more 10-40-100G. This are my 2 cent.
Why should they not selling this as an ISP?
The HW appliance that you sell, alright let's talk a little about it, do you filter something with the Chelsio card with that can go at line rate ? If I do a ddos on it with fragmented packets and a little spoofed you think that can withstand ?
Hardware appliances from the pfSense store are not sold out as a DDoS stopper but as they are a
software based firewall. And yes for sure there are also some adapters out there that could really
do the job without touching the pfSense code once! And the best no driver will be need also.
You only need some coders or programmers that are willing to do the job, thats it.
on this cards are Tilera TILE-Gx chips are working from 16 over 36 up to 72 many
core CPUs and there can be run a own system on the chip (on the cards soldered)
it selfs. Why Tilera? Because their CPUs are working in 70 % off all bigger anti
DDoS devices sold everywhere. its only a smaller form factor easily to add as a
PCIe card into appliances.Tilera original
Lanners equivalentFrom the Tilera programmers storage sorted with over 2500 .rmps you will be able
to set it up really fast and powerful. So if you are interested in to solve this problem
out, and also on a sufficient way like the big players are doing it right, start up a poll
or bounty to doing this. And please let the ASIC on the Chelsio adapters be as it is,
because then other things and functions on this cards will be not playing right as I see it. -
One option to thwart SYN attacks from spoofed sources is to lower the timeout for unestablished connections. The default is something like 90 seconds. I'm sorry, but with 300ms latencies, I expect an endpoint to respond just a wee bit faster.
To consume all of your firewall's states, you need to receive ((TotalNumberOfStates/2)64bytes8bits)/TimeOutWindow) of bandwidth. Assume 16GiB of memory, 8mil states. With a 5 second timeout instead of a 90sec timeout, that's 400Mbit/s of syn packets to consume all of your states. With a 1 second timeout, that's now 2Gbit/s of SYN packets. Establish connections are just fine.
I'm not sure how well PFSense/FreeBSD handles creating new states in the first place because that requires making new entries in the state tables. Depending on the design, that may require a write lock and still slow things down. I have no idea what they're using. The eventual per core state table will mean virtually no lock contention and will dramatically help with scaling.
I wonder how often fragmented packets happen. Possibly just block them all. IPv6 those packets, discover the path MTU or get a decent ISP!