UDP DDoS protection with pfSense
-
-
@BlueKobold:
Its a 10gb attack i cant get that big bandwidth here. and as your question my server does not respond to those, just drops them
And if you will get 10 GBit/s at the WAN and they attack you with 300 GBit/s you will loose again!
is there any way to block the attacks before it comes to my network without filling it?
Your ISP or your hoster would be setting up a device or service in front of your IP address.
in my case i have a fiber line connected through media converter and an Ethernet wire from media converter goes to switch from where all the servers get their public static ip,
Without SPI/NAT or Firewall and rules you are attaching servers to the Internet???
maybe some way to plug that main media converter Ethernet wire into firewall,
Would be a more secure solution as before you goes.
but then what will be its wan ip? so confusing!
The one you enter in the WAN menu.
There must be a way though, (ISP don't give a damn, all they do it null route my ip)
Perhaps he can´t do anything? There are some devices that can be placed in front of your business
Internet connection but they are often very expensive and there are also some services that can be
hired or rent to take the DDoS load from the line but also mostly very expensive.The Corero IPS 5500 ES-Series would be one of this devices you could try to place in front of your
firewall and then you would be back in game. Corero SmartWallCorero is using hardware from Tilera, based on so called many Core CPUs and this is purely not cheap.
Thanks for the detailed explanation! One question though that how does Hardware limit the rate? if the ddos is landing to my Corero SmartWall or whatever i device i use, won't it be the same as it landing on the firewall machine, because once it lands to me my bandwidth will be filled up again.
-
@BlueKobold:
inbound traffic is related to traffic that originated behind the NAT so NAT before state checking is needed.
And this is in my poor opinion the exactly point which is totally different each from other!
- The home or consumer grade SPI/NAT is doing something like the following:
Deny all and then have a look in the connection table for an open connection from inside
So it is wanted that all packets are staying outside. - But the SPI/NAT way from the pfSense is doing it in the total turned around direction as I see
it right, please correct me if I am wrong with this!
Let them (TCP/IP packets) all in for inspect them by one or more rules
So the many packets from an attack are able to get in and render or filling the pipe and nothing more goes.
Frank, interesting thoughts. I don't know if I'm wrong or you are or we both are, but this is becoming interesting.
So we have traffic from LAN side a.b.c.d:123456 destined for 1.2.3.4:80, with NAT enabled WAN is J.K.L.M, so NAT rewrites it to be sourced from J.K.L.M:987653. The return traffic is from 1.2.3.4, to J.K.L.M. Does pfSense translate/rewrite the packet to be to a.b.c.d:123456 and then look in the firewall rules? I'm not 100% sure, but the documentation I've read at least implies that.
Take a Linksys 54G doing similar function of NAT with stateful firewall. Does the return traffic get rewritten and then firewall rules applied? I don't know, but I think it should. The firewall state table should have the outbound packet with LAN address/port, NAT does the rewrite before it leaves on the WAN interface. To me that means the return traffic must be "de-NATted" before you look at the firewall state tables.
Any return traffic would have destaddr in the packet to be the WAN interface; NAT would have a lookup of WAN/port matching LAN/port2 in the table.
I don't think the simple lookup should be an issue, even at high inbound PPS. I think what becomes more of an issue is what happens when you get a match; you need to rewrite pieces of the packet (dest MAC, dest IP, dest port, one or two checksums) before passing the packet on. That takes resources and time. If checksumming is offloaded, then there is the potential for a context switch to get the modified packet back into the stack.I guess it's time to start sticking my nose into pf implementation on FreeBSD.
mike
- The home or consumer grade SPI/NAT is doing something like the following:
-
One question though that how does Hardware limit the rate?
That is what i have found on the Internet over those game play, but I really
know that absolutely no one will talk about the really work flow, that is called
security by obscurity I think. (Pictures: DDoS attacks Layer of defense & DME/Multi Core CPU/SME)if the ddos is landing to my Corero SmartWall or whatever i device i use, won't it be the same as it landing on the firewall machine, because once it lands to me my bandwidth will be filled up again.
At first and owed to the posts made by @johnpoz I was also thinking but then I found more and
more network draws that would be telling other things, where such a device should be or must
be placed. And this is not even the same point if we are talking about some or more different acting
companies, likes ISPs, Data Centers or Webhosters. (Picture: Coreros ReputationWatch)because once it lands to me my bandwidth will be filled up again.
The "device" is placed in front of the firewall and sort all bad things out and let only
the clean traffic passing through to the firewall then.
(Picture: DDoS attacks layer of defense & First line of defense)
-
Hello Mike,
I am not a professional or something like this, but I am really interested in the theme and perhaps
the end of this story or a solution that can be realized.I don't know if I'm wrong or you are or we both are, but this is becoming interesting.
For sure this could all be, I am not a hardware engineer, code writer, pfSense core development
member, pfSense expert or guru or a forum administrator, I am only interested in this theme and pfSense
more or less. And if a full featured software based firewall such as pfSense is would not be albe to
handle an attacks like this, but an ordinary lazy plastic home router for ~$100 is able to do, it
becomes even more and more interesting for me, sorry to tell this so plain and naive, but it
is like it is! Or come closer to this point, to find out what could be making the difference
between them would be really interesting for me.I don't think the simple lookup should be an issue, even at high inbound PPS.
And here the false is six feed under, with an lazy, tiny or very cheap ASIC/FPGA at this point
it could be running likes hell, to sort this packets out, and for sure pfSense is not needing of
this if we have a closer look at the most hardware we are talking here in the forum or the pfSense
store is offering now. There are worlds between them (home router & SG-xxx units).And I really think the NAT mechanism is more less then a difficult or tricky way.
Client A is opening behind the NAT a web page this data would be pulled
from the outside and to the Client behind the NAT and will be forwarded, all other
coming from outside will be dropped. Something really tiny and lazy it must be in
my eyes. And for sure I know that I am jumping now in an open shark mouth but
could it be that this version of doing NAT will be able to find its way inside of the
code from FreeBSD or pfSense only perhaps? Please remember I am no code writer
and developer, I don´t know anything about this and what other code or functions
on top will be affected by setting this version of up and for sure not as a replacement
for the actual NAT version or doing!!! Only perhaps as a so called drop down menu
variant where the users or customers are able to chose what kind of NAT version
they want to use, if this could be done. I really know some peoples they are aware
from this and don´t want this really since years, and for sure they are all knowing
why and why not, not likes me as a noob and beginner, but perhaps this is making
the difference in thinking of those cases.Because in my opinion, after this SPI and NAT process the firewall rules must also
only inspecting then the passing NAT traffic and not all packets that are arriving,
and for sure also the snort or suricata rules.I really don´t know if I am now misleading others or running in a so called hamster
wheel or that I am a prisoner of my own mind, I am only interested in to understand
this point, why a server grade hardware based firewall is not able and a lazy
~$100 home is able to do so. -
Thanks for the detailed explanation! One question though that how does Hardware limit the rate? if the ddos is landing to my Corero SmartWall or whatever i device i use, won't it be the same as it landing on the firewall machine, because once it lands to me my bandwidth will be filled up again.
It's not possible to do anything on your end of the line to stop the typical UDP flood DDoS, because those are bandwidth exhaustion attacks (usually DNS or NTP amplification). It's too late by the time it gets to you, you can't change the fact your connection is flooded. Your ISP has to stop it before it reaches your connection.
Where boxes like that can be useful are attacks like large scale SYN floods that go beyond what any firewall can handle in new connections/sec, but aren't so large that they completely fill your Internet connection.
@BlueKobold:
And if a full featured software based firewall such as pfSense is would not be albe to
handle an attacks like this, but an ordinary lazy plastic home router for ~$100 is able to doThere is no circumstance in which a consumer grade router is better at handling DDoS. Consumer grade devices are extremely poorly suited for resource exhaustion attacks.
-
why a server grade hardware based firewall is not able and a lazy
~$100 home is able to do so.I think you will find that many consumer grade devices don't even have anything close to what you would call a firewall. All of my original devices were not.
It's the very reason I looked up Monowall then soon after found pfSense. You got one thing right- lazy!
-
You cannot use pfsense for DDoS protection.
You can still flood it with sub 10Mbit/s traffic and it dies.
Tested on 2.2.4 AMD64.
-
And here we go again, exactly as predicted… Yay.
-
I wonder how efficient NAT is. NAT could be quite efficient if integrated into the states.
Ingress
WAN - SPI - NAT - Core logic - LAN
Egress
WAN - Core Logic - NAT - SPI - LANAssuming this modular setup, which is great of proper layering, but depending on how transparent the layering, the NAT may need to do its own lookup after the SPI has already done a lookup.
If the NAT was integrated into the SPI
Ingress
WAN - SPI/NAT - Core logic - LAN
Egress
WAN - Core Logic - SPI/NAT - LANNow you only have one lookup and all of the NAT state is stored along with the firewall state. Assuming it isn't already similar to this.
I also read that the NAT is single threaded, but which reduces the usefulness of the firewall being threaded because traffic from the firewall gets shoved through the NAT anyway, probably making things worse than just single theaded. Again, I make a lot of assumption, many of which are probably wrong because PFSense seems to have great performance as long as I don't get a lot of new states being created.
Either way, I can't wait for the line-rate stuff. There's going to be so much change that it's not very useful discussing the current system. Worry about performance tuning after 3.0 :p
-
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
-
Inbound traffic on WAN: for it to pass, it needs to be IP of the pfSense WAN port (or WAN broadcast) and the state table needs to have an entry with the source port matching the dest port of the inbound traffic, no? Lower layers of the stack should be dropping packets not destined for the WAN (assuming not promsicuous mode), so would a first level check of "does this dest port match any source port in the state table" work? I think the big writer of a state table is the outbound path, updates on a single entry on the inbound (I'm assuming very simple case here of no open ports on WAN), so if the state table were also indexed by source port it may be possible to do the lookup (read) lockless. If no match on ports, don't bother doing anything else, just drop the packet. If there's a port match, then pass it into the NAT/redir logic and onward.
Of course a lot of this depends on the definition of "handling" the line rate traffic. Simply not crashing the box (implies all you are doing is dropping packets) or actually getting legitimate work done?
Disclaimer: the code may already be doing this, I don't know. Just random thoughts on what actually happens in IPV4 NAT.
-
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
Line rate talk was packets per second, but did not mention the rate of new connections. Every packet already needs to look up in the state table, so reading from the state table will handle millions of packets per second. The next question is how quickly can you write to the state table.
Even if the state table fills up quickly, you can continue reading from it just fine. In theory, existing connections should continue to work uninhibited.
Just a random idea. They could reserve N number of states in the state table for connection going out. This way if someone SYN floods you on your WAN, you can still make some connections out even if incoming WAN connections in are rejected because of a full table.
-
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
Line rate talk was packets per second, but did not mention the rate of new connections. Every packet already needs to look up in the state table, so reading from the state table will handle millions of packets per second. The next question is how quickly can you write to the state table.
Even if the state table fills up quickly, you can continue reading from it just fine. In theory, existing connections should continue to work uninhibited.
Just a random idea. They could reserve N number of states in the state table for connection going out. This way if someone SYN floods you on your WAN, you can still make some connections out even if incoming WAN connections in are rejected because of a full table.
Hmm. Watermarking basically. "I can handle X conn/sec max, start doing RED at %y(X) so I have bandwidth for legitimate traffic". Interesting.
That ties into "handling" the load; be able to drop everything without crashing or actually do useful work.
-
@mer:
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
Line rate talk was packets per second, but did not mention the rate of new connections. Every packet already needs to look up in the state table, so reading from the state table will handle millions of packets per second. The next question is how quickly can you write to the state table.
Even if the state table fills up quickly, you can continue reading from it just fine. In theory, existing connections should continue to work uninhibited.
Just a random idea. They could reserve N number of states in the state table for connection going out. This way if someone SYN floods you on your WAN, you can still make some connections out even if incoming WAN connections in are rejected because of a full table.
Hmm. Watermarking basically. "I can handle X conn/sec max, start doing RED at %y(X) so I have bandwidth for legitimate traffic". Interesting.
That ties into "handling" the load; be able to drop everything without crashing or actually do useful work.
Not drop packets, drop new connection attempts, or do something with them to not allow new connections eat up all of the CPU.
-
Alright, my first post and well it will be in DDOS because this shit is eating my brains out. Simple thing and remember this folks pFsense is for home user grade not business like providing filtering methods for this type of attack.
Something that can take a DDOS attack and let's say not dying every time… a test setup will go with something like this:
FreeBSD with kernel optimization and NETMAP some intel 540 adaptor(I don't like them) and a sensor that will detect the attack and apply rules to your little box. Intel know about the ntuple but is limited for filtering like this, proto,ports,IP nothing more than this so hmm we can kick out some DNS,NTP amplification, and it will do it at 10G line rate but that's all. Now for the next part of my funny lab.
Anyway now I'm playing with Some 2xE5-1620 netmap and Chelsio T5, and boy I tell you this card is nice. What it can do more than those Intel well same thing but It does have ASIC hmm here is the pretty part where you can get and some filtering, you can program those ASIC to take the real beating, and over that you can redirect on fly some part's of the traffic to you machine where you can filter with ipfw and from here the game begins.
Those big solution that cost over 200k this is the method that use, and of course a big army of devs and HW that will make you drool like a fool. In near future even those solution won't help anymore because the problem begins with the ISP that allow this type of traffic to go from their network.
Some resolution would be if the traffic can be cut from the roots, but our big carrier like Telia and NTT they like you to buy more and more and more 10-40-100G. This are my 2 cent. -
@Mup:
Simple thing and remember this folks pFsense is for home user grade not business like providing filtering methods for this type of attack.
That's bullshit unless you're also going to say a Cisco ASA is for home user grade not business. The hardware appliances we sell stand up to DDoS as well as ASAs and comparable commercial firewalls costing several times as much, in some cases over 10 times as much money for the same performance in new connections/sec handling. You don't put a firewall in front of things that are subject to resource exhaustion attacks.
@Mup:
Anyway now I'm playing with Some 2xE5-1620 netmap and Chelsio T5, and boy I tell you this card is nice. What it can do more than those Intel well same thing but It does have ASIC hmm here is the pretty part where you can get and some filtering, you can program those ASIC to take the real beating, and over that you can redirect on fly some part's of the traffic to you machine where you can filter with ipfw and from here the game begins.
There are some interesting possibilities with the hardware filtering there for sure, let's just say we're aware. ;) That's the type of thing that can stand up to extreme abuse.
-
That's bullshit unless you're also going to say a Cisco ASA is for home user grade not business. The hardware appliances we sell stand up to DDoS as well as ASAs and comparable commercial firewalls costing several times as much, in some cases over 10 times as much money for the same performance in new connections/sec handling. You don't put a firewall in front of things that are subject to resource exhaustion attacks.
The HW appliance that you sell, alright let's talk a little about it, do you filter something with the Chelsio card with that can go at line rate ? If I do a ddos on it with fragmented packets and a little spoofed you think that can withstand ?
Don't bullshit next time, it say 8 mil conn right ? That will saturate fast, or it will crash for sure at 64 bytes with 1-2 mil of con… You say about the user grade business please remake your equipment with 2 power source after that you can go after other prizes. "CMB" I'm not here to kick about this solution that is not enough prof. For this I wanted to go on next thing, did you play with the Chelsio card, do you want to share more information ?
-
Alright, my first post and well it will be in DDOS because this shit is eating my brains out. Simple thing and remember this folks pFsense is for home user grade not business like providing filtering methods for this type of attack.
No one, absolutely no one is pressing you to use pfSense at your business point of presence.
If you find that this is not the real solution for business you could go with other solutions elsewhere.
There are enough out of them and also OpenSource likes pfSense is bases on.
OpenBSD & BGPDD
OpenBSD & Quagga
DD-WRT or OpenWRT
MikroTik RouterOS
fli4l & Eisfair
Endian
ZeroShell
OPNSense
Sophos UTM
Untangle UTM
ClearOS GatewayIts all free of charge and also able to use for home and for business, ok some must be paid then.
Something that can take a DDOS attack and let's say not dying every time… a test setup will go with something like this:
Why? Because you say? And why FreeBSD? Because pfSense is based on? I really think pfSense is really
far away from the original FreeBSD, ok it is based on, but after so many changes it would be not the same.And why this development peoples should do DDoS tests? Because you say or plain want this?
FreeBSD with kernel optimization and NETMAP some intel 540 adaptor(I don't like them) and a sensor that will detect the attack and apply rules to your little box.
And where we will see it then in pfSense? Never? And why then taking such a construct to test it out?
Intel know about the ntuple but is limited for filtering like this, proto,ports,IP nothing more than this so hmm we can kick out some DNS,NTP amplification, and it will do it at 10G line rate but that's all. Now for the next part of my funny lab.
It is really funny, but not more.
Anyway now I'm playing with Some 2xE5-1620 netmap and Chelsio T5, and boy I tell you this card is nice. What it can do more than those Intel well same thing but It does have ASIC hmm here is the pretty part where you can get and some filtering, you can program those ASIC to take the real beating, and over that you can redirect on fly some part's of the traffic to you machine where you can filter with ipfw and from here the game begins.
One of the Chelsio adapters are able to full offload the NAT process, and many other things also,
so that means this ASIC/FPGA is used to offload some tasks from the pfSense, and if this is to
much money and pfsense is only for home usage you can freely use the Innominate mGuard SD PCIe
or mGuard SD PCI card as well to off load the NAT process totally. And by the way if the pfSense is only for home, who should buy
a Chelsio 10 GBit/s server adapter for ~800 €? Home users?Those big solution that cost over 200k this is the method that use, and of course a big army of devs and HW that will make you drool like a fool. In near future even those solution won't help anymore because the problem begins with the ISP that allow this type of traffic to go from their network.
Bigger ISPs are often more expensive but do this like no other one to protect their clients from
this attacks or offering much better to pay for services to get rid of this hassle. But an ISP is
not an likes the other ISP.Some resolution would be if the traffic can be cut from the roots, but our big carrier like Telia and NTT they like you to buy more and more and more 10-40-100G. This are my 2 cent.
Why should they not selling this as an ISP?
The HW appliance that you sell, alright let's talk a little about it, do you filter something with the Chelsio card with that can go at line rate ? If I do a ddos on it with fragmented packets and a little spoofed you think that can withstand ?
Hardware appliances from the pfSense store are not sold out as a DDoS stopper but as they are a
software based firewall. And yes for sure there are also some adapters out there that could really
do the job without touching the pfSense code once! And the best no driver will be need also.
You only need some coders or programmers that are willing to do the job, thats it.
on this cards are Tilera TILE-Gx chips are working from 16 over 36 up to 72 many
core CPUs and there can be run a own system on the chip (on the cards soldered)
it selfs. Why Tilera? Because their CPUs are working in 70 % off all bigger anti
DDoS devices sold everywhere. its only a smaller form factor easily to add as a
PCIe card into appliances.Tilera original
Lanners equivalentFrom the Tilera programmers storage sorted with over 2500 .rmps you will be able
to set it up really fast and powerful. So if you are interested in to solve this problem
out, and also on a sufficient way like the big players are doing it right, start up a poll
or bounty to doing this. And please let the ASIC on the Chelsio adapters be as it is,
because then other things and functions on this cards will be not playing right as I see it. -
One option to thwart SYN attacks from spoofed sources is to lower the timeout for unestablished connections. The default is something like 90 seconds. I'm sorry, but with 300ms latencies, I expect an endpoint to respond just a wee bit faster.
To consume all of your firewall's states, you need to receive ((TotalNumberOfStates/2)64bytes8bits)/TimeOutWindow) of bandwidth. Assume 16GiB of memory, 8mil states. With a 5 second timeout instead of a 90sec timeout, that's 400Mbit/s of syn packets to consume all of your states. With a 1 second timeout, that's now 2Gbit/s of SYN packets. Establish connections are just fine.
I'm not sure how well PFSense/FreeBSD handles creating new states in the first place because that requires making new entries in the state tables. Depending on the design, that may require a write lock and still slow things down. I have no idea what they're using. The eventual per core state table will mean virtually no lock contention and will dramatically help with scaling.
I wonder how often fragmented packets happen. Possibly just block them all. IPv6 those packets, discover the path MTU or get a decent ISP!