UDP DDoS protection with pfSense
-
I have a very sensitive business which needs 100% up time,
Then, as mentioned by others, you probably need to hire a service to filter your traffic before it comes down the pipe from ISP to you. Or if the ISP has the capability, get them to filter your traffic instead of just null routing.
I'm curious. Do you have any inclination at all of who or the motive that is behind the attack? Competitor, someone doesn't like you, disgruntled customer or employee, extortion, etc.?
Yup its a possible competitor that's for sure :)
-
What I would like to know is what the OP was reading that pointed him to pfsense mitigating attacks?
"i found online to go with pfSense, i saw many people mitigating attacks with it too"
There are lots of threads here asking the same thing - and they always get the same answer, you can not stop a DDOS with a firewall.. So either he was not reading the full thread/article or misread the information?
If the OP business is so critical and of nature that ddos is of concern, they need to host services out of location that you can protect against it, not at your location at the end of a fiber connection provided by an ISP that doesn't provide any sort of ddos mitigation services. And from the sounds of it - not even a firewall??
This is the scary part
"maybe some way to plug that main media converter Ethernet wire into firewall, but then what will be its wan ip? so confusing!"How is this guy running a company based upon providing services connected to the internet?? I just don't get it…
Its a car tracking company, the data for cars comes in all the time, so yeah its a service connected to internet.
I read it somewhere but i knew software firewalls can't do it so i just wanted to clear it myself asking here to prove everyone whose saying they can stop UDP or Amp attacks with pfSense alone, i knew its impossible but i never saw someone denying it as in my knowledge and as in your knowledge. So things are clear now :)
And you're right some ISPs just suck and don't care to provide security to their end users, renting 6 dedicated servers were being a bit expensive for me so I didn't go online, but i guess I'll have no other choice in future if this continues, many online hosters will atleast provide you ddos solutions.
-
Running a "very sensitive business" from 'home' ??
I don't know what 'sensitive' is, but I would run any serious (critical) business from a serious server, placed on a 'serious' spot, like a good data center.
If you use a good host, think about putting another serious 'tool' in front of it, like CloudFare (just to name one).I know my 'hosting company' eats 500 Gbits DDOS like cake so I never needed 'ClouldFare', or comparable, services.
Putting yourself behind ONE incoming without protection upfront just offers you one solution : they null-route you to protect their own (== ISP) network.Did I say I'm running it from home? Sorry if i sounded that way but .. I have a business place with my own dedicated servers, I currently have 6 servers running, renting them is a lot expensive then the price i got them here (but i think i will switch to online if this continues as most data centers provide ddos solutions), and you're right ISPs mostly just null routes you thats the sad part.
-
-
@BlueKobold:
Its a 10gb attack i cant get that big bandwidth here. and as your question my server does not respond to those, just drops them
And if you will get 10 GBit/s at the WAN and they attack you with 300 GBit/s you will loose again!
is there any way to block the attacks before it comes to my network without filling it?
Your ISP or your hoster would be setting up a device or service in front of your IP address.
in my case i have a fiber line connected through media converter and an Ethernet wire from media converter goes to switch from where all the servers get their public static ip,
Without SPI/NAT or Firewall and rules you are attaching servers to the Internet???
maybe some way to plug that main media converter Ethernet wire into firewall,
Would be a more secure solution as before you goes.
but then what will be its wan ip? so confusing!
The one you enter in the WAN menu.
There must be a way though, (ISP don't give a damn, all they do it null route my ip)
Perhaps he can´t do anything? There are some devices that can be placed in front of your business
Internet connection but they are often very expensive and there are also some services that can be
hired or rent to take the DDoS load from the line but also mostly very expensive.The Corero IPS 5500 ES-Series would be one of this devices you could try to place in front of your
firewall and then you would be back in game. Corero SmartWallCorero is using hardware from Tilera, based on so called many Core CPUs and this is purely not cheap.
Thanks for the detailed explanation! One question though that how does Hardware limit the rate? if the ddos is landing to my Corero SmartWall or whatever i device i use, won't it be the same as it landing on the firewall machine, because once it lands to me my bandwidth will be filled up again.
-
@BlueKobold:
inbound traffic is related to traffic that originated behind the NAT so NAT before state checking is needed.
And this is in my poor opinion the exactly point which is totally different each from other!
- The home or consumer grade SPI/NAT is doing something like the following:
Deny all and then have a look in the connection table for an open connection from inside
So it is wanted that all packets are staying outside. - But the SPI/NAT way from the pfSense is doing it in the total turned around direction as I see
it right, please correct me if I am wrong with this!
Let them (TCP/IP packets) all in for inspect them by one or more rules
So the many packets from an attack are able to get in and render or filling the pipe and nothing more goes.
Frank, interesting thoughts. I don't know if I'm wrong or you are or we both are, but this is becoming interesting.
So we have traffic from LAN side a.b.c.d:123456 destined for 1.2.3.4:80, with NAT enabled WAN is J.K.L.M, so NAT rewrites it to be sourced from J.K.L.M:987653. The return traffic is from 1.2.3.4, to J.K.L.M. Does pfSense translate/rewrite the packet to be to a.b.c.d:123456 and then look in the firewall rules? I'm not 100% sure, but the documentation I've read at least implies that.
Take a Linksys 54G doing similar function of NAT with stateful firewall. Does the return traffic get rewritten and then firewall rules applied? I don't know, but I think it should. The firewall state table should have the outbound packet with LAN address/port, NAT does the rewrite before it leaves on the WAN interface. To me that means the return traffic must be "de-NATted" before you look at the firewall state tables.
Any return traffic would have destaddr in the packet to be the WAN interface; NAT would have a lookup of WAN/port matching LAN/port2 in the table.
I don't think the simple lookup should be an issue, even at high inbound PPS. I think what becomes more of an issue is what happens when you get a match; you need to rewrite pieces of the packet (dest MAC, dest IP, dest port, one or two checksums) before passing the packet on. That takes resources and time. If checksumming is offloaded, then there is the potential for a context switch to get the modified packet back into the stack.I guess it's time to start sticking my nose into pf implementation on FreeBSD.
mike
- The home or consumer grade SPI/NAT is doing something like the following:
-
One question though that how does Hardware limit the rate?
That is what i have found on the Internet over those game play, but I really
know that absolutely no one will talk about the really work flow, that is called
security by obscurity I think. (Pictures: DDoS attacks Layer of defense & DME/Multi Core CPU/SME)if the ddos is landing to my Corero SmartWall or whatever i device i use, won't it be the same as it landing on the firewall machine, because once it lands to me my bandwidth will be filled up again.
At first and owed to the posts made by @johnpoz I was also thinking but then I found more and
more network draws that would be telling other things, where such a device should be or must
be placed. And this is not even the same point if we are talking about some or more different acting
companies, likes ISPs, Data Centers or Webhosters. (Picture: Coreros ReputationWatch)because once it lands to me my bandwidth will be filled up again.
The "device" is placed in front of the firewall and sort all bad things out and let only
the clean traffic passing through to the firewall then.
(Picture: DDoS attacks layer of defense & First line of defense)
-
Hello Mike,
I am not a professional or something like this, but I am really interested in the theme and perhaps
the end of this story or a solution that can be realized.I don't know if I'm wrong or you are or we both are, but this is becoming interesting.
For sure this could all be, I am not a hardware engineer, code writer, pfSense core development
member, pfSense expert or guru or a forum administrator, I am only interested in this theme and pfSense
more or less. And if a full featured software based firewall such as pfSense is would not be albe to
handle an attacks like this, but an ordinary lazy plastic home router for ~$100 is able to do, it
becomes even more and more interesting for me, sorry to tell this so plain and naive, but it
is like it is! Or come closer to this point, to find out what could be making the difference
between them would be really interesting for me.I don't think the simple lookup should be an issue, even at high inbound PPS.
And here the false is six feed under, with an lazy, tiny or very cheap ASIC/FPGA at this point
it could be running likes hell, to sort this packets out, and for sure pfSense is not needing of
this if we have a closer look at the most hardware we are talking here in the forum or the pfSense
store is offering now. There are worlds between them (home router & SG-xxx units).And I really think the NAT mechanism is more less then a difficult or tricky way.
Client A is opening behind the NAT a web page this data would be pulled
from the outside and to the Client behind the NAT and will be forwarded, all other
coming from outside will be dropped. Something really tiny and lazy it must be in
my eyes. And for sure I know that I am jumping now in an open shark mouth but
could it be that this version of doing NAT will be able to find its way inside of the
code from FreeBSD or pfSense only perhaps? Please remember I am no code writer
and developer, I don´t know anything about this and what other code or functions
on top will be affected by setting this version of up and for sure not as a replacement
for the actual NAT version or doing!!! Only perhaps as a so called drop down menu
variant where the users or customers are able to chose what kind of NAT version
they want to use, if this could be done. I really know some peoples they are aware
from this and don´t want this really since years, and for sure they are all knowing
why and why not, not likes me as a noob and beginner, but perhaps this is making
the difference in thinking of those cases.Because in my opinion, after this SPI and NAT process the firewall rules must also
only inspecting then the passing NAT traffic and not all packets that are arriving,
and for sure also the snort or suricata rules.I really don´t know if I am now misleading others or running in a so called hamster
wheel or that I am a prisoner of my own mind, I am only interested in to understand
this point, why a server grade hardware based firewall is not able and a lazy
~$100 home is able to do so. -
Thanks for the detailed explanation! One question though that how does Hardware limit the rate? if the ddos is landing to my Corero SmartWall or whatever i device i use, won't it be the same as it landing on the firewall machine, because once it lands to me my bandwidth will be filled up again.
It's not possible to do anything on your end of the line to stop the typical UDP flood DDoS, because those are bandwidth exhaustion attacks (usually DNS or NTP amplification). It's too late by the time it gets to you, you can't change the fact your connection is flooded. Your ISP has to stop it before it reaches your connection.
Where boxes like that can be useful are attacks like large scale SYN floods that go beyond what any firewall can handle in new connections/sec, but aren't so large that they completely fill your Internet connection.
@BlueKobold:
And if a full featured software based firewall such as pfSense is would not be albe to
handle an attacks like this, but an ordinary lazy plastic home router for ~$100 is able to doThere is no circumstance in which a consumer grade router is better at handling DDoS. Consumer grade devices are extremely poorly suited for resource exhaustion attacks.
-
why a server grade hardware based firewall is not able and a lazy
~$100 home is able to do so.I think you will find that many consumer grade devices don't even have anything close to what you would call a firewall. All of my original devices were not.
It's the very reason I looked up Monowall then soon after found pfSense. You got one thing right- lazy!
-
You cannot use pfsense for DDoS protection.
You can still flood it with sub 10Mbit/s traffic and it dies.
Tested on 2.2.4 AMD64.
-
And here we go again, exactly as predicted… Yay.
-
I wonder how efficient NAT is. NAT could be quite efficient if integrated into the states.
Ingress
WAN - SPI - NAT - Core logic - LAN
Egress
WAN - Core Logic - NAT - SPI - LANAssuming this modular setup, which is great of proper layering, but depending on how transparent the layering, the NAT may need to do its own lookup after the SPI has already done a lookup.
If the NAT was integrated into the SPI
Ingress
WAN - SPI/NAT - Core logic - LAN
Egress
WAN - Core Logic - SPI/NAT - LANNow you only have one lookup and all of the NAT state is stored along with the firewall state. Assuming it isn't already similar to this.
I also read that the NAT is single threaded, but which reduces the usefulness of the firewall being threaded because traffic from the firewall gets shoved through the NAT anyway, probably making things worse than just single theaded. Again, I make a lot of assumption, many of which are probably wrong because PFSense seems to have great performance as long as I don't get a lot of new states being created.
Either way, I can't wait for the line-rate stuff. There's going to be so much change that it's not very useful discussing the current system. Worry about performance tuning after 3.0 :p
-
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
-
Inbound traffic on WAN: for it to pass, it needs to be IP of the pfSense WAN port (or WAN broadcast) and the state table needs to have an entry with the source port matching the dest port of the inbound traffic, no? Lower layers of the stack should be dropping packets not destined for the WAN (assuming not promsicuous mode), so would a first level check of "does this dest port match any source port in the state table" work? I think the big writer of a state table is the outbound path, updates on a single entry on the inbound (I'm assuming very simple case here of no open ports on WAN), so if the state table were also indexed by source port it may be possible to do the lookup (read) lockless. If no match on ports, don't bother doing anything else, just drop the packet. If there's a port match, then pass it into the NAT/redir logic and onward.
Of course a lot of this depends on the definition of "handling" the line rate traffic. Simply not crashing the box (implies all you are doing is dropping packets) or actually getting legitimate work done?
Disclaimer: the code may already be doing this, I don't know. Just random thoughts on what actually happens in IPV4 NAT.
-
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
Line rate talk was packets per second, but did not mention the rate of new connections. Every packet already needs to look up in the state table, so reading from the state table will handle millions of packets per second. The next question is how quickly can you write to the state table.
Even if the state table fills up quickly, you can continue reading from it just fine. In theory, existing connections should continue to work uninhibited.
Just a random idea. They could reserve N number of states in the state table for connection going out. This way if someone SYN floods you on your WAN, you can still make some connections out even if incoming WAN connections in are rejected because of a full table.
-
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
Line rate talk was packets per second, but did not mention the rate of new connections. Every packet already needs to look up in the state table, so reading from the state table will handle millions of packets per second. The next question is how quickly can you write to the state table.
Even if the state table fills up quickly, you can continue reading from it just fine. In theory, existing connections should continue to work uninhibited.
Just a random idea. They could reserve N number of states in the state table for connection going out. This way if someone SYN floods you on your WAN, you can still make some connections out even if incoming WAN connections in are rejected because of a full table.
Hmm. Watermarking basically. "I can handle X conn/sec max, start doing RED at %y(X) so I have bandwidth for legitimate traffic". Interesting.
That ties into "handling" the load; be able to drop everything without crashing or actually do useful work.
-
@mer:
… can't wait for the line-rate stuff.
I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?
Line rate talk was packets per second, but did not mention the rate of new connections. Every packet already needs to look up in the state table, so reading from the state table will handle millions of packets per second. The next question is how quickly can you write to the state table.
Even if the state table fills up quickly, you can continue reading from it just fine. In theory, existing connections should continue to work uninhibited.
Just a random idea. They could reserve N number of states in the state table for connection going out. This way if someone SYN floods you on your WAN, you can still make some connections out even if incoming WAN connections in are rejected because of a full table.
Hmm. Watermarking basically. "I can handle X conn/sec max, start doing RED at %y(X) so I have bandwidth for legitimate traffic". Interesting.
That ties into "handling" the load; be able to drop everything without crashing or actually do useful work.
Not drop packets, drop new connection attempts, or do something with them to not allow new connections eat up all of the CPU.
-
Alright, my first post and well it will be in DDOS because this shit is eating my brains out. Simple thing and remember this folks pFsense is for home user grade not business like providing filtering methods for this type of attack.
Something that can take a DDOS attack and let's say not dying every time… a test setup will go with something like this:
FreeBSD with kernel optimization and NETMAP some intel 540 adaptor(I don't like them) and a sensor that will detect the attack and apply rules to your little box. Intel know about the ntuple but is limited for filtering like this, proto,ports,IP nothing more than this so hmm we can kick out some DNS,NTP amplification, and it will do it at 10G line rate but that's all. Now for the next part of my funny lab.
Anyway now I'm playing with Some 2xE5-1620 netmap and Chelsio T5, and boy I tell you this card is nice. What it can do more than those Intel well same thing but It does have ASIC hmm here is the pretty part where you can get and some filtering, you can program those ASIC to take the real beating, and over that you can redirect on fly some part's of the traffic to you machine where you can filter with ipfw and from here the game begins.
Those big solution that cost over 200k this is the method that use, and of course a big army of devs and HW that will make you drool like a fool. In near future even those solution won't help anymore because the problem begins with the ISP that allow this type of traffic to go from their network.
Some resolution would be if the traffic can be cut from the roots, but our big carrier like Telia and NTT they like you to buy more and more and more 10-40-100G. This are my 2 cent. -
@Mup:
Simple thing and remember this folks pFsense is for home user grade not business like providing filtering methods for this type of attack.
That's bullshit unless you're also going to say a Cisco ASA is for home user grade not business. The hardware appliances we sell stand up to DDoS as well as ASAs and comparable commercial firewalls costing several times as much, in some cases over 10 times as much money for the same performance in new connections/sec handling. You don't put a firewall in front of things that are subject to resource exhaustion attacks.
@Mup:
Anyway now I'm playing with Some 2xE5-1620 netmap and Chelsio T5, and boy I tell you this card is nice. What it can do more than those Intel well same thing but It does have ASIC hmm here is the pretty part where you can get and some filtering, you can program those ASIC to take the real beating, and over that you can redirect on fly some part's of the traffic to you machine where you can filter with ipfw and from here the game begins.
There are some interesting possibilities with the hardware filtering there for sure, let's just say we're aware. ;) That's the type of thing that can stand up to extreme abuse.
