UDP DDoS protection with pfSense

Aug 5, 2015, 8:00 AM

(And someone kindly lock this, the previous 50+ pages shit was just enough.)

On one side I am with you, that a closed and locked thread should be not warmed up or onyl
tiny pushed to another on and then goes on without taking advantage from the admin´s advice.

But mostly this also is owed to the owed kind and taken manner a thread goes or will be going.
Also the style and way the thread is lead will be a respectful point to watch out.

@NOYB
I consider but then also not falling back to the way the last thread about this theme was running.

@Harvy66
If the big players in this game are using extra or special hardware, mostly or often based on the
Tilera many core cpu´s (Tile-GX) why thinking it can be done in other cases with software only?
Lanner is offering a bigger appliance like the FW-889x and a NCS-MTX401 add in card and on
this card it can be installed and running a SMP Linux that is able to offload 20 GBit/s - 40 GBit/s
packet processing related to the kind of work for sure, likes DPI, IDS/IPS, VPN crypto stuff.

So FreeBSD and/or pfSense are nor really involved in this game and can be easily native installed on those
machines or as one or more in a VM on a host like this, but owed to the fiber bypass mode, it is able to
sort out many traffic likes a synflood or DoS/DDoS attack. And yes for sure there a many other PCIe cards
out from Tilera that can be installed in ordinary existing server running pfSense.

But the real clou will be, that we are able to pay for such cards, but not really for the devices named some
posts above from me. It can be a real show-stopper to the bigger sold devices because FreeBSD must not
be touched really, the working SMP Linux is installed and homed on the cards NAND flash memory.

And the C2758, XG-1540 or following appliances would bea ble to hold one PCIe card as I see it right.
So I thing DDoS atacks could be also mitigated from 1 GBit/s to xx GBit/s.
Tilera EZ cards

Harvy66 · Aug 5, 2015, 5:57 PM

An overly simplified say to look at it is how many cycles per packet are spent. If I have a 3ghz quad core cpu, that's 12ghz of peak processing power. If you assume a large 1,000 cycles per packet (proof of concept can get it as low as 100 cycles per packet), that leaves you with 12M-pps.

Ideally with breathing room, my quad core should be able to handle near line rate of 10Gb/s 64byte packets(half-duplex). Of course life isn't this simple. We have context switches, data bouncing between cores, complex routing, a bunch of firewall rules a user made, and a host of other reasons that need to be ironed out.

When I saw my computer crapping out with 30K-pps, that places the computational load around 400K cycles per packet, or 400x worse than my 10x above simple real world placing the current system somewhere between 400x and 4000x slower than it could be. That's a lot of room for optimizations.

All I'm saying, don't say it can't be done, it just requires a lot of the work that is already being talked about. The netmap people showed a single core 900mhz CPU doing line rate 10Gb/s with a very simple single entry route and no firewall, and that was being handled in userland, not the kernel, so it could be even faster. I can't wait for 3 years from now, I expect FreeBSD to be in a very good place with network performance.

NOYB · Aug 5, 2015, 6:07 PM

I always thought it backwards from a performance perspective for the firewall to be post NAT. burning cycles NATing a bunch of traffic that is just going to be blocked anyway seems to be inefficient.

mer · Aug 6, 2015, 10:34 AM

@NOYB:

I always thought it backwards from a performance perspective for the firewall to be post NAT. burning cycles NATing a bunch of traffic that is just going to be blocked anyway seems to be inefficient.

That is an interesting POV. (thinking out loud here) Perhaps it depends on "what NAT" is involved. Inbound NAT/redirect; yes that makes a lot of sense to look at firewall first, but if the inbound traffic doesn't match any redirect/NAT rules then it doesn't really get NATted, does it? Responses to outbound traffic that was NATted should be a lookup and simple state match, no? Maybe firewall block rules, on the external interface, based on source information or dest port could actually be done prior to NAT. Like if you are not running a webserver, "block in on $ext_if dest port 80" run before any NAT or redir would make sense.

In a "typical" NAT environment (most home users, maybe SOHO use), inbound traffic is related to traffic that originated behind the NAT so NAT before state checking is needed.

Maybe we also need to think about what happens when a packet is NATted or redirected too. How much of the packet gets rewritten, what checksums need to get updated, is the checksum offloaded?

Aug 6, 2015, 11:56 AM

inbound traffic is related to traffic that originated behind the NAT so NAT before state checking is needed.

Thats the point I want to come closer to. If those traffic is generated to the outside, likes open a webpage,
the connection is placed in a connection table, if now the TCP/IP packets are coming back and wan to be
forwarded to the PC or machine hwo was calling for, the NAT process, must have a look in this connection
table if there is an opened connection entry in this table and then it will be forwarded to the PC or will be
dropped. OK for sure this can also be done native by the pfSense without such a card for sure, but if there
is then one or more opened ports, for the servers in the DMZ, it perhaps comes to the point where the pipe
gets rendered and only for this those cards I thought would be fine to do the job, proofing and dropping or
forward them.

And this is in my poor opinion the exactly point which is totally different each from other!

The home or consumer grade SPI/NAT is doing something like the following:
Deny all and then have a look in the connection table for an open connection from inside
So it is wanted that all packets are staying outside.
But the SPI/NAT way from the pfSense is doing it in the total turned around direction as I see
it right, please correct me if I am wrong with this!
Let them (TCP/IP packets) all in for inspect them by one or more rules
So the many packets from an attack are able to get in and render or filling the pipe and nothing more goes.

bilal91 · Aug 6, 2015, 7:46 PM

@NOYB:

@bilal91:

I have a very sensitive business which needs 100% up time,

Then, as mentioned by others, you probably need to hire a service to filter your traffic before it comes down the pipe from ISP to you. Or if the ISP has the capability, get them to filter your traffic instead of just null routing.

I'm curious. Do you have any inclination at all of who or the motive that is behind the attack? Competitor, someone doesn't like you, disgruntled customer or employee, extortion, etc.?

Yup its a possible competitor that's for sure :)

bilal91 · Aug 6, 2015, 8:08 PM

@johnpoz:

What I would like to know is what the OP was reading that pointed him to pfsense mitigating attacks?

"i found online to go with pfSense, i saw many people mitigating attacks with it too"

There are lots of threads here asking the same thing - and they always get the same answer, you can not stop a DDOS with a firewall.. So either he was not reading the full thread/article or misread the information?

If the OP business is so critical and of nature that ddos is of concern, they need to host services out of location that you can protect against it, not at your location at the end of a fiber connection provided by an ISP that doesn't provide any sort of ddos mitigation services. And from the sounds of it - not even a firewall??

This is the scary part
"maybe some way to plug that main media converter Ethernet wire into firewall, but then what will be its wan ip? so confusing!"

How is this guy running a company based upon providing services connected to the internet?? I just don't get it…

Its a car tracking company, the data for cars comes in all the time, so yeah its a service connected to internet.

I read it somewhere but i knew software firewalls can't do it so i just wanted to clear it myself asking here to prove everyone whose saying they can stop UDP or Amp attacks with pfSense alone, i knew its impossible but i never saw someone denying it as in my knowledge and as in your knowledge. So things are clear now :)

And you're right some ISPs just suck and don't care to provide security to their end users, renting 6 dedicated servers were being a bit expensive for me so I didn't go online, but i guess I'll have no other choice in future if this continues, many online hosters will atleast provide you ddos solutions.

bilal91 · Aug 6, 2015, 8:00 PM

@Gertjan:

Running a "very sensitive business" from 'home' ??
I don't know what 'sensitive' is, but I would run any serious (critical) business from a serious server, placed on a 'serious' spot, like a good data center.
If you use a good host, think about putting another serious 'tool' in front of it, like CloudFare (just to name one).

I know my 'hosting company' eats 500 Gbits DDOS like cake so I never needed 'ClouldFare', or comparable, services.
Putting yourself behind ONE incoming without protection upfront just offers you one solution : they null-route you to protect their own (== ISP) network.

Did I say I'm running it from home? Sorry if i sounded that way but .. I have a business place with my own dedicated servers, I currently have 6 servers running, renting them is a lot expensive then the price i got them here (but i think i will switch to online if this continues as most data centers provide ddos solutions), and you're right ISPs mostly just null routes you thats the sad part.

bilal91 · Aug 6, 2015, 8:01 PM

@NOYB:

@Gertjan:

Running a "very sensitive business" from 'home' ??

Didn't see where the OPer said anything about running business from home. Did I miss that?

You're right I never said that! lol

bilal91 · Aug 6, 2015, 8:21 PM

@BlueKobold:

Its a 10gb attack i cant get that big bandwidth here. and as your question my server does not respond to those, just drops them

And if you will get 10 GBit/s at the WAN and they attack you with 300 GBit/s you will loose again!

is there any way to block the attacks before it comes to my network without filling it?

Your ISP or your hoster would be setting up a device or service in front of your IP address.

in my case i have a fiber line connected through media converter and an Ethernet wire from media converter goes to switch from where all the servers get their public static ip,

Without SPI/NAT or Firewall and rules you are attaching servers to the Internet???

maybe some way to plug that main media converter Ethernet wire into firewall,

Would be a more secure solution as before you goes.

but then what will be its wan ip? so confusing!

The one you enter in the WAN menu.

There must be a way though, (ISP don't give a damn, all they do it null route my ip)

Perhaps he can´t do anything? There are some devices that can be placed in front of your business
Internet connection but they are often very expensive and there are also some services that can be
hired or rent to take the DDoS load from the line but also mostly very expensive.

The Corero IPS 5500 ES-Series would be one of this devices you could try to place in front of your
firewall and then you would be back in game. Corero SmartWall

Corero is using hardware from Tilera, based on so called many Core CPUs and this is purely not cheap.

Thanks for the detailed explanation! One question though that how does Hardware limit the rate? if the ddos is landing to my Corero SmartWall or whatever i device i use, won't it be the same as it landing on the firewall machine, because once it lands to me my bandwidth will be filled up again.

mer · Aug 6, 2015, 9:42 PM

@BlueKobold:

inbound traffic is related to traffic that originated behind the NAT so NAT before state checking is needed.

And this is in my poor opinion the exactly point which is totally different each from other!

The home or consumer grade SPI/NAT is doing something like the following:
Deny all and then have a look in the connection table for an open connection from inside
So it is wanted that all packets are staying outside.

But the SPI/NAT way from the pfSense is doing it in the total turned around direction as I see
it right, please correct me if I am wrong with this!
Let them (TCP/IP packets) all in for inspect them by one or more rules
So the many packets from an attack are able to get in and render or filling the pipe and nothing more goes.

Frank, interesting thoughts. I don't know if I'm wrong or you are or we both are, but this is becoming interesting.

So we have traffic from LAN side a.b.c.d:123456 destined for 1.2.3.4:80, with NAT enabled WAN is J.K.L.M, so NAT rewrites it to be sourced from J.K.L.M:987653. The return traffic is from 1.2.3.4, to J.K.L.M. Does pfSense translate/rewrite the packet to be to a.b.c.d:123456 and then look in the firewall rules? I'm not 100% sure, but the documentation I've read at least implies that.

Take a Linksys 54G doing similar function of NAT with stateful firewall. Does the return traffic get rewritten and then firewall rules applied? I don't know, but I think it should. The firewall state table should have the outbound packet with LAN address/port, NAT does the rewrite before it leaves on the WAN interface. To me that means the return traffic must be "de-NATted" before you look at the firewall state tables.

Any return traffic would have destaddr in the packet to be the WAN interface; NAT would have a lookup of WAN/port matching LAN/port2 in the table.
I don't think the simple lookup should be an issue, even at high inbound PPS. I think what becomes more of an issue is what happens when you get a match; you need to rewrite pieces of the packet (dest MAC, dest IP, dest port, one or two checksums) before passing the packet on. That takes resources and time. If checksumming is offloaded, then there is the potential for a context switch to get the modified packet back into the stack.

I guess it's time to start sticking my nose into pf implementation on FreeBSD.

mike

Aug 7, 2015, 9:56 AM

One question though that how does Hardware limit the rate?

That is what i have found on the Internet over those game play, but I really
know that absolutely no one will talk about the really work flow, that is called
security by obscurity I think. (Pictures: DDoS attacks Layer of defense & DME/Multi Core CPU/SME)

if the ddos is landing to my Corero SmartWall or whatever i device i use, won't it be the same as it landing on the firewall machine, because once it lands to me my bandwidth will be filled up again.

At first and owed to the posts made by @johnpoz I was also thinking but then I found more and
more network draws that would be telling other things, where such a device should be or must
be placed. And this is not even the same point if we are talking about some or more different acting
companies, likes ISPs, Data Centers or Webhosters. (Picture: Coreros ReputationWatch)

because once it lands to me my bandwidth will be filled up again.

The "device" is placed in front of the firewall and sort all bad things out and let only
the clean traffic passing through to the firewall then.
(Picture: DDoS attacks layer of defense & First line of defense)

Aug 7, 2015, 10:47 AM

Hello Mike,

I am not a professional or something like this, but I am really interested in the theme and perhaps
the end of this story or a solution that can be realized.

I don't know if I'm wrong or you are or we both are, but this is becoming interesting.

For sure this could all be, I am not a hardware engineer, code writer, pfSense core development
member, pfSense expert or guru or a forum administrator, I am only interested in this theme and pfSense
more or less. And if a full featured software based firewall such as pfSense is would not be albe to
handle an attacks like this, but an ordinary lazy plastic home router for ~$100 is able to do, it
becomes even more and more interesting for me, sorry to tell this so plain and naive, but it
is like it is! Or come closer to this point, to find out what could be making the difference
between them would be really interesting for me.

I don't think the simple lookup should be an issue, even at high inbound PPS.

And here the false is six feed under, with an lazy, tiny or very cheap ASIC/FPGA at this point
it could be running likes hell, to sort this packets out, and for sure pfSense is not needing of
this if we have a closer look at the most hardware we are talking here in the forum or the pfSense
store is offering now. There are worlds between them (home router & SG-xxx units).

And I really think the NAT mechanism is more less then a difficult or tricky way.
Client A is opening behind the NAT a web page this data would be pulled
from the outside and to the Client behind the NAT and will be forwarded, all other
coming from outside will be dropped. Something really tiny and lazy it must be in
my eyes. And for sure I know that I am jumping now in an open shark mouth but
could it be that this version of doing NAT will be able to find its way inside of the
code from FreeBSD or pfSense only perhaps? Please remember I am no code writer
and developer, I don´t know anything about this and what other code or functions
on top will be affected by setting this version of up and for sure not as a replacement
for the actual NAT version or doing!!! Only perhaps as a so called drop down menu
variant where the users or customers are able to chose what kind of NAT version
they want to use, if this could be done. I really know some peoples they are aware
from this and don´t want this really since years, and for sure they are all knowing
why and why not, not likes me as a noob and beginner, but perhaps this is making
the difference in thinking of those cases.

Because in my opinion, after this SPI and NAT process the firewall rules must also
only inspecting then the passing NAT traffic and not all packets that are arriving,
and for sure also the snort or suricata rules.

I really don´t know if I am now misleading others or running in a so called hamster
wheel or that I am a prisoner of my own mind, I am only interested in to understand
this point, why a server grade hardware based firewall is not able and a lazy
~$100 home is able to do so.

cmb · Aug 7, 2015, 11:48 PM

@bilal91:

Thanks for the detailed explanation! One question though that how does Hardware limit the rate? if the ddos is landing to my Corero SmartWall or whatever i device i use, won't it be the same as it landing on the firewall machine, because once it lands to me my bandwidth will be filled up again.

It's not possible to do anything on your end of the line to stop the typical UDP flood DDoS, because those are bandwidth exhaustion attacks (usually DNS or NTP amplification). It's too late by the time it gets to you, you can't change the fact your connection is flooded. Your ISP has to stop it before it reaches your connection.

Where boxes like that can be useful are attacks like large scale SYN floods that go beyond what any firewall can handle in new connections/sec, but aren't so large that they completely fill your Internet connection.

@BlueKobold:

And if a full featured software based firewall such as pfSense is would not be albe to
handle an attacks like this, but an ordinary lazy plastic home router for ~$100 is able to do

There is no circumstance in which a consumer grade router is better at handling DDoS. Consumer grade devices are extremely poorly suited for resource exhaustion attacks.

chpalmer · Aug 8, 2015, 12:00 AM

why a server grade hardware based firewall is not able and a lazy
~$100 home is able to do so.

I think you will find that many consumer grade devices don't even have anything close to what you would call a firewall. All of my original devices were not.

It's the very reason I looked up Monowall then soon after found pfSense. You got one thing right- lazy!

Supermule · Aug 9, 2015, 11:16 AM

You cannot use pfsense for DDoS protection.

You can still flood it with sub 10Mbit/s traffic and it dies.

Tested on 2.2.4 AMD64.

doktornotor · Aug 9, 2015, 12:08 PM

And here we go again, exactly as predicted… Yay.

Harvy66 · Aug 29, 2015, 5:43 PM

I wonder how efficient NAT is. NAT could be quite efficient if integrated into the states.

Ingress
WAN - SPI - NAT - Core logic - LAN
Egress
WAN - Core Logic - NAT - SPI - LAN

Assuming this modular setup, which is great of proper layering, but depending on how transparent the layering, the NAT may need to do its own lookup after the SPI has already done a lookup.

If the NAT was integrated into the SPI

Ingress
WAN - SPI/NAT - Core logic - LAN
Egress
WAN - Core Logic - SPI/NAT - LAN

Now you only have one lookup and all of the NAT state is stored along with the firewall state. Assuming it isn't already similar to this.

I also read that the NAT is single threaded, but which reduces the usefulness of the firewall being threaded because traffic from the firewall gets shoved through the NAT anyway, probably making things worse than just single theaded. Again, I make a lot of assumption, many of which are probably wrong because PFSense seems to have great performance as long as I don't get a lot of new states being created.

Either way, I can't wait for the line-rate stuff. There's going to be so much change that it's not very useful discussing the current system. Worry about performance tuning after 3.0 :p

NOYB · Aug 30, 2015, 4:12 AM

@Harvy66:

… can't wait for the line-rate stuff.

I'm going to make an assumption here too. My assumption is that the line rate talk is bits throughput, not new connections. What's the max syn packets at gigabit line rate? Couple million per second? Do you think they can process that on typical common affordable to the masses hardware?

mer · Sep 2, 2015, 8:29 AM

Inbound traffic on WAN: for it to pass, it needs to be IP of the pfSense WAN port (or WAN broadcast) and the state table needs to have an entry with the source port matching the dest port of the inbound traffic, no? Lower layers of the stack should be dropping packets not destined for the WAN (assuming not promsicuous mode), so would a first level check of "does this dest port match any source port in the state table" work? I think the big writer of a state table is the outbound path, updates on a single entry on the inbound (I'm assuming very simple case here of no open ports on WAN), so if the state table were also indexed by source port it may be possible to do the lookup (read) lockless. If no match on ports, don't bother doing anything else, just drop the packet. If there's a port match, then pass it into the NAT/redir logic and onward.

Of course a lot of this depends on the definition of "handling" the line rate traffic. Simply not crashing the box (implies all you are doing is dropping packets) or actually getting legitimate work done?

Disclaimer: the code may already be doing this, I don't know. Just random thoughts on what actually happens in IPV4 NAT.