What is the biggest attack in GBPS you stopped
-
Two things: increase your state table to 8M states. The UI and the rest of the box will be fine.
Doesn't that require about 8GB of RAM? Who has that in their firewall?
-
It actually takes 16GB to do that if all 8M is used.
-
Who has that in their firewall?
RAM is cheap at this times 4 x 8 GB of ECC RAM is really payable for the masses, and why not?
-
@KOM:
Two things: increase your state table to 8M states. The UI and the rest of the box will be fine.
Doesn't that require about 8GB of RAM? Who has that in their firewall?
I have 4GB of RAM and it worked without a hitch. Memory utilization was actually very low, <50% I believe. The raw data is buried somewhere in this thread.
I think, but cannot recall specifically, that it only hit about 4.7M states at the top end before the IRQ interrupt storm warning started. That's also why I think the state table never filled up.
-
So you are not sure??
Just because I pissed you off…
I thought you were older than the "name calling" age.
Tell me what I used... And since you have offered little to no help at all in configuring these "sub optimal" configurations, its very clear to everybody that nothing good comes from ESF other than bickering and "sub optimal" communication.
If I really DDoS'ed you then you wouldnt be online yet.
@cmb:
@KOM:
So we can't expect the pfSense team to fix a problem that's not in pfSense, and they are at the mercy of the FreeBSD code releases.
While we can't expect them to, they certainly have fixed some FreeBSD bugs and submitted the patches upstream, which were accepted for inclusion by the FreeBSD team.
Yup. And if there were truly a world-is-ending issue here, we'd already have done the same already.
All firewalls require some degree of tuning to stand up to resource exhaustion attacks. Settings appropriate for such circumstances aren't defaults because they would break many more circumstances than they would help, as they're too aggressive for many VoIP configurations, among other possible problems. Also general default behaviors that most people want, like logging all blocked traffic, are really terrible in that circumstance. Among a variety of other problems I've pointed out with the scenarios in these threads.
I've seen enough of Supermule's super-secret DDoS stuff from when he DDoSed our forum several times (not going to believe it wasn't him), and some things I've gathered from others, to be able to put together a proper analysis to get upstream at some point.
It's all doable with hping or other tools. Supermule uses shady illegal services you pay for in Bitcoin that use criminal-run botnets. The same circumstance can be lab replicated without paying criminals.
Investigating this further is still on my radar. There have just been real problems that affect many reasonable real world use cases that have taken precedence. Anyone trying to stop DDoS with a firewall of any type is doing it wrong.
Brian, this is unacceptable.
Final warning. Be nice, or you are banned from this forum.
-
I am nice.
But you would be pissed as well if you were accused of things you didnt do.
It goes both ways.
-
How much power does the cycling monkey have?
Can it beat Chris Froome?
Should I hire it to pedal on the back of my tandem?I'll admit my shell scripts can't (yet?) pedal a bike. :)
You hired a power-cycling monkey?
No, I wrote one. :D
Borrowing the chaos monkey name.
Just scripting SNMP sets against an IP PDU to effectively yank the power plug, power it back up, wait for it to reply to pings, rinse and repeat. Many thousands of times. One box close to 20,000 times now, one over 10,000 times, others well into the thousands.
-
Come on guys…
Clearly there is an issue and it
s not only caused by resource exaustion as cmb says. Mine went down with 3Mbps of traffic and I have 40/100 and 20/20 pipes. Everything went down and he didnt even touch my internal webserver but firewall itself.Why can
t you work together, Im really dissapointed in the way things went on this issue.pfSense is really cool and serious project and forums didn`t let me down since I first posted here (ok besides dok and his sarcasm which I got used to and it amuses me every time hehe ;) ) but I always got help or at least hint where to start.
Open source guys…
My 2c...
-
I am nice.
But you would be pissed as well if you were accused of things you didnt do.
It goes both ways.
You may have been falsely accused. How you react to this is your choice.
There is a difference that seems to need need stating:
Chris is a co-owner, as am I. You are a guest.
While you are helpful, and respectful, you are welcome here. When that is no longer true, I will (and take this with all the requisite gravitas), remove you from this community. You will not return. I removed my former persona ("gonzopancho") in a way that I could not ever recover, because I found that I could no longer respond in a reasonable manner.
You have been far over "the line" of reasonable response on many occasions. This is your final warning. How you react is your choice.
I think you have value that you can bring, but your behavior in this and other things is, in the balance, largely negative toward the project and its owners.
-
Come on guys…
Clearly there is an issue and it
s not only caused by resource exaustion as cmb says. Mine went down with 3Mbps of traffic and I have 40/100 and 20/20 pipes. Everything went down and he didnt even touch my internal webserver but firewall itself.Why can
t you work together, Im really dissapointed in the way things went on this issue.pfSense is really cool and serious project and forums didn`t let me down since I first posted here (ok besides dok and his sarcasm which I got used to and it amuses me every time hehe ;) ) but I always got help or at least hint where to start.
Open source guys…
My 2c...
The issue is resource exhaustion. The 'attack' (as it were) runs pf out of states. It is not (as some seem to want to state) related to virtualization or (as others seem to want to state) interrupt load on a single CPU.
To my knowledge, Brian has never shared his scripts/code, but we do have what I believe to be similar internally. We have shown (internally)
that the problem is endemic to the pf in FreeBSD. It is not specific to pfSense, or any 'forks' of same. I have not verified that the problem occurs on OpenBSD, or another 'stateful' firewall.The problem is not made worse by the lack of dtrace on the image.
Your disappointment is not inducement to work on the problem, nor am I aware of what you mean by "Open source guys…"
We, or someone else, will eventually fix the issue. It may be quite difficult. The pf codebase is not well-structured.
-
Thing is Chris… Franco is very kind to people.
I dont really dig into the opnsense/pfsense feud since its meaningless.
I see. So is your assertion is that Brian/Supermule on forum.opnsense.org is not the same as Brian/Supermule on forum.pfsense.org?
Because if these are the same, then you said something quite different only two weeks ago:
https://forum.opnsense.org/index.php?topic=581.msg2799#msg2799
I take several issues with what you said:
Issue IMHO is that pfSense is nothing more than a name and logo.
I assert that this is false.
All the code and contributions are open source
This is true, and always has been.
and they want to change that so they can make money of other peoples work and contributions.
We have not changed that pfSense is open source, nor do we "want to change that". Your ugly accusation is false, Brian.
I dont like it and thats why I am here.
This is, of course, your choice, but I don't see why you feel allowed to be two-faced about it without challenge.
-
Its not. Plenty of ressources left as you can see. Its routing and the way pf handles it.
The issue is different.
-
OK, so 45 forum pages later, we have determined that there is a problem in pf that may or may not get fixed by someone sometime, perhaps.
Can we close this thread now already?
-
Regardless of whether or not this low bandwidth DDOS traffic should be stopped upstream, 3 mbps of any kind of traffic should not render a modern business class firewall useless. Period.
The root issue being in FreeBSD pf doesn't changes that. And since pfSense is built on FreeBSD it is by extension a pfSense problem too.
What specific work has the pfSense project done with the FreeBSD project to resolve this specific issue and provide a fix?
-
Regardless of whether or not this low bandwidth DDOS traffic should be stopped upstream, 3 mbps of any kind of traffic should not render a modern business class firewall useless. Period.
The root issue being in FreeBSD pf doesn't changes that. And since pfSense is built on FreeBSD it is by extension a pfSense problem too.
What specific work has the pfSense project done with the FreeBSD project to resolve this specific issue and provide a fix?
I outlined what we've done above. We have, what I believe to be, a similar program which can cause a similar issue.
If we develop a fix, we will attempt to upstream it to FreeBSD.
Right now, nobody is actively working on this "issue", because it is, fundamentally, a misapplication of technology.
-
@KOM:
OK, so 45 forum pages later, we have determined that there is a problem in pf that may or may not get fixed by someone sometime, perhaps.
Can we close this thread now already?
+1
-
@jwt:
Regardless of whether or not this low bandwidth DDOS traffic should be stopped upstream, 3 mbps of any kind of traffic should not render a modern business class firewall useless. Period.
The root issue being in FreeBSD pf doesn't changes that. And since pfSense is built on FreeBSD it is by extension a pfSense problem too.
What specific work has the pfSense project done with the FreeBSD project to resolve this specific issue and provide a fix?
I outlined what we've done above. We have, what I believe to be, a similar program which can cause a similar issue.
If we develop a fix, we will attempt to upstream it to FreeBSD.
Right now, nobody is actively working on this "issue", because it is, fundamentally, a misapplication of technology.
Is there a bug report with freebsd so we can keep tabs on it?
-
Is there a bug report with freebsd so we can keep tabs on it?
Read this post: FreeBSD/pf and SYN ACK flooding. Then you'll understand why there's no upstream bug anywhere. (Unlike here, they've been able to stop the nonsense in just a single page, not 45 and counting… ::))
-
@jwt:
Regardless of whether or not this low bandwidth DDOS traffic should be stopped upstream, 3 mbps of any kind of traffic should not render a modern business class firewall useless. Period.
The root issue being in FreeBSD pf doesn't changes that. And since pfSense is built on FreeBSD it is by extension a pfSense problem too.
What specific work has the pfSense project done with the FreeBSD project to resolve this specific issue and provide a fix?
I outlined what we've done above. We have, what I believe to be, a similar program which can cause a similar issue.
If we develop a fix, we will attempt to upstream it to FreeBSD.
Right now, nobody is actively working on this "issue", because it is, fundamentally, a misapplication of technology.
So in other words the pfSense project has not done specific work with the FreeBSD project to resolve this specific issue and provide a fix.
Could you please explain the fundamental misapplication of technology?
Thanks
-
Is there a bug report with freebsd so we can keep tabs on it?
Read this post: FreeBSD/pf and SYN ACK flooding. Then you'll understand why there's no upstream bug anywhere. (Unlike here, they've been able to stop the nonsense in just a single page, not 45 and counting… ::))
https://en.wikipedia.org/wiki/SYN_flood
Can we enquire considering the botnet element what solution is being considered?
A few thoughts on this, is if a server/service is behind an ip address where more than 1 ip address is available at a moments notice but not all ip's used, could existing states be maintained, whilst DNS is updated and services moved to another ip address a bit like the multiwan setup where wan1 is incoming and outgoing goes out on Wan2 for example.
If a server/service regularly only handles traffic from a specific region or country, could something like pfblocker be made to block anything from outside that region/country on the fly? I suspect this is why google diverts searchs to their local country domains.
What are the pro's/con's from having a recycled log of good ip addresses that have previously connected in the last x minutes or by some other pattern, which are allowed in automatically whilst the unknown ip addresses perhaps have a shorter timeout or treated with some other caution. Its possible to know that some IP address ranges will have good speed and latency by virtue of who is allocated them, whilst others may not, thinking mobile phone/satellite for the latter, so could something adaptive be considered?
Where would I find the failed ACK's being reported by pfsense in this scenario?
Edit: As theres a fundamental difference between the wiki link and this link http://security.radware.com/knowledge-center/DDoSPedia/syn-ack-flood/
namely the later appears to be exploiting out of order packets and the subsequent overhead processing them, although nothing can stop any amount of bandwidth domination irrespective of what its called be it flooding, (D)Dos, if the attacks by SM have been syn ack floods described in the 2nd link, as PF in freebsd handles this, will this not require a bit of a rewrite of the fundamentals of PF to speed up its handling of this situation?
