What is the biggest attack in GBPS you stopped

tim.mcmanus · Jun 5, 2015, 5:39 PM

Smoking gun?

https://forums.freebsd.org/threads/freebsd-10-1-network-unaccessible-after-high-traffic.51743/

Potential fix? (synproxy state)

https://forums.freebsd.org/threads/enabling-ipv6-eventually-freezes-the-gateway.47959/#post-267820

See #4 from this link:

http://www.onlamp.com/pub/a/bsd/2004/04/15/pf_developers.html

https://forums.freebsd.org/threads/freebsd-9-0-release-syn-flood.29071/#post-163533

Notice how all of this freakin' awesome information is coming from the FreeBSD forums. I strongly suggest you consolidate your findings in this thread and move the discussion over to the FreeBSD networking forums. They have dozens of discussions on SYN floods with some excellent guidance given.

Supermule · Jun 5, 2015, 7:36 PM

Yes. It just doesnt work as expected.

Until the drop in traffic occurs. So I need to find a way to see whats initiated when the traffic drops.

tim.mcmanus · Jun 5, 2015, 8:06 PM

@Supermule:

Yes. It just doesnt work as expected.

Until the drop in traffic occurs. So I need to find a way to see whats initiated when the traffic drops.

https://forums.freebsd.org/forums/networking.7/

Supermule · Jun 6, 2015, 11:56 AM

Yes….one more forum to watch....................... :D

Supermule · Jun 7, 2015, 7:09 PM

Just a small update..

Changed NIC's to vmxnet3 in vmware and this happened.

Youtube Video

Same traffic drop but curves are different. On E1000 they follow each other. On vmxnet3 they are totally seperated. Red is 15mbit/s and black is almost zero.

Suddenly then traffic drops on incoming WAN and it begins to flow traffic and respond to ping for Google DNS.

No change what so ever to anything.

vmxnet3 drivers are much worse handling this kind of attack than the E1000 drivers.

EDIT: When Core nr 3 drops, the FW comes alive and begins to route packets. Overall load is cut in half.

EDIT EDIT: Getting 100% interrupt in the top -HSP on the console. Again the E1000 almost had zero.

Logs are flooded with this.

| Jun 7 19:54:14 | check_reload_status: Reloading filter |
| Jun 7 19:54:14 | check_reload_status: Restarting OpenVPN tunnels/interfaces |
| Jun 7 19:54:14 | check_reload_status: Restarting ipsec tunnels |
| Jun 7 19:54:14 | check_reload_status: updating dyndns Yousee |
| Jun 7 19:54:01 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:53:49 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:53:43 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:53:38 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:53:33 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:53:28 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:53:22 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:53:15 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:53:06 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:53:01 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:56 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:51 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:45 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:40 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:35 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:28 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:23 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:18 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:13 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:07 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:52:02 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:51:56 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:51:50 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:51:46 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:51:40 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:51:34 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:51:29 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:51:24 | check_reload_status: Reloading filter |
| Jun 7 19:51:24 | check_reload_status: Restarting OpenVPN tunnels/interfaces |
| Jun 7 19:51:24 | check_reload_status: Restarting ipsec tunnels |
| Jun 7 19:51:24 | check_reload_status: updating dyndns Yousee |
| Jun 7 19:51:22 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:51:16 | kernel: vmx0: watchdog timeout on queue 0 |
| Jun 7 19:51:11 | kernel: vmx0: watchdog timeout on queue 0 |

firewalluser · Jun 7, 2015, 8:08 PM

And an update from me.

Currently downloading and getting 5.42Mbps according to the dashboard, yet when you did the ddos the otherday that was only giving showing 2.42Mbps.

So why does a ddos cause my bandwidth to drop down? Could it be related to this post https://forum.pfsense.org/index.php?topic=94903.0 ie a hw issue, or is freebsd or pfsense.

I have a huwei supplied router dont know if it can go into modem only mode, but if it can I will check the speeds again with this device if you dont mind doing a Britney Spears and hit me one more time? ;)

Supermule · Jun 7, 2015, 8:11 PM

No problem.

Let me know when youre ready and what IP/port it is.

Supermule · Jun 28, 2015, 4:47 PM

.-….........

Harvy66 · Jun 28, 2015, 4:38 PM

How resilient? No longer an issue for established connections as long as there is enough bandwidth, kind of resilient? You tone makes it sound as if it is effectively fixed. Time to party? 8)

Supermule · Jun 28, 2015, 4:47 PM

EDIT: It was my bandwith that caused to attacked IP to survive.

No change in resilience. (Sorry).

Harvy66 · Jun 28, 2015, 8:04 PM

@Supermule:

EDIT: It was my bandwith that caused to attacked IP to survive.

No change in resilience. (Sorry).

It was your bandwidth? Isn't that the goal? The weakest link should be the bandwidth, not the firewall, yes? What issue(s) still remain? Same?

Supermule · Jun 28, 2015, 8:38 PM

As soon as I was able to push 10mbit/s out on my private line, then it failed again….

Harvy66 · Jun 29, 2015, 12:15 AM

So 10Mb/s of special traffic against your greater than 10Mb/s link caused the link to fail? Did the firewall stop working or only the link go down?

Supermule · Jun 29, 2015, 8:25 AM

Firewall stopped routing traffic. Link was fine.

Lots of packetloss once again.

Harvy66 · Jun 29, 2015, 1:21 PM

So same issue as before, firewall is the weakest link, not the bandwidth? Maybe 2.3.

Any news from the FreeBSD side of things?

Supermule · Jun 29, 2015, 1:22 PM

Not yet. They are digesting the attack that I did yesterday and curretnly looking at states not beeing freed as they should…. AFAIK.

torontob · Jul 9, 2015, 2:26 AM

Did this end up in nowhere with the issue still being there?

Supermule · Jul 9, 2015, 2:33 AM

Still working on it.

Supermule · Jul 23, 2015, 7:16 AM

https://lists.freebsd.org/pipermail/freebsd-announce/2015-July/001655.html

Latest update.

peterclark4 · Jul 23, 2015, 7:25 AM

@Supermule:

https://lists.freebsd.org/pipermail/freebsd-announce/2015-July/001655.html

Latest update.

Does this need to be included in 2.2.4 before it is released?