6100 SLOW in comparison to Protectli FW6E

[manilx]

@stephenw10 The issue is with Suricata. If I disable that on the LAN port I get 930+

I also have pfblockerng, tailscale, wireguard, Status_Traffic_Totals, cron, nut,

[manilx]

@stephenw10

[stephenw10]

Ah almost all Suricata. Interesting, I'd say some tuning is in order there then.

Looks like it's using netmap so it's configured in in-line mode?

It also looks like it's all on ix3 which is the WAN by default. You've using that NIC as LAN though?

[manilx]

@stephenw10 Yes, inline checking of WAN only.

Not a problem AT ALL on the Protectli.

[SteveITS]

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

inline checking of WAN only

Ah. You wrote "If I disable that on the LAN" :)

Try Suricata on LAN, because on WAN it will see and scan every packet even those that will be dropped by the firewall a millisecond later. That will at least reduce the number of packets scanned.

I have not used Inline much at all but from posts here, it is highly NIC driver dependent. On post
https://forum.netgate.com/topic/138613/configuring-pfsense-netmap-for-suricata-inline-ips-mode-on-em-igb-interfaces
see "3a. disable all flow control" ?
also "How can I reduce CPU usage or increase Suricata throughput?"

[manilx]

@steveits "If your interface is not named eg "em0" or "igb0" or similar, these instructions may be of limited use to you."

My interfaces are ix3 and igc0.......

[SteveITS]

@manilx I know that's why I picked out those two things. :) If it doesn't help then disregard.

Changing the Suricata interface would also change the driver used...though those are both Intel.

[manilx]

@steveits Disabled it on WAN and enabled on LAN.
Throughput improves as I mentioned above (meant WAN when I wrote LAN, sorry).

[stephenw10]

I agree there is no point running Suricata on WAN unless you're hosting services behind the firewall. Running on LAN means you are only parsing the traffic that counts and that you can see the internal IPs involved so you can see specific clients.

Both ix and igc should be netmap native because of iflib.

Steve

[manilx]

@stephenw10 Problem is: I HAVE open ports and therefore was running on WAN.
After further tests changing to LAN doesn't help. Speed is reduced the same. ONLY disabling completely "fixes" this.

And I come to the conclusion that I was a LOT better served by the Protectli!

This CPU is to slow. 6100 I thought was good. It isn't!

[stephenw10]

It's surprising how much load Suricata is adding there. I imagine you must have a lot of signature lists loaded?

[manilx]

@stephenw10 Feodo Tracker Botnet C2 IP Rules
ABUSE.ch SSL Blacklist Rules
emerging-ciarmy.rules
emerging-compromised.rules
emerging-drop.rules
emerging-dshield.rules
emerging-malware.rules
emerging-mobile_malware.rules
emerging-phishing.rules
emerging-trojan.rules
emerging-worm.rules

And this:

[SteveITS]

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

I HAVE open ports and therefore was running on WAN

For reference, having open ports is kind of irrelevant...the inbound packets would be scanned on the LAN side anyway, and dropped there so won't get to devices. See the comment in this post by the package maintainer, on putting IDS on LAN.

Also, Spamhaus DROP, etc. lists of IPs can be handled in firewall rules via pfBlockerNG(-devel) lists. DShield is in the ET_Block list there. That would pull the packet checking out of Suricata, might help a bit.

[manilx]

@steveits Thanks for all the info.
I have removed rules I had in pfblockedng, put Suricata on the LAN side.
Helped and I'm now at 700+ but far from 920 without it.
I'm now wonderin if I should use it at all and if it really is needed (on LAN) to protect my open ports.

[SteveITS]

@manilx Usually one can adjust the rulesets to only what they need, e.g. a web server doesn’t need SQL rules in front of it. Also encrypted traffic like HTTPS can’t be scanned, it just passes through.

[manilx]

@steveits What would you suggest to activate as rules for:
NFS
Plex
Emby

Appreciate your help.

[bmeeks]

If you are using Suricata with Inline IPS Mode, you may notice a throughput improvement by switching the Suricata runmode from "autofp" to "workers" on the INTERFACE SETTINGS tab.

The "workers" runmode can work better with netmap. The default runmode is "autofp" as that is a good general purpose starting point, but some configurations will benefit by switching to "workers". Any change you make to the runmode requires restarting Suricata on the interface before it is effective.

I am interested in what impact switching the runmode has for you, so please report back in this thread. "Workers" mode is best for multi-queue NICs.

Also, if you don't mind, please post the hardware specs (CPU type and speed, particularly) of the Protectli unit you had previously.

[manilx]

@bmeeks Did that change and it did a HUGE boost!
I had only snort rules with IPS balanced and policy and had 815 download. After changing this setting it maxed out at 935 which is near the maximum I get. I will again select more rules I had before and recheck the speed now.

The Protectli was a FW6E – 6 Port Intel i7
Intel i7 8550U Quad Core with Hyperthreading (8 threads) at up to 4GHz (turbo boost)
Dual DDR4 memory up to 64GB
mSATA and/or 2.5″ solid state drive
6 Intel Gigabit Ethernet NIC ports
AES-NI

[manilx]

@manilx I have reenabled the emerging-3coresec as a test.
Run speedtest and got full download speed of 930. Excellent!

BUT in the middle of repeating that I lost access to the 6100, no ping. LED was blinking blue, normal.
Scary!
Powered it off by 5s press to power button. Got an orange LED.
Various tries to power it on by 5s press were uncuccessful. Pulled the power plug.
Turned on again.

Repeated tests. 930 download.

[manilx]

@manilx After some more speedtests the unit died on me twice again! Completely unstable.

This setting kills it after a short while.
Rolling back to a zfs snapshot.