6100 SLOW in comparison to Protectli FW6E

[manilx]

Just installed my new 6100 coming from Protectli.

Running a speed test I get a LOT slower speed in comparison and the CPU goes way up to 100%!!!

With Suricata:

Without Suricata:

I chose this model because it's on the high end of the spectrum of products BUT what a disappointment!

The Protectli has always 930+ download with Surticata and CPU never reach more than 50%.

Is there anything I can do in settings to speed this up?

Otherwise I'm inclinded to return this unit.

[manilx]

This post is deleted!

[stephenw10]

The 6100 should easily pass >1Gbps in a default config. If the CPU goes to 100% whilst passing only 600Mbps I assume you have some packages installed? Which ones?

If you connect to the command line whilst testing and run top -HaSP what is shown using the CPU?

Steve

[manilx]

@stephenw10 The issue is with Suricata. If I disable that on the LAN port I get 930+

I also have pfblockerng, tailscale, wireguard, Status_Traffic_Totals, cron, nut,

[manilx]

@stephenw10

[stephenw10]

Ah almost all Suricata. Interesting, I'd say some tuning is in order there then.

Looks like it's using netmap so it's configured in in-line mode?

It also looks like it's all on ix3 which is the WAN by default. You've using that NIC as LAN though?

[manilx]

@stephenw10 Yes, inline checking of WAN only.

Not a problem AT ALL on the Protectli.

[SteveITS]

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

inline checking of WAN only

Ah. You wrote "If I disable that on the LAN" :)

Try Suricata on LAN, because on WAN it will see and scan every packet even those that will be dropped by the firewall a millisecond later. That will at least reduce the number of packets scanned.

I have not used Inline much at all but from posts here, it is highly NIC driver dependent. On post
https://forum.netgate.com/topic/138613/configuring-pfsense-netmap-for-suricata-inline-ips-mode-on-em-igb-interfaces
see "3a. disable all flow control" ?
also "How can I reduce CPU usage or increase Suricata throughput?"

[manilx]

@steveits "If your interface is not named eg "em0" or "igb0" or similar, these instructions may be of limited use to you."

My interfaces are ix3 and igc0.......

[SteveITS]

@manilx I know that's why I picked out those two things. :) If it doesn't help then disregard.

Changing the Suricata interface would also change the driver used...though those are both Intel.

[manilx]

@steveits Disabled it on WAN and enabled on LAN.
Throughput improves as I mentioned above (meant WAN when I wrote LAN, sorry).

[stephenw10]

I agree there is no point running Suricata on WAN unless you're hosting services behind the firewall. Running on LAN means you are only parsing the traffic that counts and that you can see the internal IPs involved so you can see specific clients.

Both ix and igc should be netmap native because of iflib.

Steve

[manilx]

@stephenw10 Problem is: I HAVE open ports and therefore was running on WAN.
After further tests changing to LAN doesn't help. Speed is reduced the same. ONLY disabling completely "fixes" this.

And I come to the conclusion that I was a LOT better served by the Protectli!

This CPU is to slow. 6100 I thought was good. It isn't!

[stephenw10]

It's surprising how much load Suricata is adding there. I imagine you must have a lot of signature lists loaded?

[manilx]

@stephenw10 Feodo Tracker Botnet C2 IP Rules
ABUSE.ch SSL Blacklist Rules
emerging-ciarmy.rules
emerging-compromised.rules
emerging-drop.rules
emerging-dshield.rules
emerging-malware.rules
emerging-mobile_malware.rules
emerging-phishing.rules
emerging-trojan.rules
emerging-worm.rules

And this:

[SteveITS]

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

I HAVE open ports and therefore was running on WAN

For reference, having open ports is kind of irrelevant...the inbound packets would be scanned on the LAN side anyway, and dropped there so won't get to devices. See the comment in this post by the package maintainer, on putting IDS on LAN.

Also, Spamhaus DROP, etc. lists of IPs can be handled in firewall rules via pfBlockerNG(-devel) lists. DShield is in the ET_Block list there. That would pull the packet checking out of Suricata, might help a bit.

[manilx]

@steveits Thanks for all the info.
I have removed rules I had in pfblockedng, put Suricata on the LAN side.
Helped and I'm now at 700+ but far from 920 without it.
I'm now wonderin if I should use it at all and if it really is needed (on LAN) to protect my open ports.

[SteveITS]

@manilx Usually one can adjust the rulesets to only what they need, e.g. a web server doesn’t need SQL rules in front of it. Also encrypted traffic like HTTPS can’t be scanned, it just passes through.

[manilx]

@steveits What would you suggest to activate as rules for:
NFS
Plex
Emby

Appreciate your help.

[bmeeks]

If you are using Suricata with Inline IPS Mode, you may notice a throughput improvement by switching the Suricata runmode from "autofp" to "workers" on the INTERFACE SETTINGS tab.

The "workers" runmode can work better with netmap. The default runmode is "autofp" as that is a good general purpose starting point, but some configurations will benefit by switching to "workers". Any change you make to the runmode requires restarting Suricata on the interface before it is effective.

I am interested in what impact switching the runmode has for you, so please report back in this thread. "Workers" mode is best for multi-queue NICs.

Also, if you don't mind, please post the hardware specs (CPU type and speed, particularly) of the Protectli unit you had previously.