Supermicro 5018D-FN8T Tweaks

yorke

pfSense 2.4.4-RELEASE-p2
Supermicro SuperServer 5018D-FN8T Xeon D Mini
Intel(R) Xeon(R) CPU D-1518

CPU usage 2%
Memory usage 10%

I see others are also using the Supermicro SuperServer 5018D-FN8T
can someone share their bios settings /pfsense tweaks to get the most out of this setup, with a 1 GIG connection from isp ?
Thank you

stephenw10

The default settings should be pretty good on that. Are you not seeing the full bandwidth through it?

Steve

yorke

Are you not seeing the full bandwidth through it?
Not seeing full bandwidth.
Speed listed below
Testing download speed................................................................................
Download: 343.78 Mbit/s
Testing upload speed................................................................................................
Upload: 26.89 Mbit/s

package install
pfBlockerNG-devel , suricata

If i remove and plug into the modem
Testing download speed..............................................................................
Download: 1106.53 Mbit/s
Testing upload speed................................................................................................
Upload: 45.21 Mbit/s

stephenw10

Run top -aSH at the command line while testing. See what is using CPU cycles and on which cores.

How are you testing?

Steve

yorke

@stephenw10 said in Supermicro 5018D-FN8T Tweaks:

Run top -aSH at the command line while testing

Run top -aSH at the command line while testing

last pid: 18725; load averages: 0.49, 0.26, 0.19 up 5+18:40:15 13:50:08
673 processes: 9 running, 556 sleeping, 108 waiting

Mem: 406M Active, 7725M Inact, 3333M Wired, 180K Buf, 20G Free
ARC: 880M Total, 639M MFU, 221M MRU, 762K Anon, 3388K Header, 15M Other
328M Compressed, 1332M Uncompressed, 4.06:1 Ratio
Swap: 20G Total, 20G Free

PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
11 root 155 ki31 0K 128K CPU2 2 135.8H 100.00% [idle{idle: cpu2}]
11 root 155 ki31 0K 128K CPU7 7 135.7H 100.00% [idle{idle: cpu7}]
11 root 155 ki31 0K 128K CPU1 1 135.5H 99.85% [idle{idle: cpu1}]
11 root 155 ki31 0K 128K CPU4 4 135.9H 99.46% [idle{idle: cpu4}]
11 root 155 ki31 0K 128K RUN 5 135.8H 99.46% [idle{idle: cpu5}]
11 root 155 ki31 0K 128K CPU3 3 135.8H 98.68% [idle{idle: cpu3}]
11 root 155 ki31 0K 128K CPU6 6 135.7H 97.85% [idle{idle: cpu6}]
11 root 155 ki31 0K 128K CPU0 0 136.2H 97.27% [idle{idle: cpu0}]
17051 root 45 0 43404K 37952K select 0 0:00 3.56% /usr/local/bin/python2.7 /usr/local/bin/speedtest-cli
35630 root 20 0 855M 807M uwait 6 1:51 0.20% /usr/local/bin/suricata -i igb4 -D -c /usr/local/etc/suricata/suricata_60829_igb4/suricata.yaml --pidfile /var/run/suricata_igb4
88664 root 20 0 881M 840M uwait 1 0:20 0.20% /usr/local/bin/suricata -i igb3 -D -c /usr/local/etc/suricata/suricata_3581_igb3/suricata.yaml --pidfile /var/run/suricata_igb3
88664 root 20 0 881M 840M uwait 6 0:10 0.20% /usr/local/bin/suricata -i igb3 -D -c /usr/local/etc/suricata/suricata_3581_igb3/suricata.yaml --pidfile /var/run/suricata_igb3
88664 root 20 0 881M 840M uwait 4 1:52 0.10% /usr/local/bin/suricata -i igb3 -D -c /usr/local/etc/suricata/suricata_3581_igb3/suricata.yaml --pidfile /var/run/suricata_igb3
85154 root 21 0 96076K 38408K piperd 6 0:30 0.10% php-fpm: pool nginx (php-fpm)
81880 unbound 20 0 10446M 5647M kqread 4 19:09 0.00% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound}
19455 root 20 0 8948K 2596K nanslp 6 14:14 0.00% [dpinger{dpinger}]
12 root -60 - 0K 1728K WAIT 4 8:47 0.00% [intr{swi4: clock (0)}]
12 root -92 - 0K 1728K WAIT 4 4:27 0.00% [intr{irq295: ix3:q4}]

How are you testing?
Installed speedtest-cli on pfsense

stephenw10

Ok so not the same test as directly into the modem then. Can you test with that client behind pfSense instead?
Though I would certainly expect a far higher result that that anyway.

Check for errors on the WAN interface in Status > Interfaces

Steve

yorke

@stephenw10 said in Supermicro 5018D-FN8T Tweaks:

test with that client behind pfSense instead?
test with that client behind pfSense instead?
download mbps
417.52
upload mbps
35.1

errors on the WAN interface
In/out errors
0/7

stephenw10

What sort of modem is it? What connection type?

Grab a packet capture on WAN look for packet fragments, re-transmits etc.

Steve

tman222

I have been using this system for a couple years now with a symmetric 1Gbit fiber connection and it is definitely capable of passing gigabit speeds even with IDS/IPS enabled (in my case I run Snort). Here a couple more suggestions:

Networking tweaks - put these four lines below in your loader.conf.local file (if you're using the SFP+ ports replace igb with ix):

hw.igb.rx_process_limit="-1"
hw.igb.tx_process_limit="-1"
hw.igb.txd="2048"
hw.igb.rxd="2048"

I'd also recommend disabling flow control and energy efficient ethernet, unless you have a specific need/use for them.
Another good thread with tuning tips:

https://forum.netgate.com/topic/101391/loader-conf-local-tuning-for-modern-hardware/

Finally, if you disable Suricata temporarily, do you get full speed with a client behind pfSense, or does it not make a difference?

Hope this helps.