Low bandwidth on initial install
-
I have a new install on a new i5-7267U setup. My speedtest with with pfSense get about 900MB/s down and 40 MB/s up.
Out of the box configuration pfSense get only about 1/2 of the download ~400MB/s down and ~30MB/s up.Any suggestions on how to better optimize it would be greatly appreciated.
My System:
Name pfSense.localdomain User admin@192.168.1.100 (Local Database) System pfSense Netgate Device ID: BIOS Vendor: American Megatrends Inc. Version: 5.12 Release Date: Mon Jan 13 2020 Version 2.4.5-RELEASE (amd64) built on Tue Mar 24 15:25:50 EDT 2020 FreeBSD 11.3-STABLE The system is on the latest version. Version information updated at Mon Jun 8 7:08:57 EDT 2020 CPU Type Intel(R) Core(TM) i5-7267U CPU @ 3.10GHz Current: 3000 MHz, Max: 3001 MHz 4 CPUs: 1 package(s) x 2 core(s) x 2 hardware threads AES-NI CPU Crypto: Yes (active) Hardware crypto AES-CBC,AES-XTS,AES-GCM,AES-ICM Kernel PTI Enabled MDS Mitigation VERW Uptime 09 Hours 40 Minutes 28 Seconds Current date/time Mon Jun 8 8:03:06 EDT 2020 DNS server(s) 127.0.0.1 75.75.75.75 75.75.76.76 2001:558:feed::1 2001:558:feed::2 Last config change Mon Jun 8 7:08:39 EDT 2020 State table size 0% (138/805000) Show states MBUF Usage 1% (10126/1000000) Temperature 27.9°C Load average 0.07, 0.13, 0.11 CPU usage 0% Memory usage 4% of 8058 MiB SWAP usage 0% of 2689 MiB Disk usage: / 2% of 55GiB - ufs /var/run 4% of 3.4MiB - ufs in RAMAnd this is under load when doing SpeedTest
DiagnosticsSystem Activity CPU Activity last pid: 82718; load averages: 0.25, 0.17, 0.12 up 0+09:32:41 07:55:19 208 processes: 5 running, 154 sleeping, 49 waiting Mem: 19M Active, 104M Inact, 316M Wired, 19M Buf, 7371M Free Swap: 2689M Total, 2689M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 155 ki31 0K 64K CPU0 0 569:42 99.46% [idle{idle: cpu0}] 11 root 155 ki31 0K 64K RUN 3 569:48 99.37% [idle{idle: cpu3}] 11 root 155 ki31 0K 64K CPU2 2 569:43 97.07% [idle{idle: cpu2}] 11 root 155 ki31 0K 64K CPU1 1 569:51 96.78% [idle{idle: cpu1}] 12 root -92 - 0K 784K WAIT 1 0:12 1.86% [intr{irq267: igb0:que 1}] 12 root -92 - 0K 784K WAIT 3 0:17 1.56% [intr{irq269: igb0:que 3}] 12 root -92 - 0K 784K WAIT 2 0:12 1.46% [intr{irq268: igb0:que 2}] 12 root -92 - 0K 784K WAIT 0 0:12 0.78% [intr{irq271: igb1:que 0}] 69320 root 26 0 95000K 35084K piperd 1 0:06 0.59% php-fpm: pool nginx (php-fpm) 20 root -16 - 0K 16K pftm 3 1:26 0.00% [pf purge] 12 root -60 - 0K 784K WAIT 2 0:38 0.00% [intr{swi4: clock (0)}] 0 root -16 - 0K 944K swapin 3 0:15 0.00% [kernel{swapper}] 12 root -92 - 0K 784K WAIT 2 0:14 0.00% [intr{irq273: igb1:que 2}] 12 root -92 - 0K 784K WAIT 1 0:12 0.00% [intr{irq272: igb1:que 1}] 12 root -92 - 0K 784K WAIT 3 0:12 0.00% [intr{irq274: igb1:que 3}] 12 root -92 - 0K 784K WAIT 0 0:11 0.00% [intr{irq266: igb0:que 0}] 22 root -16 - 0K 16K - 1 0:08 0.00% [rand_harvestq] 344 root 52 0 95128K 34816K accept 2 0:06 0.00% php-fpm: pool nginx (php-fpm) 345 root 52 0 95000K 34808K accept 3 0:05 0.00% php-fpm: pool nginx (php-fpm) 48612 root 52 0 95000K 34528K accept 0 0:05 0.00% php-fpm: pool nginx (php-fpm) 73588 root 20 0 23680K 8832K kqread 2 0:04 0.00% nginx: worker process (nginx) 74466 root 20 0 12464K 5748K select 1 0:03 0.00% /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf -p /var/run/ntpd.pid{ntpd} 86252 root 20 0 6904K 2368K nanslp 1 0:03 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP6 -B fe80::2e0:5cff:fe68:1859%igb0 -p /var/run/dpinger_WAN_DHCP6~fe80::2e0:5cff:fe68: 80252 root 20 0 6412K 2468K select 0 0:03 0.00% /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/log -P /var/run/syslog.pid -f /etc/syslog.conf 16746 root 20 0 6964K 2728K bpf 3 0:03 0.00% /usr/local/sbin/filterlog -i pflog0 -p /var/run/filterlog.pid 85654 root 20 0 6904K 2336K nanslp 2 0:03 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP -B 10.0.0.87 -p /var/run/dpinger_WAN_DHCP~10.0.0.87~10.0.0.1.pid -u /var/run/dpinger 22202 root 52 20 6976K 2480K wait 1 0:02 0.00% /bin/sh /var/db/rrd/updaterrd.sh 80917 root 20 0 6264K 2208K select 1 0:02 0.00% /usr/local/sbin/radvd -p /var/run/radvd.pid -C /var/etc/radvd.conf -m syslog 343 root 20 0 94868K 25460K kqread 0 0:01 0.00% php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm) 12 root -92 - 0K 784K WAIT 0 0:01 0.00% [intr{irq275: igb1:link}] 12 root -92 - 0K 784K WAIT 2 0:01 0.00% [intr{irq270: igb0:link}] 86252 root 20 0 6904K 2368K nanslp 0 0:01 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP6 -B fe80::2e0:5cff:fe68:1859%igb0 -p /var/run/dpinger_WAN_DHCP6~fe80::2e0:5cff:fe68: 85654 root 20 0 6904K 2336K nanslp 3 0:01 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP -B 10.0.0.87 -p /var/run/dpinger_WAN_DHCP~10.0.0.87~10.0.0.1.pid -u /var/run/dpinger 85654 root 20 0 6904K 2336K sbwait 1 0:01 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP -B 10.0.0.87 -p /var/run/dpinger_WAN_DHCP~10.0.0.87~10.0.0.1.pid -u /var/run/dpinger 86252 root 20 0 6904K 2368K sbwait 1 0:01 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP6 -B fe80::2e0:5cff:fe68:1859%igb0 -p /var/run/dpinger_WAN_DHCP6~fe80::2e0:5cff:fe68: 28 root 16 - 0K 16K syncer 1 0:01 0.00% [syncer] 80641 dhcpd 20 0 16460K 10136K select 0 0:01 0.00% /usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid igb1 23 root -16 - 0K 48K psleep 0 0:01 0.00% [pagedaemon{dom0}] 73492 root 20 0 23680K 8260K kqread 2 0:01 0.00% nginx: worker process (nginx) 26 root 20 - 0K 32K sdflus 1 0:01 0.00% [bufdaemon{/ worker}] 16 root -68 - 0K 80K - 2 0:00 0.00% [usb{usbus0}] 21 root -16 - 0K 16K tzpoll 3 0:00 0.00% [acpi_thermal] 26 root -16 - 0K 32K psleep 3 0:00 0.00% [bufdaemon{bufdaemon}] 29 root -16 - 0K 16K vlruwt 0 0:00 0.00% [vnlru] 27 root -16 - 0K 16K - 2 0:00 0.00% [bufspacedaemon] 12 root -88 - 0K 784K WAIT 0 0:00 0.00% [intr{irq265: ahci0}] 12222 _dhcp 20 0 6456K 2416K select 3 0:00 0.00% dhclient: igb0 (dhclient) 0 root 8 - 0K 944K - 2 0:00 0.00% [kernel{thread taskq}] 26596 root 20 0 6340K 2260K select 1 0:00 0.00% /usr/local/sbin/dhcp6c -d -c /var/etc/dhcp6c_wan.conf -p /var/run/dhcp6c_igb0.pid igb0 14 root -8 - 0K 48K - 0 0:00 0.00% [geom{g_event}] 12 root -72 - 0K 784K WAIT 3 0:00 0.00% [intr{swi1: netisr 0}] 12 root -72 - 0K 784K WAIT 1 0:00 0.00% [intr{swi1: netisr 2}] 78424 unbound 20 0 52132K 23896K kqread 3 0:00 0.00% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound} 74091 root 20 0 6376K 2372K nanslp 0 0:00 0.00% /usr/sbin/cron -s 12 root -72 - 0K 784K WAIT 2 0:00 0.00% [intr{swi1: netisr 1}] 68 root -8 - 0K 16K mdwait 2 0:00 0.00% [md0] 358 root 40 20 6752K 2548K kqread 0 0:00 0.00% /usr/local/sbin/check_reload_status 423 root 20 0 9156K 4976K select 2 0:00 0.00% /sbin/devd -q -f /etc/pfSense-devd.conf 1 root 20 0 5012K 840K wait 3 0:00 0.00% [init] 16 root -68 - 0K 80K - 1 0:00 0.00% [usb{usbus0}] 85654 root 20 0 6904K 2336K accept 3 0:00 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP -B 10.0.0.87 -p /var/run/dpinger_WAN_DHCP~10.0.0.87~10.0.0.1.pid -u /var/run/dpinger 86252 root 20 0 6904K 2368K accept 2 0:00 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP6 -B fe80::2e0:5cff:fe68:1859%igb0 -p /var/run/dpinger_WAN_DHCP6~fe80::2e0:5cff:fe68: 9 root -16 - 0K 16K - 2 0:00 0.00% [soaiod2] 8 root -16 - 0K 16K - 1 0:00 0.00% [soaiod1] 78424 unbound 20 0 52132K 23896K kqread 1 0:00 0.00% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound} 18 root -16 - 0K 16K - 0 0:00 0.00% [soaiod4] 17 root -16 - 0K 16K - 0 0:00 0.00% [soaiod3] 81134 root 20 0 6192K 1912K nanslp 1 0:00 0.00% minicron: helper /usr/local/bin/ping_hosts.sh (minicron) 78424 unbound 20 0 52132K 23896K kqread 0 0:00 0.00% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound} 78424 unbound 20 0 52132K 23896K kqread 2 0:00 0.00% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound} 7 root -16 - 0K 32K - 1 0:00 0.00% [cam{scanner}] 7 root -16 - 0K 32K - 2 0:00 0.00% [cam{doneq0}] 26346 root 20 0 6724K 2732K wait 0 0:00 0.00% login [pam] (login) 12 root -96 - 0K 784K WAIT 2 0:00 0.00% [intr{irq296: hdac0}] 12 root -88 - 0K 784K WAIT 2 0:00 0.00% [intr{irq264: xhci0}] 26628 root 21 0 6976K 2916K wait 0 0:00 0.00% -sh (sh) 82718 root 26 0 7820K 3160K CPU3 3 0:00 0.00% /usr/bin/top -baHS 999 27519 root 52 0 6976K 2724K ttyin 2 0:00 0.00% /bin/sh /etc/rc.initial 14 root -8 - 0K 48K - 3 0:00 0.00% [geom{g_down}] 16 root -68 - 0K 80K - 0 0:00 0.00% [usb{usbus0}] 23792 root 52 0 6316K 2032K ttyin 3 0:00 0.00% /usr/libexec/getty Pc ttyv2 23708 root 52 0 6316K 2032K ttyin 1 0:00 0.00% /usr/libexec/getty Pc ttyv1 24397 root 52 0 6316K 2032K ttyin 0 0:00 0.00% /usr/libexec/getty Pc ttyv5 24967 root 52 0 6316K 2032K ttyin 0 0:00 0.00% /usr/libexec/getty Pc ttyv7 24640 root 52 0 6316K 2032K ttyin 2 0:00 0.00% /usr/libexec/getty Pc ttyv6 24094 root 52 0 6316K 2032K ttyin 1 0:00 0.00% /usr/libexec/getty Pc ttyv3 24162 root 52 0 6316K 2032K ttyin 3 0:00 0.00% /usr/libexec/getty Pc ttyv4 25 root 155 ki31 0K 16K pgzero 0 0:00 0.00% [pagezero] 81486 root 20 0 6192K 1912K nanslp 2 0:00 0.00% minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts (minicron) 73202 root 52 0 21632K 7168K pause 3 0:00 0.00% nginx: master process /usr/local/sbin/nginx -c /var/etc/nginx-webConfigurator.conf (nginx) 19 root -16 - 0K 16K waitin 3 0:00 0.00% [sctp_iterator] 7849 root 52 0 6456K 2308K select 2 0:00 0.00% dhclient: igb0 [priv] (dhclient) 14 root -8 - 0K 48K - 3 0:00 0.00% [geom{g_up}] 86252 root 52 0 6904K 2368K uwait 0 0:00 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP6 -B fe80::2e0:5cff:fe68:1859%igb0 -p /var/run/dpinger_WAN_DHCP6~fe80::2e0:5cff:fe68: 85654 root 52 0 6904K 2336K uwait 0 0:00 0.00% /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP -B 10.0.0.87 -p /var/run/dpinger_WAN_DHCP~10.0.0.87~10.0.0.1.pid -u /var/run/dpinger 65276 root 52 20 4144K 1824K nanslp 3 0:00 0.00% sleep 58698 72950 root 52 20 4144K 1824K nanslp 2 0:00 0.00% sleep 60 81807 root 52 0 6192K 1896K wait 1 0:00 0.00% /usr/local/bin/minicron 86400 /var/run/update_alias_url_data.pid /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data 80965 root 52 0 6192K 1896K wait 1 0:00 0.00% /usr/local/bin/minicron 240 /var/run/ping_hosts.pid /usr/local/bin/ping_hosts.sh 81180 root 52 0 6192K 1896K wait 1 0:00 0.00% /usr/local/bin/minicron 3600 /var/run/expire_accounts.pid /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts 0 root -52 - 0K 944K - 3 0:00 0.00% [kernel{mca taskq}] 360 root 52 20 6752K 2396K kqread 1 0:00 0.00% check_reload_status: Monitoring daemon of check_reload_status (check_reload_status) 82247 root 52 0 6192K 1912K nanslp 1 0:00 0.00% minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data (minicron) 65035 root 52 20 6976K 2696K wait 2 0:00 0.00% /bin/sh /etc/rc.update_pkg_metadata 12 root -52 - 0K 784K WAIT 2 0:00 0.00% [intr{swi6: Giant taskq}] 12 root -52 - 0K 784K WAIT 2 0:00 0.00% [intr{swi6: task queue}] 0 root -76 - 0K 944K - 1 0:00 0.00% [kernel{softirq_1}] 0 root -76 - 0K 944K - 2 0:00 0.00% [kernel{softirq_2}] 0 root -76 - 0K 944K - 0 0:00 0.00% [kernel{softirq_0}] 0 root 8 - 0K 944K - 0 0:00 0.00% [kernel{firmware taskq}] 0 root -8 - 0K 944K - 1 0:00 0.00% [kernel{CAM taskq}] 13 root -16 - 0K 64K sleep 0 0:00 0.00% [ng_queue{ng_queue0}] 0 root -76 - 0K 944K - 3 0:00 0.00% [kernel{softirq_3}] 0 root 8 - 0K 944K - 1 0:00 0.00% [kernel{acpi_task_2}] 13 root -16 - 0K 64K sleep 1 0:00 0.00% [ng_queue{ng_queue1}] 0 root -76 - 0K 944K - 0 0:00 0.00% [kernel{if_io_tqg_0}] 0 root 8 - 0K 944K - 3 0:00 0.00% [kernel{linuxkpi_short_wq_1}] 0 root 8 - 0K 944K - 2 0:00 0.00% [kernel{kqueue_ctx taskq}] 0 root 8 - 0K 944K - 1 0:00 0.00% [kernel{acpi_task_0}] 0 root 8 - 0K 944K - 1 0:00 0.00% [kernel{aiod_kick taskq}] 10 root -16 - 0K 16K audit_ 1 0:00 0.00% [audit] 0 root -76 - 0K 944K - 1 0:00 0.00% [kernel{if_io_tqg_1}] 0 root -92 - 0K 944K - 1 0:00 0.00% [kernel{igb4 que (qid 0)}] 0 root 8 - 0K 944K - 1 0:00 0.00% [kernel{linuxkpi_short_wq_2}] 0 root 8 - 0K 944K - 2 0:00 0.00% [kernel{linuxkpi_short_wq_0}] 0 root 8 - 0K 944K - 3 0:00 0.00% [kernel{linuxkpi_long_wq_2}] 0 root 8 - 0K 944K - 2 0:00 0.00% [kernel{linuxkpi_long_wq_1}] 0 root 8 - 0K 944K - 1 0:00 0.00% [kernel{linuxkpi_long_wq_0}] 0 root 8 - 0K 944K - 3 0:00 0.00% [kernel{linuxkpi_short_wq_4}] 0 root 8 - 0K 944K - 2 0:00 0.00% [kernel{linuxkpi_short_wq_3}] 0 root -20 - 0K 944K - 1 0:00 0.00% [kernel{crypto_3}] 0 root -76 - 0K 944K - 2 0:00 0.00% [kernel{if_io_tqg_2}] 0 root -76 - 0K 944K - 3 0:00 0.00% [kernel{if_io_tqg_3}] 0 root -92 - 0K 944K - 1 0:00 0.00% [kernel{igb0 que (qid 0)}] 0 root -92 - 0K 944K - 2 0:00 0.00% [kernel{igb1 que (qid 0)}] 0 root 8 - 0K 944K - 2 0:00 0.00% [kernel{mlx4}] 0 root 8 - 0K 944K - 1 0:00 0.00% [kernel{acpi_task_1}] 0 root 8 - 0K 944K - 2 0:00 0.00% [kernel{linuxkpi_long_wq_4}] 0 root 8 - 0K 944K - 1 0:00 0.00% [kernel{linuxkpi_long_wq_3}] 13 root -16 - 0K 64K sleep 1 0:00 0.00% [ng_queue{ng_queue2}] 23 root -16 - 0K 48K umarcl 2 0:00 0.00% [pagedaemon{uma}] 23 root -16 - 0K 48K launds 2 0:00 0.00% [pagedaemon{laundry: dom0}] 5 root -16 - 0K 16K crypto 1 0:00 0.00% [crypto returns 2] 2 root -16 - 0K 16K crypto 1 0:00 0.00% [crypto] 24 root -16 - 0K 16K psleep 2 0:00 0.00% [vmdaemon] 3 root -16 - 0K 16K crypto 1 0:00 0.00% [crypto returns 0] 4 root -16 - 0K 16K crypto 0 0:00 0.00% [crypto returns 1] 0 root -20 - 0K 944K - 0 0:00 0.00% [kernel{crypto_2}] 0 root -20 - 0K 944K - 3 0:00 0.00% [kernel{crypto_1}] 16 root -72 - 0K 80K - 0 0:00 0.00% [usb{usbus0}] 16 root -76 - 0K 80K - 0 0:00 0.00% [usb{usbus0}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb0 que (qid 1)}] 0 root -92 - 0K 944K - 1 0:00 0.00% [kernel{igb5 que (qid 1)}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb5 que (qid 0)}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb4 que (qid 2)}] 0 root -92 - 0K 944K - 2 0:00 0.00% [kernel{igb4 que (qid 1)}] 0 root -92 - 0K 944K - 2 0:00 0.00% [kernel{igb3 que (qid 2)}] 0 root -92 - 0K 944K - 0 0:00 0.00% [kernel{igb3 que (qid 1)}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb3 que (qid 0)}] 0 root -92 - 0K 944K - 2 0:00 0.00% [kernel{igb2 que (qid 3)}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb2 que (qid 1)}] 0 root -92 - 0K 944K - 1 0:00 0.00% [kernel{igb2 que (qid 0)}] 0 root -92 - 0K 944K - 2 0:00 0.00% [kernel{igb1 que (qid 2)}] 0 root -92 - 0K 944K - 1 0:00 0.00% [kernel{igb1 que (qid 1)}] 0 root -92 - 0K 944K - 1 0:00 0.00% [kernel{run0 net80211 taskq}] 0 root -92 - 0K 944K - 0 0:00 0.00% [kernel{igb5 que (qid 3)}] 13 root -16 - 0K 64K sleep 1 0:00 0.00% [ng_queue{ng_queue3}] 6 root -16 - 0K 16K crypto 0 0:00 0.00% [crypto returns 3] 15 root -16 - 0K 16K seqsta 0 0:00 0.00% [sequencer 00] 0 root -20 - 0K 944K - 1 0:00 0.00% [kernel{crypto_0}] 0 root -76 - 0K 944K - 1 0:00 0.00% [kernel{if_config_tqg_0}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb5 que (qid 2)}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb2 que (qid 2)}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb4 que (qid 3)}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb0 que (qid 2)}] 0 root -92 - 0K 944K - 3 0:00 0.00% [kernel{igb0 que (qid 3)}] 0 root -92 - 0K 944K - 0 0:00 0.00% [kernel{igb3 que (qid 3)}] 0 root -92 - 0K 944K - 1 0:00 0.00% [kernel{igb1 que (qid 3)}] 12 root -56 - 0K 784K WAIT -1 0:00 0.00% [intr{swi5: fast taskq}] 12 root -60 - 0K 784K WAIT -1 0:00 0.00% [intr{swi4: clock (1)}] 12 root -60 - 0K 784K WAIT -1 0:00 0.00% [intr{swi4: clock (3)}] 12 root -60 - 0K 784K WAIT -1 0:00 0.00% [intr{swi4: clock (2)}] 12 root -64 - 0K 784K WAIT -1 0:00 0.00% [intr{swi3: vm}] 12 root -72 - 0K 784K WAIT -1 0:00 0.00% [intr{swi1: netisr 3}] 12 root -72 - 0K 784K WAIT -1 0:00 0.00% [intr{swi1: pf send}] 12 root -72 - 0K 784K WAIT -1 0:00 0.00% [intr{swi1: pfsync}] 12 root -76 - 0K 784K WAIT -1 0:00 0.00% [intr{swi0: uart uart}] 12 root -84 - 0K 784K WAIT -1 0:00 0.00% [intr{irq1: atkbd0}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq281: igb3:que 0}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq282: igb3:que 1}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq283: igb3:que 2}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq284: igb3:que 3}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq285: igb3:link}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq286: igb4:que 0}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq287: igb4:que 1}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq288: igb4:que 2}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq289: igb4:que 3}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq290: igb4:link}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq291: igb5:que 0}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq292: igb5:que 1}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq276: igb2:que 0}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq293: igb5:que 2}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq294: igb5:que 3}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq295: igb5:link}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq277: igb2:que 1}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq279: igb2:que 3}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq278: igb2:que 2}] 12 root -92 - 0K 784K WAIT -1 0:00 0.00% [intr{irq280: igb2:link}]Thanks!
-
So, I'm poking around my system.
And I ran this command:pciconf -l -BbcevV igb0@pci0:1:0:0to check the PCIe bus capability of the IGB INTEL NIC. The NIC should capable of handling 1GB speeds. However it is connected via PCIe 2.0 x 1???
This would mean that the MAX bus throughput speed is 500MB/s which would be in line with my tests.
Can someone please confirm this?
Thanksigb0@pci0:1:0:0: class=0x020000 card=0x0000ffff chip=0x15338086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = 'I210 Gigabit Network Connection' class = network subclass = ethernet bar [10] = type Memory, range 32, base 0xdf500000, size 524288, enabled bar [18] = type I/O Port, range 32, base 0xe000, size 32, enabled bar [1c] = type Memory, range 32, base 0xdf580000, size 16384, enabled cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit, vector masks cap 11[70] = MSI-X supports 5 messages Table in map 0x1c[0x0], PBA in map 0x1c[0x2000] cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO NS link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1) ecap 0001[100] = AER 2 0 fatal 0 non-fatal 0 corrected ecap 0003[140] = Serial 1 00e05cffff681859 ecap 0017[1a0] = TPH Requester 1 -
look elsewhere for the reason for the value obtained....
the Intel I210 is a particularly good NIC for pfSense:
https://www.freebsd.org/cgi/man.cgi?igb(4)
perhaps:
https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html
https://docs.netgate.com/pfsense/en/latest/config/advanced-setup.html#Networkingedit: BTW: we know nothing else about its configuration, so we would welcome a more detailed description
(ISP, WAN type, installed packages, cable connections, etc.) -
@DaddyGo
Thanks for the reply, before even getting into the NIC and drivers:Per lane (each direction): PCI-E v1.x: 250 MB/s (2.5 GT/s) PCI-E v2.x: 500 MB/s (5 GT/s) PCI-E v3.0: 1 GB/s (8 GT/s) PCI-E v4.0: 2 GB/s (16 GT/s) 16 lane slot (each direction): PCI-E v1.x: 4 GB/s (40 GT/s) PCI-E v2.x: 8 GB/s (80 GT/s) PCI-E v3.0: 16 GB/s (128 GT/s)[https://serverfault.com/questions/399866/what-is-the-maximum-supported-data-rate-for-pcie]
With pciconfig, it reports
cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO NS link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)PCI-Express 2 x1(x1)
So the theoritical max based on the PCI-E speed is only 500MB/s regardless what the card can do.My setup is:
-
ISP: xfinity; I can get consistent 900+MB/s with speedtest on multiple computers.
-
pfSense - vanilla install, I just downgraded to v 2.4.4 p3 and it seems more consistent with bandwidth. No more run-away 'pf purge' process taking 10-15% CPU when the box is idle.
Also, I applied all the tweaks from:
https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html
The only ones I did not do is the chkecksum offload; that is still default setting. -
WAN cable connected to the pfSense box (20 feet)
-
LAN cable connected between pfSense box and my Laptop (3 feet)
When using the WAN cable in my laptop, I get 900+MB/s download consistently.
When I add pfSense and route traffic through it, I only gets ~420-450MB/s download.I will set up iperf on another local computer to further isolate the problem.
I also noticed the igb driver is quite old, anybody tried to get the latest intel driver for FreeBDS installed?
Thanks!
-
-
what kind of MOBO is this, that has a 2.0 pcie version?
pfSense is always current for the FreeBSD version (this is the basis) not as easy to replace a driver as under windows
FYI:
https://forum.netgate.com/topic/154337/pfsense-2-4-5-release-p1-now-availableolder versions are not recommended
-
@DaddyGo
https://forum.netgate.com/topic/154337/pfsense-2-4-5-release-p1-now-available
Nice! Hot off the press!"What kind of MOBO is this, that has a 2.0 pcie version?"
I know, I could not believe it either... But if we are to trust pciconf then it reports it as PCI-Express 2 link x1
Would you know any other way to confirm this in FreeBSD?
This is the box:
https://www.aliexpress.com/snapshot/0.html?spm=a2g0s.9042647.0.0.1f5b4c4d5KCYT1&orderId=5004244627695702&productId=4000618318595I figure'd I give it a try, but if PCIE 2.0 turns out to be true... then its going back.
Thanks!
-
@twoj Would you know any other way to confirm this in FreeBSD?
try install a plain linux on it first and see what your box knowsand
This is the box:
https://www.aliexpress.com/snapshot/0.html?spm=a2g0s.9042647.0.0.1f5b4c4d5KCYT1&orderId=5004244627695702&productId=4000618318595huhuhuhu these are Chinese wonders, rather get this (either
can be good): https://www.pfsense.org/products/ -
@twoj said in Low bandwidth on initial install:
PCI-Express 2 x1(x1)
So the theoritical max based on the PCI-E speed is only 500MB/s regardless what the card can do.You have to distinguish between bits and bytes.
Your Card have 2,5GT/s PCIe, its round about 500MB/s, but its LAN Speed ist round about 112MB/s.
Your bottleneck is somewhere else. -
@NOCling
NoCling... I believe you are right.I just tested pfSense with ipef (over a static route) and I'm getting 920Mbits/sec transfer rate with IS 1GB/s
So back to square one. Ahhh... I wished this was hardware problem.
In my current setup I'm my xfinity router is connected to the WAN interface of pfSense; my laptop is connected to the LAN interface of pfSense.
My intention is to throw out the xFinity router/modem and use Arris Modem instead, which would then connect to pfSense WAN interface.
Is the existing DHCP setup much different than the PPOE in terms of performance?
Thanks!
-
Try 2.4.5p1, its there.
Both NICs (LAN/WAN) Intel I210?
-
@NOCling
Yes. All 6 are I210. :-)I'm running 2.4.5.p1; I'm applying all harware tweaks again, still no joy. iPerf going 920MB/s regardless.
-
I think it's unthinkable, even if it's Chinese hardware, to install 6 pieces of I210 and they don't perform.
What do you want to achieve?
@twoj "Is the existing DHCP setup much different than the PPOE in terms of performance?"
Yes, PPPOE has its drawbacks, but it depends on what you use the box for,.....in normal use this is not significant.
Just think of MTU:
https://www.sonicwall.com/support/knowledge-base/how-can-i-optimize-pppoe-connections/170505851231244/do you want to run IPS (netmap) things on the WAN interface?
although Bill does not describe the goal (IPS setup) as having the IPS on the WAN interface
(it prefers this to the LAN interface, if you just don't want to observe what's happening on the WAN) -
@DaddyGo said in Low bandwidth on initial install:
I think it's unthinkable, even if it's Chinese hardware, to install 6 pieces of I210 and they don't perform.
I'm very confused.
-
iperf is good at ~920MB/s: testing from within LAN over to WAN then then to the local server.
-
CPU utilization is very low in 2.4.5p1; way better than in their older version; so its not the CPU problem. I'd say its about 3-5% for short burst times.
-
I tried most of the HW teaks and still no change. What the teaks do change is how quickly I can get to top speed of ~400MB/s with speed test.
-
I have replaced psSense setup with Zyxcel USG210 firewall, this one too tops off at ~400MB/s
Any suggestions even where I should look at next?
a) Could it be something else like the SSD that is slow and making things lag?
b) Is speedtest appropriate test for bandwidth? Or is there something else that I can use to reverify the Internet speeds?Thanks
-
-
the case is starting to get a little complicated
so I asked what you want to achieve and where exactly
can you make a system drawing and draw exactly what and where you are measuring?you say that Zyxcel get exactly the same speed, then - this is not a case of the pfSense box and setup
-
the case is starting to get a little complicated
I thought it would be a slam dunk. It never fails. I may try load up pfsense on a real fast BOX, with higher end hardware just to see if I can get up to the 900MB/s range.
so I asked what you want to achieve and where exactly
I want to get rig of the xfinity cable modem/router/wifi device. I used to be able to manage it, but now they are getting more and more restricting. I have Arris modem that I plan to use instead, and I want to couple that with a real firewall setup. I could take the easy way out with just a router, but in today's times I feel I need something more.
My setup is pretty simple actually:
Current:
(Internet From ISP) -> xFinity Router -> WAN of pfSense -> LAN of pfSense -> Test Laptop. Here pfSense WAN is in DHCP mode.Future:
(Internet From ISP) -> Arris modem -> WAN of pfSense -> LAN of pfSense -> 16Port Netgear Switch-> Home computers/devices. Here pfSense WAN will be in DHCP mode as well. No PPPoe.exactly what and where you are measuring?
In 'Current' setup: SpeedTest -> from Test Laptop running speedtest to the internet.
In 'Current' setup: iperf -> from Test Laptop running speedtest to another computer that is connected to the xFinity Router (internal LAN, but on the 'WAN pfSense' side)you say that Zyxcel get exactly the same speed, then - this is not a case of the pfSense box and setup
I did not run the iperf over the Zyxcel yet. It would be interesting to see what the numbers are.
Any other things I should be looking into in the pfSense box?
Thanks!
-
so you have a double -NAT here:
@twoj
Current:
(Internet From ISP) -> xFinity Router -> WAN of pfSense ->try it with Zyxcel too, it will be a good starting point
what did you say your exact ISP speed (subscription)+++++++
https://forum.netgate.com/topic/100945/how-much-throughput-lost-using-pfsense
https://docs.netgate.com/pfsense/en/latest/interfaces/low-throughput-troubleshooting.html -
@DaddyGo said in Low bandwidth on initial install:
What did you say your exact ISP speed (subscription)
1GB/s roughly..... 900MB/s Down and 40MB/s up. I have no problem with hitting 40MB/s upload of course.
try it with Zyxcel too, it will be a good starting point
I did, similar result of about 350-425MB/s. Which makes me think that is the connection between:
xFinity Router -> WAN of (whatever firewall ) is where problem can exists. Possible MTU missmatch? -
we have talked about so much already, about a lot of things
the fact is that you get a bad result with the other firewall too, I understand right?
to me this, definitely seems to be the maximum coming out of the xFinity Router (not normal operation, because your subscription is higher, but you need to check this device)
do one thing more please:
- connect directly to the xFinity Router with a laptop and measure one
PS:
this is not an MTU problem, but this is how you can test
https://kb.netgear.com/19863/Ping-Test-to-determine-Optimal-MTU-Size-on-Router -
we have talked about so much already, about a lot of things
Yes we have! I appreciate all the good info.
the fact is that you get a bad result with the other firewall too, I understand right?
Yes.
to me this, definitely seems to be the maximum coming out of the xFinity Router (not normal operation, because your subscription is higher, but you need to check this device)
Yea... it seems right now that the normal computer connected to xFinity can negotiate at top speed and pfSense and other hardware firewall appliance can not. I'd love to swap out the xFinity modem/router for the Arris right now, but I have to wait until school is over, since my wife teaches from home. I'd hate to be down to some unforeseen problem from xfitiny's side.
I may try running pfSense on beefy computer with enterprise nics in it; just to see what is what.
connect directly to the xFinity Router with a laptop and measure one
Any computer connect to the local LAN is getting top speed ~900MB/s down, but I will retest by directly plugging in.
I WILL GET TO THE BOTTOM OF THIS. :-)
-
it is clear what you need:
xFinity Router in bridge mode, if it exists for this type and your ISP allows it
or you mention a modem (Arris modem) that does not contain NAT per se and you get a public IP directlythe difference between the measurements is very large approx. 900 and 400
we didn't get ahead professionally, because this difference is not justified by the dual -NAT throughput, so there is still a cat hiding somewhere in the bag
if you have the opportunity to exchange, please come back to us afterwards (the curiosity moves the whole world )
