Installing pfSense on XG105Rev3
-
@sysadminfromhell
i changed a few things in the /boot/loader.conf as mentioned in a different Post with a Intel i211 NIC:kern.cam.boot_delay=10000 # agree with Intel license terms legal.intel_ipw.license_ack=1 legal.intel_iwi.license_ack=1 # This is the magic. If you don't set this, queues won't be utilized properly. # Allow multiple processes for receive/transmit processing. hw.igb.0.rx_process_limit="-1" hw.igb.0.tx_process_limit="-1" hw.igb.1.rx_process_limit="-1" hw.igb.1.tx_process_limit="-1" hw.igb.2.rx_process_limit="-1" hw.igb.2.tx_process_limit="-1" hw.igb.3.rx_process_limit="-1" hw.igb.3.tx_process_limit="-1" #net.pf.states_hashsize=2097152 hw.igb.0.rxd=4096 hw.igb.0.txd=4096 hw.igb.1.rxd=4096 hw.igb.1.txd=4096 hw.igb.2.rxd=4096 hw.igb.2.txd=4096 hw.igb.3.rxd=4096 hw.igb.3.txd=4096 #net.inet.tcp.syncache.hashsize="1024" #net.inet.tcp.syncache.bucketlimit="100" kern.ipc.nmbclusters="1000000" kern.ipc.nmbjumbop="524288" kern.ipc.nmbjumbo9="524288" autoboot_delay="3" net.link.ifqmaxlen="128"but this didnt change a thing (maybe outdated?)
I updated to 2.6.0 (2.6.0.r.20220124.1828) as someone mentioned better beformance with Nic I211 as in the XG105Rev3.But at all the performance between the interfaces remains slow.
I mean in one way (igb1->igb2) the other way seems okay.
Whats weird is, that when i iperf3 agains the IP of igb2 its also slow. So the Routing between igb1->igb2 seems to be slow.maybe someone has an idea in part of "Intel Nic I211" or something, I dont know :(
-
Mmm, well I'd expect at least 600Mbps from that. Try running
top -aSHat the same time and see if one of the cores is maxed out. -
@stephenw10
Hi there, thanks for the reply, I tried that but the cpu is not maxing out. It sticks at around 20% while the test is running -
On both cores?
-
@stephenw10
Hi there, will check this again soon but yes afaik it was on both cores 20% usage. -
Hmm, I would expect more usage than that on that CPU at 600Mbps.
-
@stephenw10 hi there i checked again and I am at 91%. sorry for the misinformation around there.
-
@stephenw10 but the main problem remains: why is the network speed between 2 networks (2 nics) so slow?
when i check iperf3 from igb1(192.168.178.0/24) to igb2(172.16.24.0/24) its super super slow (3,1 MBit/s) and the CPU usage on the pfSense is low (about 5%, peek to 20/25%).
Is there something i can do/tune/change in the setup? -
That sort of throttling is almost certainly one of the NICs not linked correctly. When you see less that 100Mbps on what should be a Gigabit link it's usually because the autonegotiation has failed and something is running at the fallback default of 100M half-duplex. So check the ifconfig output on pfSense and check the link speed at each host.
Steve
-
@stephenw10 i forced it via pfsense to 1000Tbase Full duplex but still the same issue:
Connecting to host 172.16.24.100, port 5201 [ 4] local 192.168.178.110 port 52362 connected to 172.16.24.100 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.01 sec 256 KBytes 2.08 Mbits/sec [ 4] 1.01-2.00 sec 384 KBytes 3.17 Mbits/sec [ 4] 2.00-3.01 sec 128 KBytes 1.04 Mbits/sec [ 4] 3.01-4.00 sec 256 KBytes 2.11 Mbits/sec [ 4] 4.00-5.01 sec 512 KBytes 4.16 Mbits/sec [ 4] 5.01-6.00 sec 256 KBytes 2.11 Mbits/sec [ 4] 6.00-7.01 sec 256 KBytes 2.08 Mbits/sec [ 4] 7.01-8.00 sec 128 KBytes 1.06 Mbits/sec [ 4] 8.00-9.01 sec 384 KBytes 3.12 Mbits/sec [ 4] 9.01-10.01 sec 384 KBytes 3.17 Mbits/sec [ 4] 10.01-11.01 sec 512 KBytes 4.16 Mbits/sec [ 4] 11.01-12.01 sec 384 KBytes 3.17 Mbits/sec [ 4] 12.01-13.02 sec 384 KBytes 3.12 Mbits/sec [ 4] 13.02-14.01 sec 256 KBytes 2.11 Mbits/sec [ 4] 14.01-15.00 sec 512 KBytes 4.22 Mbits/sec [ 4] 15.00-16.01 sec 640 KBytes 5.20 Mbits/sec [ 4] 16.01-17.00 sec 256 KBytes 2.11 Mbits/sec [ 4] 17.00-18.01 sec 384 KBytes 3.12 Mbits/sec [ 4] 18.01-19.00 sec 256 KBytes 2.11 Mbits/sec [ 4] 19.00-20.01 sec 256 KBytes 2.08 Mbits/sec [ 4] 20.01-21.00 sec 384 KBytes 3.17 Mbits/sec [ 4] 21.00-22.01 sec 512 KBytes 4.16 Mbits/sec [ 4] 22.01-23.01 sec 384 KBytes 3.17 Mbits/sec [ 4] 23.01-24.01 sec 256 KBytes 2.08 Mbits/sec [ 4] 24.01-25.01 sec 384 KBytes 3.17 Mbits/sec [ 4] 25.01-26.00 sec 384 KBytes 3.17 Mbits/sec [ 4] 26.00-27.01 sec 384 KBytes 3.12 Mbits/sec [ 4] 27.01-28.00 sec 384 KBytes 3.17 Mbits/sec [ 4] 28.00-29.01 sec 384 KBytes 3.12 Mbits/sec [ 4] 29.01-30.00 sec 256 KBytes 2.11 Mbits/sec [ 4] 30.00-31.01 sec 384 KBytes 3.12 Mbits/sec [ 4] 31.01-32.00 sec 384 KBytes 3.17 Mbits/sec [ 4] 32.00-33.01 sec 512 KBytes 4.16 Mbits/sec [ 4] 33.01-34.00 sec 512 KBytes 4.22 Mbits/sec [ 4] 34.00-35.01 sec 256 KBytes 2.08 Mbits/sec [ 4] 35.01-36.01 sec 128 KBytes 1.06 Mbits/sec [ 4] 36.01-37.01 sec 128 KBytes 1.04 Mbits/sec [ 4] 37.01-38.01 sec 640 KBytes 5.28 Mbits/sec [ 4] 38.01-39.00 sec 384 KBytes 3.17 Mbits/sec [ 4] 39.00-40.01 sec 384 KBytes 3.12 Mbits/sec [ 4] 40.01-41.00 sec 512 KBytes 4.22 Mbits/sec [ 4] 41.00-42.01 sec 256 KBytes 2.08 Mbits/sec [ 4] 42.01-43.00 sec 256 KBytes 2.11 Mbits/sec [ 4] 43.00-44.01 sec 384 KBytes 3.12 Mbits/sec [ 4] 44.01-45.00 sec 512 KBytes 4.22 Mbits/sec [ 4] 45.00-46.01 sec 640 KBytes 5.20 Mbits/sec [ 4] 46.01-47.01 sec 256 KBytes 2.11 Mbits/sec [ 4] 47.01-48.01 sec 128 KBytes 1.04 Mbits/sec [ 4] 48.01-49.01 sec 640 KBytes 5.28 Mbits/sec [ 4] 49.01-50.00 sec 256 KBytes 2.10 Mbits/sec [ 4] 50.00-51.01 sec 384 KBytes 3.13 Mbits/sec [ 4] 51.01-52.00 sec 384 KBytes 3.17 Mbits/sec [ 4] 52.00-53.01 sec 256 KBytes 2.08 Mbits/sec [ 4] 53.01-54.00 sec 128 KBytes 1.06 Mbits/sec [ 4] 54.00-55.01 sec 512 KBytes 4.16 Mbits/sec [ 4] 55.01-56.00 sec 384 KBytes 3.17 Mbits/sec [ 4] 56.00-57.01 sec 640 KBytes 5.20 Mbits/sec [ 4] 57.01-58.00 sec 256 KBytes 2.11 Mbits/sec [ 4] 58.00-59.01 sec 512 KBytes 4.16 Mbits/sec [ 4] 59.01-60.01 sec 128 KBytes 1.06 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-60.01 sec 21.1 MBytes 2.95 Mbits/sec sender [ 4] 0.00-60.01 sec 21.0 MBytes 2.93 Mbits/sec receiverEDIT: all my hosts are at 1GBit (1000M) the whole time. My switches even say, that the Uplink to the router (pfsense) is 1000M. I forced it but still no change.
-
@sysadminfromhell Well, have you checked the obvious?
Cables/patch cords.
There isn't much to tune in any case and intel cards and pfsense are considered perfect matches -
Hmm, was anything forced before? Forcing the link speed is usually the cause of these problems. Gigabit Ethernet requires autoselect
-
@stephenw10 no nothing was forced before.
-
@netblues yes i checked everything and even changed the cables of the hosts and the switch to the pfsense but the problem remains
-
Errors shown on the interfaces?
Throttling to 3Mbps almost has to be something at layer 1....
-
@stephenw10 where I can check that?
-
In Status > Interfaces is easiest.
-
@stephenw10 no errors in/out
-
@stephenw10 good morning, I changed the igb from 2 to 3 to check if the port is damaged/broken but the problem remains the same. I will do the following today:
As this subnet has the same as the ISP router gives the pesense gives igb0 (172.16.1.10/29) I will change this subnet from the ISP to something else like 10.0.1.0/29 or something like this and see if this problem remains. I do not know if this interfering in any way but let’s see, shall we?Edit: I updated the subnet from the ISP router to: 10.0.1.252/30 so the igb0 now has the IP 10.0.1.254. I testet the performance and Download is now OK (300 MBit - 980MBit) but Upload remains the same problem: 3,14MBit/s.
-
Hmm, OK changing the WAN subnet should make absolutely no difference to traffic between internal subnets unless there were a conflict caused by dhcp.
So that suggests something is not configured as expected. Like those interfaces are somehow linked via some other path. Are they bridged? Are they even direct ports on that device?Steve
