Baffling pfSense 2.6.0 Issue (10G Performance)
-
It shows that pfSense is a bad TCP endpoint but that is known. It's configured to be a router not a server. Testing to or from pfSense directly always gives poor results. It's a useful test to prove it's linked at >1Gbps but not much else.
It also shows the receive side is significantly more processor intensive.The UDP test shows zero packet loss at 10G line rate?
Steve
-
Packet loss per iperf UDP test is about 0.15% both ways.
Also, enabling flow control looked like a red herring, I'm back at a steady 4.3 Gbps. I also noticed the same thing when enabling powerD - seemed like the problem was fixed but then after a few hours the problem reappeared. Something to do with changing settings and rebooting?
I guess if you don't have any other thoughts I'll reconfigure to use the x552 for both LANs and see what happens -- I'm getting desperate!
-
Yeah, I would try to test using the x500 for both if you can.
If you simply reboot does it pass at full speed for some time?
Steve
-
A reboot doesn't change anything. But, I did notice with the powerD change, and then with the flow control change, throughput was restored for a short time. I've made tons of tunable changes with reboots that have had no impact on throughput.
I'll reconfigure the x500s and report back. Thanks again for helping me.
-
Steve - you were correct, moving both LANs to the x552s solved the problem. I now get 9.8 Gbps both ways on iperf single stream and real-world file transfers are at 10G speeds. I am very grateful for your help!
But, any guess as to why the x710 doesn't perform? I now need to use the x710 for WAN connections so I still have an interest in sorting this out.
-
Hmm, well it's almost certainly some change in the ixl driver.
What exact card are you using? How does it appear in
pciconf -lv?Are you able to test a 2.7 snapshot? It's possible this has already been solved.
Steve
-
In pciconf is appears as "Ethernet Controller X710 for 10GbE SFP+"
I'll give 2.7 a try and report back.
-
Do you have the actual PCI IDs shown? It could be something specific to the chip.
-
We see the same odd performance on several NIC and its related to cache and writes on the hardware.
-
Hi Steve,
Here is the full output:ixl0@pci0:6:0:0: class=0x020000 card=0x02581374 chip=0x15728086 rev=0x02 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller X710 for 10GbE SFP+'
class = network
subclass = ethernetixl1@pci0:6:0:1: class=0x020000 card=0x00001374 chip=0x15728086 rev=0x02 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller X710 for 10GbE SFP+'
class = network
subclass = ethernet -
Thank you -- can you point me to any online discussion/thread where this is discussed so I can follow along?
