PFSENSE 2.72, can't get the 1gb internet working on 10G LAN
-
Hello.
I have this strange issue:
I bought a Barebone PC Qotom Q1077GE-1U Mini PC 8X 2.5G LAN 16G RAM 128G Msata WiFi,i7 10710U Firewall Router with 8 Intel i226 -V NICs , My Network is a 10G with a Managed Switch TP-Link TL-SX3008F | 8 Port 10G SFP+ Enterprise Level Switch | L2+ Smart Managed | Omada SDN Integrated as the main switch and 5 MikroTik 5-Port Desktop Switch, 4 SFP+ 10Gbps Ports (CRS305-1G-4S+IN) fot a total of 3 PCs with windows server ( file servers ) and 8 PC with windows 11 as render and 3D animation workstations all with SFP+ Intel 5520-2 PCIE 10GB cards.
I have 2 Internet gateways one 300/300 and one 1gb/1gb
After setting up the Mini PC with PFSENSE 2.7.2 the 10Gnetwork is working fine but couldn't get the internet working all PC's have internet access and i am able to do a speed test but once i want to download/upload the speed is just 1.5 kb/s no matter which gateway i use, after lots of testing i decided to connect one pc to internt using the 2.5 LAN integrated in MoBo and PUM ! internet is 100%, 300/300 and 960/960 ... rolled back the pc to use the 10G network to connect to internet and again 1,5kb/s in both gateways...
i then connect a PC laptop using the 1g lan adapter to tp-link switch and internet works flawlessly ....
So bottom line all pc's using 1g integrated lan adapters connect to internet at full speeds, but none of the pc's in the 10G can get more than 1,5kb/s.
My topology:
ISP routers ( 2 ) --> Barebone PC PFSense --> TP-link 10G switch --> 3 pc's directly and 5 Mikrotik 10G switches to distribute to 8 more PC's in another room.
Any Ideas ???
Thanks
JOhanns
-
@Johanns Am I reading it correct that it's only the PC's connected with 10G to the TL-SX3008F that have problems? Or are you saying that all 10G PC's have this problem??
What length and type of cables to you have between switch and PC's? Do they connect fine and run at 10G internally but not over the internet?First off, the TP-Link will not autonegotiate speeds (at least I have never been able to get it to work) so you have to set to 1G manually for any connection not capable of 10G, but I guess you have already done that.
I have tested with Cisco RJ45 modules and they work but I had problems with the unit locking up because it was overheating and ended up putting a Noctua fan in it. Right now I only have DAC's attached to it and it works flawlessly.
And related to that, when using RJ45 modules, make sure not to place them in adjacent cages... -
@Gblenn The problem actually occurs in all the 10G LAN network.
All the PCs are connected to Microtik 10Gb switches using SPF+ cables. The uplink from the Microtiks goes to the TPLink using a direct fiber link between on Microtik and the TPLink (there are several Microtiks because of the type of building that does now allow to run more fiber or cables on walls).
One of the problems we see with the "new pfSense" Qotom, those are intel i226v 2.56gb NICS and the internet speed problem only happens with this device. If we switch back to the PC pfsense (is a big pc and was an overkill just to have as pfsense so that is why it was replaced with the Qotom) that has 1gb incorporated NIC, the speed problems are gone, with no network changes.
I see 2 possible problems:
There is a 10gb transceiver being used to connect the Qotom to the pfSense. This could be causing problems.
the intel i226v are kind of new and not sure if there are still problems with the drivers on pfSense 2.7.2.We were reading about bufferbloat, but I guess this is still not the case because switch to just the old PC with Gigabit only, was not giving this type of problems.
I'm also not sure if the microtik-to-tplink should be a trunk, there are really not vlans just the default one. But it seems that when 10g packages arrive to the pfSense is where thinks get screwed.
Larry
-
@larrygs Ah, so the PC's also have SFP+ ports, not RJ45?
Anyway, sounds like you are on to something there, with the Qotom (2.5G) and 10G (TP-Link) connection. So in that case you have an RJ45 module plugged into the TP-Link and ethernet cable to the Qotom, right?
And that port is set manually to 1G in pfsense as well, or is it set to auto?I have read that there were problems with the i226's but I thought it was fixed in 2.7.2. And one solution is actually to virtualize pfsense on the Qotom (Proxmox) and give it a virtual NIC, instead of a full pass thru. Assuming the drivers in debian are working... You will not have any problems getting the full 1G even in such a setup (actually way more than that with that CPU).
The only issue I have had with the SX3008 is that it doesn't autonegotiate to 1G and that it overheated. But I have not tested with any of my devices that have 2.5G NIC's, as they are connected to a SX3206HPP which works fine @2.5G.