2.5 Gbps Hardware

stephenw10

@jsmiddleton4 said in 2.5 Gbps Hardware:

I'm getting 1300+mbs via Fast.com

Nice! I can only dream of that here.

When you bridged the NICs did you assign the bridge and put the DHCP server on that?
If not you might hit the issue where if the interface with it enabled is disconnected everything else in the bridge will fail.

So if you power cycle pfSense it boots back up to the interfaces assign screen?
But not if you just reboot?
That starts to sound like some low-level issue. Is the BIOS current on that box?

You can run pciconf -lv any time. It just reads the devices, it doesn't anything. My edge device for example:

[2.5.2-RELEASE][admin@pfsense.fire.box]/root: pciconf -lv
hostb0@pci0:0:0:0:	class=0x060000 card=0x04101106 chip=0x04101106 rev=0x80 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series Host Bridge: Host Control'
    class      = bridge
    subclass   = HOST-PCI
hostb1@pci0:0:0:1:	class=0x060000 card=0x14101106 chip=0x14101106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series Error Reporting'
    class      = bridge
    subclass   = HOST-PCI
hostb2@pci0:0:0:2:	class=0x060000 card=0x24101106 chip=0x24101106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series CPU Bus Controller'
    class      = bridge
    subclass   = HOST-PCI
hostb3@pci0:0:0:3:	class=0x060000 card=0x34101106 chip=0x34101106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series DRAM Bus Control'
    class      = bridge
    subclass   = HOST-PCI
hostb4@pci0:0:0:4:	class=0x060000 card=0x44101106 chip=0x44101106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series Power Management and Chip Testing Control'
    class      = bridge
    subclass   = HOST-PCI
hostb5@pci0:0:0:5:	class=0x060000 card=0x54101106 chip=0x54101106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series APIC and Central Traffic Control'
    class      = bridge
    subclass   = HOST-PCI
hostb6@pci0:0:0:6:	class=0x060000 card=0x64101106 chip=0x64101106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series Scratch Registers'
    class      = bridge
    subclass   = HOST-PCI
hostb7@pci0:0:0:7:	class=0x060000 card=0x74101106 chip=0x74101106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series North-South Module Interface Control'
    class      = bridge
    subclass   = HOST-PCI
vgapci0@pci0:0:1:0:	class=0x030000 card=0x71221106 chip=0x71221106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Graphics [Chrome9 HD]'
    class      = display
    subclass   = VGA
pcib1@pci0:0:3:0:	class=0x060400 card=0xa4101106 chip=0xa4101106 rev=0x00 hdr=0x01
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series PCI Express Root Port 0'
    class      = bridge
    subclass   = PCI-PCI
pcib2@pci0:0:3:1:	class=0x060400 card=0xb4101106 chip=0xb4101106 rev=0x00 hdr=0x01
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series PCI Express Root Port 1'
    class      = bridge
    subclass   = PCI-PCI
pcib3@pci0:0:3:2:	class=0x060400 card=0xc4101106 chip=0xc4101106 rev=0x00 hdr=0x01
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series PCI Express Root Port 2'
    class      = bridge
    subclass   = PCI-PCI
pcib4@pci0:0:3:3:	class=0x060400 card=0xd4101106 chip=0xd4101106 rev=0x00 hdr=0x01
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series PCI Express Root Port 3'
    class      = bridge
    subclass   = PCI-PCI
hostb8@pci0:0:3:4:	class=0x060000 card=0x00000000 chip=0xe4101106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series PCI Express Physical Layer Electrical Sub-block'
    class      = bridge
    subclass   = HOST-PCI
none0@pci0:0:11:0:	class=0x028000 card=0xa4091106 chip=0xa4091106 rev=0x10 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX855/VX875/VX900 Series USB Device Controller'
    class      = network
sdhci_pci0@pci0:0:12:0:	class=0x080501 card=0x95d01106 chip=0x95d01106 rev=0x10 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX800/820/900 Series SDIO Host Controller'
    class      = base peripheral
    subclass   = SD host controller
none1@pci0:0:13:0:	class=0x050100 card=0x95301106 chip=0x95301106 rev=0x61 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX800/820/900 Series Secure Digital Memory Card Controller'
    class      = memory
    subclass   = flash
atapci0@pci0:0:15:0:	class=0x01018f card=0x90011106 chip=0x90011106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series Serial-ATA Controller'
    class      = mass storage
    subclass   = ATA
uhci0@pci0:0:16:0:	class=0x0c0300 card=0x30381106 chip=0x30381106 rev=0xa0 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VT82xx/62xx/VX700/8x0/900 UHCI USB 1.1 Controller'
    class      = serial bus
    subclass   = USB
uhci1@pci0:0:16:1:	class=0x0c0300 card=0x30381106 chip=0x30381106 rev=0xa0 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VT82xx/62xx/VX700/8x0/900 UHCI USB 1.1 Controller'
    class      = serial bus
    subclass   = USB
uhci2@pci0:0:16:2:	class=0x0c0300 card=0x30381106 chip=0x30381106 rev=0xa0 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VT82xx/62xx/VX700/8x0/900 UHCI USB 1.1 Controller'
    class      = serial bus
    subclass   = USB
uhci3@pci0:0:16:3:	class=0x0c0300 card=0x30381106 chip=0x30381106 rev=0xa0 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VT82xx/62xx/VX700/8x0/900 UHCI USB 1.1 Controller'
    class      = serial bus
    subclass   = USB
ehci0@pci0:0:16:4:	class=0x0c0320 card=0x31041106 chip=0x31041106 rev=0x90 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'USB 2.0 EHCI-Compliant Host-Controller'
    class      = serial bus
    subclass   = USB
isab0@pci0:0:17:0:	class=0x060100 card=0x84101106 chip=0x84101106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX900 Series Bus Control and Power Management'
    class      = bridge
    subclass   = PCI-ISA
hostb9@pci0:0:17:7:	class=0x060000 card=0xa3531106 chip=0xa3531106 rev=0x00 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX8xx/900 Series South-North Module Interface Control'
    class      = bridge
    subclass   = HOST-PCI
pcib5@pci0:0:19:0:	class=0x060401 card=0xb3531106 chip=0xb3531106 rev=0x00 hdr=0x01
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VX855/VX875/VX900 PCI to PCI Bridge'
    class      = bridge
    subclass   = PCI-PCI
igb0@pci0:1:0:0:	class=0x020000 card=0x000015bb chip=0x15108086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82580 Gigabit Backplane Connection'
    class      = network
    subclass   = ethernet
igb1@pci0:1:0:1:	class=0x020000 card=0x000015bb chip=0x15108086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82580 Gigabit Backplane Connection'
    class      = network
    subclass   = ethernet

Steve

jsmiddleton4

@stephenw10

Bios current. A14. Updated yesterday.

I’ve totally snarfed everything trying to get IPV6 to work.

Had to reinstall 2.6 and now I can’t get it to accept a DHCP range for the server. Keeps telling me I’m trying to use ranges that are outside the subnet. When they are the same numbers as I had working before.

Had to pull the PC server out and go back to just router setup. I can’t have family mbrs down.

stephenw10

If it says the range is outside the subnet then it is. Perhaps you changed the subnet and didn't apply that change?

We'd have to see screenshots to speculate further.

That's not a 2.5G NIC issue though. You should open a new thread for that.

Steve

jsmiddleton4

@stephenw10

Cleared it by booting.

I'm still majorly stuck on IPV6.

I think I have DNS stuff sorted.

I've been at this too long today. Thanks for all your help. I'm reading through the trouble shooting FAQ's on the pfsense forum. Problem is there's so many options I can't remember what I changed to make it work, to make it not work, etc.

ChanceTran

Hi....past 1Gb ISP gadget's typically serves the bw through different ports, a.k.a. 2Gb organization implies you can interface with two home gadgets at the same time and influence 1+1Gb. So the contention to coordinate with the home organization adjusted to higher bw still legitimate (shockingly for some time perceiving the amount 10G home switches, network cards cost), however in case you're glad to straightforwardly interface two gadgets.

jsmiddleton4

@chancetran

PFSense 2.5.2 works fine with the Intel 2.5gb cards. Just need to check the option to disable checksum in Advanced, Network settings.

I tried 2.6.0 simply because of the “better” NIC support. It has other issues though like getting “dynamic” for the IPV6 gateway address instead of the IPV6 address. So IPV6 didn’t work as intended.

2.5.2 IPV6 works great.

jsmiddleton4

Any other 2.5gb NIC chipsets supported now?

jsmiddleton4

New problem.....

Updated to 2.7.0.

1gb clients connected to two newly added 2.5gb NIC's are not getting IPV4/IPV6 information so they obviously can't connect.

When plugging those clients into a 1gb router which is connected via 2.5gb ports, the 1gb clients work fine.

Problem only shows up with those 1gb clients are connected directly to the 2.5gb cards.

I've tried manually setting speeds. No change.
|
All 2.5gbe clients connected to any, including the 2 new ones, 2.5gbe NIC's on the PFSense box work great.

I did unclick Disable hardware checksum offload when moving from 2.5.2 to 2.6.0. Which is still unclicked.

I can't click it and reboot at the moment. People using it.

I've checked gateways, firewall rules, etc., for the new NIC's. Feels like I'm missing some option.

Even more confusing, at least to me, if I connect either 1gb client to the built in LAN port in the PFSense box, they connect fine. Same rules, etc.

Has to be something about the 2.5gb NIC's not falling back to 1gb properly???

Or a horse of a different color?

jsmiddleton4

Not saying this is the "right" way to set this up but did make it work.

My NIC's are bridged.

Formerly had the tunable set to use the bridge interface.

net.link.bridge.pfil_member Packet filter on the member interface 0
net.link.bridge.pfil_bridge Packet filter on the bridge interface 1

Had one rule in the firewall for the LAN.

I changed it to filter on the member.

net.link.bridge.pfil_member Packet filter on the member interface 1
net.link.bridge.pfil_bridge Packet filter on the bridge interface 0

I added rules for each member, just took a second, in the firewall. Left the existing LAN rule in place.

Every client is happy now regardless of which port, 1gb or 2.5gb, connected to on the PFSense box.

I've checked each rule and there is data transmission through each, LAN and members.

While it works, not saying its right and I still feel like I've missed something in terms of a setting by adding additional 2.5gb cards to the box/bridge.

stephenw10

If you have people using and depending on that box you should not be running 2.7-Alpha. Yopu should go back to the 2.6-RC branch when you can.

Steve

A Former User

@stephenw10, I totally agree with you, because if they know they need the system 100%, do they use development versions?

jsmiddleton4

@stephenw10

Had I known 2.7.0 was an alpha version would've stayed clear.

Any insight on what happened because I put 2 new 2.5gb NIC's in?

stephenw10

If it was in addition to the existing 2.5G NICs it probably re-ordered them and the old interfaces had pass rules which then no longer applied.

But without seeing what was happening at the time that's just a guess.

jsmiddleton4

@stephenw10

It did of course reassign them. igc0 became ig4, etc.

I put all the NIC's in the bridge list and saved it.

Would make sense if all clients couldn't connect. It was just the 1gb ones.

Even if some assignment got confused in the firewall rules for the bridge lan, how would 2.5gb clients get through?

Edit: Additional piece just found out. The two that would not connect are work PC's. They are configured to use only IPV6. They connect to the employers VPN. Only show "Internet Connectivity" with IPV6 even though IPV4 is present in their NIC's Status information.

stephenw10

Ok, then I'd guess this is an IPv6 issue and not a 1G vs 2.5G problem.

jsmiddleton4

@stephenw10

As noted connected to the 1gb port on the same box, the built in Realtek LAN port, worked fine.

"re0" woulda stayed the same though......I didn't put more Realtek based cards in.

Now that I have, like I did before and removed them, rules for each card, member of the bridge, can I remove the LAN rule I created?

The Asus AX86U which worked when those two clients were connected to its 1gb ports, what I didn't do was plug one of those clients into the 2.5gb port the Asus AX86U was connected to.

That's the one 2.5gb port I didn't test.

stephenw10

If you moved filtering from the bridge interface to the member interfaces then you don't need a rule on the assigned bridge interface, but it doesn't hurt to leave it there.

jsmiddleton4

@stephenw10

Thanks. It is getting traffic at least the counter is showing so.

jsmiddleton4

@stephenw10

For kicks I set everything back the way it was for the tunable to track the bridge, not member interfaces, and had just the LAN firewall rule enabled.

It works for all clients.