NAT'ed LAN to pfSense box very slow after hardware upgrade

cFire

Hello,

I got some newer hardware for my pfSense box but I'm running into an odd issue;
Traffic from the LAN to the pfSense box and from the LAN to the WAN is very slow. (About 100 Mbit speed between the pfsense box and any machine in the LAN, even though it's a gigabit link. Between 2Mb/s and 1Kb/s to the WAN.)
Traffic from the pfSense box to the WAN is normal. (What it's always been, close to 100 mbit.)
It uses the exact same two PCI NICs for the LAN and the WAN interface, so there should be no firmware issues. I also transplanted the NICs from the old box, where everything was working fine.
I imported the config .xml from the old setup too so in theory that should not be the issue either, but I do suspect some software/routing condition to be causing this.

In short: I'm a little confused as to what is causing this.
Does anyone have any suggestions for what I might try?

muswellhillbilly

I've seen this effect occur when you have a NIC and the port it's connected to both set to auto-negotiate the speed and duplex. If you're using a managed switch you might want to check to see if the speed on the port is hard-set or set to 'auto'. Same on the network card.

cFire

I've set the link speeds/duplex manually in pfsense and it does seem to be a lot better now. I'm not sure off hand if it's really fixed yet. I'll keep you updated.

muswellhillbilly

Good to hear it's improved. If this does fix the issue, do post a confirmation - all grist to the mill, as they say.

cFire

Update: It's definitely a lot better now with speeds/duplex set manually, but it's not fixed.
It's still especially noticeable when doing transfers from the pfsense box to a machine in the LAN. It's reaching nowhere near gigabit speeds. More like 120-150 mbit right now.

muswellhillbilly

I take it you're testing the speed internally from one particular host machine to the pfSense. Have you confirmed the speed issue exists on more than one PC/host? And have you checked the speed set on the ports your using on your switch? And what speed does the NIC on your internal PC show? If you run an 'ifconfig' in a command shell on the pfSense, what speed does the internal (LAN) NIC show on the output?

cFire

@muswellhillbilly:

I take it you're testing the speed internally from one particular host machine to the pfSense. Have you confirmed the speed issue exists on more than one PC/host?

Yes, the issue is the same on all internal hosts, I've tested from these hosts to the router and between the hosts, only the connection to the router is slow. Between the hosts I get normal gigabit speeds.

@muswellhillbilly:

And have you checked the speed set on the ports your using on your switch? And what speed does the NIC on your internal PC show? If you run an 'ifconfig' in a command shell on the pfSense, what speed does the internal (LAN) NIC show on the output?

All hosts and the router report 1000baseT. The switch is unmanaged so I can't check what the ports there are reporting, however given that traffic between hosts goes over this same switch and does reach expected speeds, it's unlikely there is an issue with the switch.

I was also getting better speeds before the router hardware upgrade so this would mean the uplink port would have broken in the time between the upgrade. Possible, but not terribly likely. I'll try switching the uplink port to make absolutely sure.

muswellhillbilly

@cFire:

I've tested from these hosts to the router and between the hosts, only the connection to the router is slow. Between the hosts I get normal gigabit speeds.

Just so we're clear here, are you actually testing the speed between your internal hosts and the pfSense machine, between your internal hosts and your router or between your internal hosts and an external point on the internet? There is a difference - lag between your hosts and the pfSense might imply a problem with the port the pfSense machine is using or possibly with the network card on the pfSense, whereas lag between a host and your router could imply an issue with the port(s) between the pfSense and router. And how are you testing the speed issue?

cFire

After some more testing it seems the NIC was somehow damaged during the hardware upgrade. Even though it worked fine in the old machine, by the time it was in the new one it had broken.
Replacing the NIC with a new one has completely resolved the issue and link speed does not need to be set manually anymore either.

Thanks for your thoughts and support! It is greatly appreciated.