NAT overhead
-
Does it make sense that my ethernet connected machines (3) drop to about 150 Mbs when connected via the Netgate 3100, but if I hook any one of them up to the cable modem WAN feed the get over 800 Mbs?
My service with Spectrum is supposed to be gigabit. Seems as if I'm not quite getting that, but I am getting much better than what I see when I introduce the Netgate 3100 as my NAT router.
Is that kind of overhead cost expected with the PfSense + as my firewall and NAT routing?
-
@dsegui As far as I know, you should be able to get much faster speeds on a 3100, but I guess that also depends on how many firewall rules you have and if you have any high CPU usage packages installed such as NtopNG, etc. When you are transferring something that appears speed limited, can you show us the output of DIAGNOSTICS/System Activity. This will show if the CPU appears saturated by something.
-
@USSZulu Thanks for the reply. When I have some time I'll run the bandwidth test that I have been using and try to capture the stats that you mentioned and post them up here. My use of the 3100 is really minimal.
I haven't created firewall rules and don't run anything other than PfSense on the router. I bought it thinking that I might set it up to offer VPN access from outside the house but that need never materialized. So the 3100 is really much more router than I need which is why it's frustrating to see the slow throughput.
-
@USSZulu Here are screen captures from the System Diagnostics/System Activity page. The 'baseline' capture was with nothing going on. The 'during bandwidth test' was taken during the running of the bandwidth test.
-
@dsegui re: negative idle %, that's a known issue:
https://redmine.pfsense.org/issues/11473"On the SG-3100 the first output from 'top -aSH' shows invalid data for system idle usage.
Subsequent output is correct but because the System Activity page uses single count output it is always incorrect"re: speed, we have a client currently running a 3100 on a gigabit fiber and when I ran a quick speed test during setup, during the day, it was around 800-900 Mbps up and down.
150 Mbps would be higher than a 100 Mbps port speed so that's not it. Is the port set to full duplex? What if you put a small switch between the 3100 and the ISP router?
re: the idle bug, I doubt they will fix it at this point...in case you hadn't heard FreeBSD will not be releasing future software for 32 bit ARM so 23.09 might be the last version of pfSense for it. They sent out an email a month or so ago.
-
@SteveITS Thanks, Steve. If I'm understanding what you said, it sounds as if you are thinking that I am using an ISP supplied router. That's not the case.
The ISP hardware is just the cable modem. The ethernet feed out of that is going directly to the 3100. As a diagnostic to determine the bandwidth that the ISP is providing, I connected that ethernet feed out of the cable modem directly, one at a time, to each of my three ethernet connected computers. When I did that I was able to get bandwidth of 800Mps+ in each case. But my normal configuration is to have that ethernet feed going directly to the 3100.
I'll look for the duplex configuration option that you mentioned and see how it is set. Thanks for the suggestions.
-
@dsegui I was generalizing, sorry, try the switch between the 3100 and the cable modem then. It's a long shot but I've seen others say that fixes weird port issues. Also check status/interfaces for error counts.
-
@dsegui said in NAT overhead:
one at a time, to each of my three ethernet connected computers
So you rebooted the cable modem each time? All of the cable modems have ever seen requires them to be rebooted when changing the device connected to them. Unless you were cloning the mac address.
Unless you have a gateway device and its actually doing nat?
Nat overhead is not going to be anything you should ever be able to notice from a speed point of view.
I have an older sg4860 and is more than capable of handling 500mbps, have seen upwards of 700.. From my isp 500/50 plan. Your 3100 should be able to see 800..
If your only seeing 150ish, something is not right that is for sure.. Had you done anything with shaping in pfsense?
As mentioned by @SteveITS its possible you could be having some sort of duplex mismatch.. That for sure would put your speed into the dirt.. Are you running anything else in pfsense like IPS? Or captive portal? Ntop? Things that hook into the interfaces can have performance issues..
-
@johnpoz said in NAT overhead:
shaping in pfsense
Ah, yes, I always forget that, and for my above 3100 that was actually the case. It had shaping to prioritize voice traffic from the 75 Mbps connection and it threw me for a second I was only getting that on gigabit.
-
@johnpoz Yes - I did have to reboot the cable modem to test each direct connection.
-
@SteveITS said in NAT overhead:
Is the port set to full duplex?
Steve - I have checked the configuration on the LAN ports of the 3100. Each is configured for "Default (no preference, typically autoselect)".
I did notice that one of the three active ports showed a status of "Ethernet autoselect (1000baseT <full-duplex,master>)" while the other two were "Ethernet autoselect (1000baseT <full-duplex>)". What is the significance of the 'master' suffix?
-
@dsegui there is a master slave setup with gig, which is why your not suppose to hard code it.
If I recall correctly the master clocking comes from the local source, while the "slave" uses the loop timing.. Been really long time since dove into that stuff. It shouldn't really matter which side of the connection is master or slave.
One side of the connection will be master and provide the clock source, the other will be slave. Some setups might show you if master or not, etc. some might not show it and just show the gig connection, yeah its going to be full duplex.
-
I'm sure you have tested for this but as the circa 150 Mbps speed figure has a physical significance (indicative of an ethernet pair-to-pair short) can you just confirm that you have checked cables and the physical ports that they are connected to?
️ -
@RobbieTT Haven't messed with the cables yet, other than confirming they are either cat-5e or cat6. Swapping in a new, known good cable (like the one I used for the direct connect testing) would be a good way to rule out bad cabling.
-
@RobbieTT Just realized that I can't use the same cable as I did for the direct connection tests because that one comes up from the cable modem in the basement. It's a long run, all tacked up along the way. Not something I'm willing to pull down just to do a test.
But I did have another unused cat-5e so I replaced it for one of the runs from the 3100 to a computer. No change in throughput with that. The cable coming up from the basement, that delivered the 800 Mbs+ when directly connected to a computer, is also cat-5e. So I know that a cat-5e cable can carry the higher bandwidth.
-
@dsegui 5E is fine for gigabit.
Did you look into the traffic shaping suggested above? Sometimes people forget they set it up 8 years ago until they change ISPs.
Along those lines, temporarily reset your 3100 to defaults and see if it still occurs. You can always restore the config again from your backup.
-
@SteveITS Resetting to factory defaults is a good idea, and I'll do that. But honestly, I just set this thing up using the wizard. I haven't added any firewall rules or 'shaping'. About the only configuration change I made was to stipulate the DNS services to use rather than accepting what the my ISP's DHCP suggests because I didn't what to be using Spectrum's DNS servers. But I have tried undoing that configuration and it made no difference.
When I had 400 Mbs service from Time Warner (the ISP now known as Spectrum) it didn't bother me so much that my throughput was just over 1/3 of that rating. But when I upgraded to gigabit service and didn't see an improvement is when I began to question things. And when I saw that I can get close to gigabit service by avoiding the 3100 is when I started this thread.
I'll report back on whether a return to factory defaults has any effect.
-
@dsegui said in NAT overhead:
About the only configuration change I made was to stipulate the DNS services to use rather than accepting what the my ISP's DHCP suggests
That is not really how pfsense works out of the box, out of the box even if your ISP from dhcp hands you dns - pfsense would be resolving. It wouldn't be forwarding dns to anywhere. even if your isp ones were listed - they wouldn't used unless the local unbound failed. And then that would only be for pfsense own use, if unbound wasn't running because it failed for some reason - clients dns would fail.
-
@dsegui said in NAT overhead:
) it didn't bother me so much that my throughput was just over 1/3 of that rating
So you have been getting low throughput for a long time then? If was paying for 400, and only getting like 100 something I would be complaining or digging into why that is for sure.
90%, ok during prime time 80% of what I pay for - but 30% yeah I would be digging into why that was for damn sure..
But a 3100 should be able to do 900s - i think there is a lawrence teardown and review when it first came out showing benchmarks in the 900s..
If your seeing 150ish - yeah got something wrong that is for sure.. You could take your isp out of the equation for sure.. Put something on your pfsense wan running iperf, and then from a client on the wan do a benchmark - this would be doing nat, etc.
