Speed Limited to about 28 Mbps?

hackin8

Changed interrupts (didn't need to disable anything - used 9 and 10 that were already free). Didn't make any difference to speed but will have to wait and see whether it stops the (very occasional) hardware failure I asked about in another thread. Interestingly - now the LAN interface is completely clean - where before it was showing a few collisions.

Still no success with speed - despite trying different network cards. So ATM my belief is that it may still relate to inability to set maximum MTU - although it still seems surprising it has so much effect.

Am planning to try iperf - to see whether the pfs setup I have is at least capable of higher speeds. Will take a little while to setup and test - will report back later.

Thanks again

hackin8

Managed to run iperf, results as follows:

Direct LAN PC1-PC2 (No pfs) 94Mbps - as expected on 10/100 NICS
PC1-pfs-PC2 50 Mbps - very consistent +/- 0.5Mbps (Using PC2 effectively as WAN)

Then using speedtest.net (Java and file DL)

PC1-PPPoE 77Mbps
PC1-pfs-PPPoE 24-28 Mbps

So - pfs is certainly capable of higher speeds - ran iperf continuously for 10 mins at 50Mbps and no sign of overloading / dropped packets etc

Seems to me that problem must relate to PPPoE implementation and/or MTU? pfs and/or BSD?

I am now out of ideas - will have to live with 28 Mbps unless any further suggestions / ideas :)

stephenw10

Check the WAN NIC is set to auto negotiate the media type and that it is negotiating correctly. I would guess you have 100Mb half duplex connection perhaps.

Steve

hackin8

Have double checked - It is set to auto - currently running at 100 full duplex.

hackin8

Final (?) request.

Based upon testing so far it seems the only thing left is the MTU. I gather (from other threads) that the MTU for a PPPoE link is fixed at 1492 - even if you change it in pfs it will stay at 1492. Ideally this should be at 1500 for the BT fttc connection - or at least that is what the direct PC-PPPoE connection uses and gets 77Mbps.

Is there anyway to force the issue through (manually) changing MTU settings in BSD / pfs? This would seem to be the only possible way to increase my speed to closer to the actual limit?

Thanks in advance for any suggestions.

stephenw10

I don't believe it has anything to do with mtu. I'm using the default pfsense pppoe settings and have no problem getting full speed from my fttc connection.
Are you sure it was no better using faster hardware?

Steve

hackin8

I am positive that faster hardware made no difference - still seemed to limit at the same speed. Also I have managed to get twice the speed through this setup using the same NIC's etc - but connecting to a PC instead of via PPPoE

I have settled on MTU simply because I can't think of anything else to try!! What VDSL modem do you use for the fttc - maybe I could try changing it? (Although I have tried two different models from BT)

Again - suggestions welcome.

wallabybob

@hackin8:

Thanks in advance for any suggestions.

On Status -> Interfaces do any of the interfaces show non-zero error counts?
Do the error counts change after running a speedtest?

What is the output of pfSense shell command```
ifconfig -i ; netstat -s -p ip ; netstat -s -p tcp


If you have a Linux or Unix system downstream of pfSense on which you can run the speed test please post the output of shell command```
ifconfig -i ; netstat -s -p ip ; netstat -s -p tcp
```executed on that system.

Perhaps your pfSense is running out of puff and occasionally dropping packets forcing a TCP timeout and consequent loss of throughput.

hackin8

Just double checked. When I connect PC directly to VDSL modem using PPPoE I get high speed transfer (76/19) - but on checking the mtu is 1480 (netsh interface ipv4 show subinterfaces) compared to the "normal" WIN7 LAN connection setting of 1500.

Checking pfS (route get domain) shows mtu of 1492. Even if I change it in interface advance settings - it stays at 1492. Is there anyway to change it to 1480 to see if this makes any difference?

Thanks

stephenw10

Have you upgraded to 2.0.2? I see there were some MTU fixes that update.

Steve

Klaws

@wallabybob:

Perhaps your pfSense is running out of puff and occasionally dropping packets forcing a TCP timeout and consequent loss of throughput.

In that case, the traffic graph should show the "ramp pattern" - thoughput steadily increasing, then abruptly dopping down, then ramping up again…and so on.

Does the traffic graph show a (sort of) flat 28Mbps line or does it show much variance?

hackin8

Thanks for the suggestion - I am still on 2.0.1 - have been waiting for 2.0.3 to be finalised before updating.

As I am running a live production server would be reluctant to upgrade unless a real chance of resolving problem.

Does anyone have any confirmation that changng PPPoE MTU has been "fixed" in 2.0.2?

stephenw10

I was looking at the 2.0.2 release notes.

@http://doc.pfsense.org/index.php/2.0.2_New_Features_and_Changes:

Properly obey MTU set on Interface page for PPP type WANs.

Steve

hackin8

Klaws - attached image is from speedtest.net - peaking at about 29Mbps. It did (on this run) tail off a little at the end - but this doesn't usually happen. It is not flat-lining (indiocating a limiter / throttling) but neither is it saw-tooth (running out of resources etc). Although the PC I am running pfS on is quite limited - I did previously try with a much more powerful PC - but the results were identical.

Pattern is pretty much the same each time, although I find with firefox I get about 26-28 and with IE9 I get 30-32 when connected through pfS. when using same PC with PPPoE direct to vdsl modem I get 75-77 and 68-72 respectively.

stephenw10 - thanks for that, I think I will try upgrading to 2.0.3 but first I thought I would try direct PPPoE and changing MTU from 1480 to 1492 to see if that makes any difference.

I am still not really convinced that MTU is the problem - but it is the only thing I can think that might make a difference.

Will report back on trials - unless any other suggestions in the meantime….

speedtest.png_thumb

stephenw10

I have to doubt it's an mtu issue too. As I said my own fttc connection works fine with the default settings. I'm using the BT supplied Huawei HG612.

Steve

hackin8

StephenW10 - thanks for the comments - I also have the HG612. Default settings in 2.0.1 - no packages installed

I believe you have 40Mbps fttc - what speed do you get from speedtest.net using that line only?

I have 80Mbps - one thing I have noticed this evening is that manually selecting another test server on speedtest I can get up to 39-41Mbps.
It seems that the normal auto-selected server (lowest ping is Maidenhead Xilo) cannot go faster than about 30.
Selecting any of 5 other close servers (London) does give me a faster rate of around 40.

Better, but still not as good as the 75-77 I can get when connecting the modem directly via PPPoE.

I haven't been able to do any further testing this evening - will have a try tomorrow.

Klaws

You might try to run a packet capture on pfSense - I guess that if there are MTU/fragmentation issues, you should see corresponding ICMP messages.

Edit: I just remembered to have heard that some modems filter these ICMP packets. So absense of such ICMP messages might not automatically mean that the MTU is okay…

Edit 2: Wait, there might be an easier way to check the MTU. On Windows,
ping google.com -l 1472 -f
will send an ICP Echo Request with a size of 1500 bytes (1472 bytes payload plus 18 bytes ICMP/IPv4 overhead).

solignis

I have heard of a similar problem before on other appliances, mainly the Untangle UTM appliances.

The problem the folks using Untangle were experiencing was related to the type of NIC they were using. The realtek nics and other "cheap" nics heavily leverage the system to do most of the calculation for network traffic. There is supposed to be some theoretical celing to the these NICs and its +/- 20mbps. Once people replace the realteks with server class NICs like Intel Pro1000 PT the problems were resolved.

Disclaimer The above statement is not my own words, it was an idea I read on the Untangle forums Disclaimer

I wonder if this type of situation could be your problem? Have you tried a decent hardware based NIC?

P.S. Oh yeah I probably should also mention that Untangle is based on Debian Linux and NOT FreeBSD so a problem with Untangle might now carry over to pfSense.

hackin8

Well, I tried direct PPPoE to PC. Changed MTU from 1480 (Win7 default) to 1492 - no difference in speed. So I guess MTU is not the issue.

solignis - thanks for the suggestion - that at least is another path to follow. I had a look at the Pro1000 PT - but it is for PCI-E (which I don't have) and in any case is more than the price of my whole setup!!

I have three different NIC's I am using - 1 is built in (fxp = Intel etherexpress Pro?) plus two PCI NIC's detected as re (Realtek 8169) and rl (Realtek 8139) I have checked - all interfaces are running at full duplex - they appear to work fine but I could quite believe that the combination of an "underpowered" PC with a cheap NIC could create issues.

I have tried swapping them around - including with the fxp on the affected WAN but no combination works. (Of course with fxp on WAN I have to use one of the Realteks for the LAN - so maybe still an issue?)

I would like to try another NIC but frankly don't know how to determine what might be better? For example would the Intel Pro100S be any better than the cheap ones I have? Any other suggestion for a suitable PCI card that I could pick up a cheap used one on Ebay?

Thanks again for the suggestion - at least something else to try!!

stephenw10

I would be suprised to see the Realtek nics limiting to this extent. I have seen very low quality nics limit connection speed to, say, 80Mbps where an Intel card will get 95Mbps+.
To rule this out use your fxp nic as WAN and test the download directly to the pfSense box as you did earlier.

Steve