Breaking the 6MB Barrier
-
Hi all,
I have 2 pipes coming into my Pfsense box (and old Barracuda box).
1 is 20/3 and the other is 75/5.
When I do speed tests, downloads, normal activity, all is well.
However, when trying to upload into a machines sitting behind the firewall from the outside world (2 server with 100mbps ports in 2 different DCs.), I cant seem to get passed 6megs.
This is happening when using Rsync, FTP and Openvpn.I am open to any and all suggestions because after a month of trying all this, Im losing my mind.
Thank you!
-
Are you saying that transfers stop after 6 MB or that you can't exceed 6 Mb/s?
-
I'm saying it tops off at 6Mb/s. :'(
-
Bump
-
Do you mean "while saturating your upload, your download is limited" ?
http://cable-dsl.navasgroup.com/#Asymmetry
-
you only have 3Mb and 5Mb are the links bonded for upload, does pfsnse do bonding of links to combine the bandwidth?
what do the upload speeds show for your speed tests?
-
you only have 3Mb and 5Mb are the links bonded for upload, does pfsense do bonding of links to combine the bandwidth?
what do the upload speeds show for your speed tests?
Ok, let me fill in some data here based on continued testing and a little recap.
Pfsense is sitting on a barracuda router (hardware) connected to 2 ISPs.
ISP1 has a 75/5Mb pipe on WAN.
ISP 2 has a 20/3Mb on Opt 1.
Machine 1 is sitting next to firewall with a 10/100 connection using Cat5 in Miami.
Server is sitting in a data center with a 100Mb connection in Los angeles.
There is a rule on Pfsense to allow a connection from Server 1 based on IP to Machine 1 on the Rsync port on WAN.
Server 1 runs rsync setup to connect to the external IP of WAN on pfsense.
The connection goes fine, starts at 17.74MB/s and then slowly (within 30 seconds) drops to 669.62kB/s and stays there.While testing, I have tried this with an without a VPN connection.
I have also assumed Rsync was the issue so I tried a results Windows file system copy with similar results.Speed test from Machine 1: http://www.speedtest.net/my-result/3114504233
Speed test from server: http://www.speedtest.net/my-result/3114488512Hopefully that clarifies a bit and someone can help me find a solution.
Thanks much!
-
Can you answer the two questions above?
And did you read the link I provided?
-
I did indeed.
In terms of bonding, no, they are setup as failover.
In terms of symmetry, and this goes beyond my knowledge level, how is that affecting me when I'm essentially only downloading? Fro example, if I download something direct from http or ftp from a random place, I get the full 70+megs, but why am I getting the bottleneck on the upload (which is a download on my end)? -
Have you tested straightup disk throughput? dd if=/dev/zero of=~/test bs=512k count=1000 should give you a general idea of if your disk is fast enough to keep up. If disk is fine, check your CPU during transfers. If that's fine, check your memory. You're I/O-bound somewhere from the sound of it.
-
Have you tested straightup disk throughput? dd if=/dev/zero of=~/test bs=512k count=1000 should give you a general idea of if your disk is fast enough to keep up. If disk is fine, check your CPU during transfers. If that's fine, check your memory. You're I/O-bound somewhere from the sound of it.
On the firewall?
If yes:
[2.1-RELEASE][root@local]/root(1): ~/test bs=512k count=1000
/root/test: Command not found.
[2.1-RELEASE][root@local]/root(2): dd if=/dev/zero of=~/test bs=512k count=1000
dd: ~/test: No such file or directory -
Oops, meant to say dd if=/dev/zero of=/root/test bs=512k count=1000
Do keep in mind that this will create a 512MB file, so if you don't have a lot of space you will want to alter your count argument. -
Oops, meant to say dd if=/dev/zero of=/root/test bs=512k count=1000
Do keep in mind that this will create a 512MB file, so if you don't have a lot of space you will want to alter your count argument.\1000+0 records in
1000+0 records out
524288000 bytes transferred in 9.265605 secs (56584325 bytes/sec)I ran this while doing a transfer.
Now, here's an interesting bit of info… I ran 3 rsyncs at the same time from the machine.
Each, hit the 760-780KB/s mark and sat there running around the same range.
This gave me about 24 Mb/s which now has me REAL confused because I think that would rule out any bandwidth or hardware issues.
It almost feels like something is limiting on a per "pipe" or connection basis. -
Screen shot of system stats attached.
-
Bump. :-\
-
Buler…. Buler.... anyone?! :-\
-
Now, here's an interesting bit of info… I ran 3 rsyncs at the same time from the machine.
Each, hit the 760-780KB/s mark and sat there running around the same range.
This gave me about 24 Mb/s which now has me REAL confused because I think that would rule out any bandwidth or hardware issues.
It almost feels like something is limiting on a per "pipe" or connection basis.Are you running any traffic shaping/QoS? Are you sure? Have you ever run traffic shaping?
Steve
-
Are you running any traffic shaping/QoS? Are you sure? Have you ever run traffic shaping?
SteveSteve,
Just double checked traffic shaping, disabled (screen shot attached in case I'm wrong).
It may have been enabled at some point while playing around.
I'd be glad to delete anything related if I knew how. ???Thank you for your help!
-
Just to be sure I'd look in the config.xml file and check you nothing in the <shaper>or <l7shaper>sections.
It's hard to imagine anything else that might limit you speed on a per connection basis.
Steve
Edit: reading through this again it looks like this could still be a limit on the remote machine. How did you test the connection speed with the 'Windows file system copy'?</l7shaper></shaper>
-
This is probabably not entirely helpful but I had issues with encrypted file transfers being rate limited to 1mbps. Anything that went through openssl just couldn't pass that barrier. I could start up multiple instances, and they would both be 1mbps - but that I found was because openssl was single threaded and I had multiple cores to handle the transfer (I think..) Is your FTP transfer using SFTP?
