FTP throughput pfsense to WAN
-
Hi Guys,
i have a very special case. I am have a Synology NAS which running FTPS SSL/TLS Server and a Fiber WAN Connection with 500/200 Mbits to share Files with my friends. Technically everything is working fine but...
If someone is downloading files from the FTP the throughput is limited to 120 Mbits in Peak and its struggeling 40-120 Mbits. Very Strange.
But if someone is uploading files to FTP the thougput is much higher (example 270 Mbit constantly)
If i connect to FTP via LAN the speed is capped at 48 MB/s (up and down). I think thats the protocoll itself.Hardware Checksum Offloading, Hardware TCP Segmentation Offloading and Hardware Large Receive Offloading are already disabled.
pfsense: 2.6.0-RELEASE (amd64)
pfsense hardware: Super Micro C2758 with 16GB ECC RAM and Samsung 840 pro SSD
pfsense Interface Statistics: No error, no retransmission, no drops
NAS: DS1817 Raid 5 7x WDC WD100EFAX (NAS is definitely not the bootleneck)for alternatives i tried SFTP which is horrible slow (12 mbits), FTP uncrypted is no option.
any ideas ? what can be the bottleneck ?
Regards
Voosa
-
@voosa2k
i have the same C2758, no problem with up to 500/100 Mbits. I would investigade this DS1817, WD100EFAX are very slow disks, in a Raid 5 configuration with maybe full disks, all of your IOPS are used by this Raid 5 configuration. Try to SFTP to a PC with Windows 10 (with some kind of SFTP service) to sort out this DS1817. The firewall isn't involved in SFTP performance, so most likely the DS1817 is somehow the bottlenek. Maybe there are some traffic limiters configured on the DS1817? -
The NAS isnt the bottleneck. As i wrote if i connect to FTPS via LAN i get 45 MB/s (read and write). BTW over SMB3 ich can write for 550 MB /s and Read over 700 MB/s. Raid 5 with these Drives is very Fast. No problem with that. Even 20 MB/s would be enough over WAN but its struggeling between 40 to 120 Mbits.
SCP to another PC is even the same super slow about 12 Mbits (1,25 MB/s). It have to be something on WAN (pfsense). But i have no idea what i could be ..... -
@voosa2k
ok, did you check with speedtest, if there is problem between your pfsense wan and the internet? You may check with iperf3 (open a port on the wan side) between your friend and your pc if there is any problem there. The firewall doesnt now about the data transfered, if its encrypted or not, if its sftp or something else... . If iperf (server-client) is ok, then maybe there is a problem on the other side, not on your side.... -
Wan seems to be fine as usual:
speedtest.net
https://www.speedtest.net/result/13302091694
iperf UP:
.\iperf3.exe -c speedtest.wtnet.de -p 5200 -P 10 -4
Connecting to host speedtest.wtnet.de, port 5200
[ 4] local 192.168.178.110 port 50794 connected to 213.209.106.95 port 5200
[ 6] local 192.168.178.110 port 50795 connected to 213.209.106.95 port 5200
[ 8] local 192.168.178.110 port 50796 connected to 213.209.106.95 port 5200
[ 10] local 192.168.178.110 port 50797 connected to 213.209.106.95 port 5200
[ 12] local 192.168.178.110 port 50798 connected to 213.209.106.95 port 5200
[ 14] local 192.168.178.110 port 50799 connected to 213.209.106.95 port 5200
[ 16] local 192.168.178.110 port 50800 connected to 213.209.106.95 port 5200
[ 18] local 192.168.178.110 port 50801 connected to 213.209.106.95 port 5200
[ 20] local 192.168.178.110 port 50802 connected to 213.209.106.95 port 5200
[ 22] local 192.168.178.110 port 50803 connected to 213.209.106.95 port 5200
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 2.75 MBytes 23.0 Mbits/sec
[ 6] 0.00-1.00 sec 2.00 MBytes 16.7 Mbits/sec
[ 8] 0.00-1.00 sec 2.25 MBytes 18.8 Mbits/sec
[ 10] 0.00-1.00 sec 2.00 MBytes 16.7 Mbits/sec
[ 12] 0.00-1.00 sec 2.38 MBytes 19.8 Mbits/sec
[ 14] 0.00-1.00 sec 4.62 MBytes 38.6 Mbits/sec
[ 16] 0.00-1.00 sec 4.75 MBytes 39.7 Mbits/sec
[ 18] 0.00-1.00 sec 3.00 MBytes 25.0 Mbits/sec
[ 20] 0.00-1.00 sec 3.50 MBytes 29.2 Mbits/sec
[ 22] 0.00-1.00 sec 2.00 MBytes 16.7 Mbits/sec
[SUM] 0.00-1.00 sec 29.2 MBytes 244 Mbits/secIperf Down
.\iperf3.exe -c speedtest.wtnet.de -p 5200 -P 10 -4 -R
Connecting to host speedtest.wtnet.de, port 5200
Reverse mode, remote host speedtest.wtnet.de is sending
[ 4] local 192.168.178.110 port 50782 connected to 213.209.106.95 port 5200
[ 6] local 192.168.178.110 port 50783 connected to 213.209.106.95 port 5200
[ 8] local 192.168.178.110 port 50784 connected to 213.209.106.95 port 5200
[ 10] local 192.168.178.110 port 50785 connected to 213.209.106.95 port 5200
[ 12] local 192.168.178.110 port 50786 connected to 213.209.106.95 port 5200
[ 14] local 192.168.178.110 port 50787 connected to 213.209.106.95 port 5200
[ 16] local 192.168.178.110 port 50788 connected to 213.209.106.95 port 5200
[ 18] local 192.168.178.110 port 50789 connected to 213.209.106.95 port 5200
[ 20] local 192.168.178.110 port 50790 connected to 213.209.106.95 port 5200
[ 22] local 192.168.178.110 port 50791 connected to 213.209.106.95 port 5200
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 5.93 MBytes 49.7 Mbits/sec
[ 6] 0.00-1.00 sec 6.03 MBytes 50.6 Mbits/sec
[ 8] 0.00-1.00 sec 5.07 MBytes 42.5 Mbits/sec
[ 10] 0.00-1.00 sec 6.84 MBytes 57.4 Mbits/sec
[ 12] 0.00-1.00 sec 6.96 MBytes 58.4 Mbits/sec
[ 14] 0.00-1.00 sec 5.29 MBytes 44.4 Mbits/sec
[ 16] 0.00-1.00 sec 5.48 MBytes 45.9 Mbits/sec
[ 18] 0.00-1.00 sec 6.72 MBytes 56.3 Mbits/sec
[ 20] 0.00-1.00 sec 5.45 MBytes 45.7 Mbits/sec
[ 22] 0.00-1.00 sec 5.75 MBytes 48.3 Mbits/sec
[SUM] 0.00-1.00 sec 59.5 MBytes 499 Mbits/sec -
What speeds are you expecting?
I would try running an iperf server on your LAN and set up a port forward to it. Now test against that from the same remote client.
What is the available bandwith at the client end?
Steve
-
Strange.
Remote Client Iperf
Connecting to host XXX, port 5201
Reverse mode, remote host XXXX is sending[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 2.78 MBytes 23.3 Mbits/sec
[ 6] 0.00-1.00 sec 1.89 MBytes 15.8 Mbits/sec
[ 8] 0.00-1.00 sec 4.48 MBytes 37.5 Mbits/sec
[ 10] 0.00-1.00 sec 2.71 MBytes 22.7 Mbits/sec
[ 12] 0.00-1.00 sec 4.70 MBytes 39.3 Mbits/sec
[SUM] 0.00-1.00 sec 16.6 MBytes 139 Mbits/secConnecting to host XXX port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 5.25 MBytes 43.9 Mbits/sec
[ 6] 0.00-1.00 sec 5.12 MBytes 42.8 Mbits/sec
[ 8] 0.00-1.00 sec 5.00 MBytes 41.8 Mbits/sec
[ 10] 0.00-1.00 sec 5.12 MBytes 42.8 Mbits/sec
[ 12] 0.00-1.00 sec 5.12 MBytes 42.8 Mbits/sec
[SUM] 0.00-1.00 sec 25.6 MBytes 214 Mbits/secI would expect that my WAN speed is limiting but it isnt.
-
What is the remote client available bandwidth?
What latency is there between the sites?
That 139Mbps doesn't seem that far off the peak values you saw for FTPS.
Steve
-
@stephenw10 said in FTP throughput pfsense to WAN:
available bandwidth?
available bandwidth is 250 / 250
latency i dont know but its Fiber to Fiber i think max 20 ms -
Hmm, so I would not expect to ever see faster then 250/200 but you're actually seeing 214Mbps in the server to client direction.
You should check the latency to be sure. It doesn't appear to related to FTPS specifically though. -
done. Host 2 Host Latency 24 ms.
i tested 2 hours again. iperf speed (upload to him) is not quiet goodDownload from my NAS to him
\iperf3.exe -c XXX -p 5201 -P 5 -R
Connecting to host XXX, port 5201
Reverse mode, remote host XXX is sending
[ 4] local 192.168.178.23 port 58873 connected to 217.252.53.45 port 5201
[ 6] local 192.168.178.23 port 58874 connected to 217.252.53.45 port 5201
[ 8] local 192.168.178.23 port 58875 connected to 217.252.53.45 port 5201
[ 10] local 192.168.178.23 port 58876 connected to 217.252.53.45 port 5201
[ 12] local 192.168.178.23 port 58877 connected to 217.252.53.45 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 2.78 MBytes 23.3 Mbits/sec
[ 6] 0.00-1.00 sec 1.89 MBytes 15.8 Mbits/sec
[ 8] 0.00-1.00 sec 4.48 MBytes 37.5 Mbits/sec
[ 10] 0.00-1.00 sec 2.71 MBytes 22.7 Mbits/sec
[ 12] 0.00-1.00 sec 4.70 MBytes 39.3 Mbits/sec
[SUM] 0.00-1.00 sec 16.6 MBytes 139 Mbits/secSend to my NAS from him (thats fine.)
Connecting to host XXX, port 5201
[ 4] local 192.168.178.23 port 58857 connected to 217.252.53.45 port 5201
[ 6] local 192.168.178.23 port 58858 connected to 217.252.53.45 port 5201
[ 8] local 192.168.178.23 port 58859 connected to 217.252.53.45 port 5201
[ 10] local 192.168.178.23 port 58860 connected to 217.252.53.45 port 5201
[ 12] local 192.168.178.23 port 58861 connected to 217.252.53.45 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 5.25 MBytes 43.9 Mbits/sec
[ 6] 0.00-1.00 sec 5.12 MBytes 42.8 Mbits/sec
[ 8] 0.00-1.00 sec 5.00 MBytes 41.8 Mbits/sec
[ 10] 0.00-1.00 sec 5.12 MBytes 42.8 Mbits/sec
[ 12] 0.00-1.00 sec 5.12 MBytes 42.8 Mbits/sec
[SUM] 0.00-1.00 sec 25.6 MBytes 214 Mbits/secDont know why iperf to my friends Host is so slow, FTPS, iperf and anything else is struggeling from 40 -140 Mbits. Although if he do a speedtest or iperf to external iperf server he got 245 down and 241 Up.
Moreover my speedtest is absolutly fine. 500 / 200 (iperf test to speedtest.wtnet.de )Makes absolutly no sense to me. Some ISP Bullshit maybe? i cant imagine.
-
Running a speedtest at each end if not using the same route as traffic between the sites so you may simply not be passing whichever hop is throttling you. If you can't see the speed with iperf though you will never see it in FTPS.
Steve
