Try to download a 500GB file/VM via FTP and see how packetloss and timeouts affect performance
-
Hi there
Downloading a wide variety of VM's and pfsense just chokes despite Suricata disabled and running on 1 core only.
It takes about 20 seconds for the transfer to resume and timeouts every 8-10 minutes.
Any ideas?
Nothing in the logs at all.
-
do you have some sort of monitoring in place that could help investigate the bottleneck ?
-
Nothing in the server logs at all (application, security, setup, system)
Everything is running as it should serverside.
Only thing that sticks out.
-
hmm maybe try using legacy blocking mode (pcap) instead of netmap ?
-
Well I don't think I have ever downloaded a single 500GB file.. But I have downloaded say 30GB worth of files that are like 2GB each via ftp recently as a test for ftp (dead protocol) speed vs syncing the data via resilio And never seen an issue..
But I have many days of downloading from my server. with large amounts of data.. Looking sofar for may I am at 2.86TB for the month.. Most likely I could top 3TB this month.. I have quite a few files to still grab..
I have never seen such an issue downloading anything.. And I download quite a bit, but its mostly via https from my server and using multiple streams to max it out my connection 500Mbps.. I normally get 60MBps - and this runs at that speed sustained for the transfer... Not sure I want to test with 500GB file ;) but sure could zip up some files on the server to create a decent sized file and download that via ftp..
It's zipping up now.. Will download and let you know how it goes. I need to grab the files anyway.
-
Filezilla is the FTP client but could try WinSCP instead.
30GB is not a problem banging away with 400mbit. It needs to run for a while.
Maybe 100GB would cut it.
-
Ok 55ish gig file created on server.. Its downloading now - I will keep an eye on it... Got it up on my other monitor, see if I notice if it stalls.
edit: Ok 10 min in... Haven't noticed any sort of stalls or blips.. Its just rocking away.
edit2: Ok 20 min in... Have not noticed any sort of blips or stalls.. I would think such a stall would of been a big hit on the transfer rate.. Which is staying pretty freaking close to what it started at.
edit3: ok complete - 30 min transfer.. Never saw any blips...
You saying your seeing it stall like every 8-10 I should of seen atleast 2..
Status: File transfer successful, transferred 55,293,613,316 bytes in 1802 secondsPretty much sustained 30MBps the whole time..
Something on the box your downloading from, their connection, your connection... But I don't see it as something wrong with pfsense.. Other than an increase in the interrupts.. My 4860 didn't even seem to break a sweat..
-
Thanks for the efforts. I am seeing a 6% packetloss so far.
Have to look into some hardware issues. BRB.
-
Changed the NIC to an Intel ET adapter from a Marwell Yukon. No packetloss and so far still a stable connection.
Looks like the Marwell driver has a memory leak.
