FTP passive port on demand opening
-
(decrypt 150gb on the fly instead of 5gb it's different)
Multiply this CPU usage for 5 contemporary connections transfer of each client, and multiply for 10 clients = 50. (it doesn't matter if you have a single core or eight cores, the server will became unstable).
And I'm not considering possible contemporary uploads of other users at the same time.. With others 20 connections for example..
.. With out transfer that are decrypted and input transfer that are encrypted.. (other CPU load)SFTP is very powerful and secure solution, but not applicable in heavy load environments.
Thank you :)
-
If you are running a current business and talking about "heavy load" but still use an antique protocol like FTP I'd say you're having other problems. No company I know, not even heavy hitters with big data pipes, are still using FTP these days. As @johnpoz said, it's 2020. There are other and modern ways to transfer files - securely! - without using antique techniques.
Also running 150GB files? What's the use case in that? I've seen companies dealing with 3D printing and architecture or constructions or media production using files this size, but even they won't touch FTP for transferring such numbers. And multiple clients that do up- and downloads of such overly big files at the same time? To somewhere through the internet into some cloud location? Seems strange.
(BTW: just have a look at how YouTube et al handle video upload. Ever seen a video in 4k/8k with 60fps? And they push that via HTTPS, too. Noone decrypts/encrypts the file on the fly because the services use TLS layer to encrypt. The file isn't encrypted - sent - decrypted. So I don't understand your logic with 150GB being more CPU bound then 5GB while transferring via modern protocols?)
-
I'm talking about the local storage encrypt, non about the transfer ;) all the files are encrypted on the disk. Files are decrypted when downloaded and are transfered over https.
I already use https from browser for upload and download and with webdav client the same using https.
-
Anyway, my question was for the on-demand opening of passive doors...
....and can be possible to interact with pfsense using API from a web page? .. i want to create a PASS firewall rule when a client compile a protected form where will write her static ip , in order to allow access only to him.
-
What indeed means, that it has nothing at all to do with running another protocol but is due to your local storage encryption thingy. So again, don't understand why you keep on mentioning that 150GB files would be harder/more CPU bound to transfer with any other possible (current, modern) transportation method. SFTP, HTTPS, etc. wouldn't need any more CPU cycles for 150G or 5G. If SFTP is slow I'd check with your target. Never seen that much difference on our customer hosting servers at all. More often it's a thing with the customers not updating their client software and using old/deprecated SSH/SSL ciphers/settings. I'd try updating clients etc. to ensure best protocols are used with hardware encryption in use etc. Had that discussions too many times like "But that tool worked for years!" - "Are there updates?" - updated tool multiple major versions "Wow it really is faster/is working now!" ;)
But again, to stay in the OP: No passive port opening automagically as passive ports are configured on the server side and every server can declare them as they want. pfSense has an active FTP helper for outbound connections that could help with that but it relies on the other side offering active FTP - not everyone does that (or uses FTP at all these days).
@openaspace said in FTP passive port on demand opening:
i want to create a PASS firewall rule when a client compile a protected form where will write her static ip , in order to allow access only to him.
Why? Passive FTP means you have inbound (if you host the server) or outbound connections with 21/tcp as well as xxxxx-yyyyy/tcp. So if it's your own FTP server where you upload your huge file stuff, why not allow your passive port range to your specific IP? Don't get the problem with opening passive ports here?
-
As I already answered in my first post, there is no passive helper or way for pfsense to auto open the passive ports. If you want to provide passive ftp as an option to your ftp server then you need to set your ftp server to use a specific range of ports for its passive connections. And then forward that range.
You can for sure lock down the source IPs on this forward/firewall rule.
-
Yes, I will see if is possible to interact with pfsense using api to do this
-
@JeGr it's well known that an sftp process use more cpu
-
Your more than welcome to tinker all you want - but would be akin to coming up with a faster way to read floppy disks.. Nobody uses them ;)
Sure there is a huge user base for something like this - the 3 people still using ftp will love you I am sure ;)
-
@johnpoz said in FTP passive port on demand opening:
Nobody uses them ;)
that's where you're wrong ;)
-
Yeah there are still people using horses to pull their carts as well.. Not my point!
-
@johnpoz said in FTP passive port on demand opening:
Yeah there are still people using horses to pull their carts as well.. Not my point!
in the economic crisis language, it's called "happy decrease"!
-
@openaspace said in FTP passive port on demand opening:
@JeGr it's well known that an sftp process use more cpu
Just to add 0.02$: Again depending on the cipher and setup used. And it's documented, that it strongly depends on you configuration of SSH. Also speed comparisons are available that show that modern versions of SSH that are patched for the old problems of high latency or window sizes aren't far behind in transfer speed. So modern ciphers like AES-CTR deal with single/multi-core problems and you can hit a 1gpbs bandwith limit if you do a little bit of homework and don't use ice-age-old versions of your toolchain ;)
-
Only 10 sftp connections for a total 60MB/s
-
10 webdav connections over https at 90MB/s ;)
-
Besides your question solved - what about actually reading what I wrote? You show some graphs with no intel whatsoever. As I said, if you set it up right, it CAN work the same without pulling your leg. How should I know what old software you're running? I can show you graphs all day long - without details they tell you nothing other that you seem to be having a bad SSH setup. shrug But hey, if one wants to see problems as nails to use their hammer, that's nice. Have fun hammering :)
-
What cipher is being used for these connections... There can be some huge difference in speed and cpu cost.. And if the box supports say aes-ni, etc..
Simple test here locally.. using just chacha20 vs aes256-ctr and go from like 75MBps, to full gig at 111 and pretty good drop in cpu usage as well. Now I am a fan of chacha.. But if what your looking for is fastest speed with lowest resources used... Then yes you have to take a few minutes to config your stuff and not just turn it on.. This goes for everything in the IT world.
-
@johnpoz I denote a certain polemical vein
