FTP using SSL/TLS from Client behind pfSense 2.3.4 to Server outside fails

[knatterton]

Hi to all you helping hands :-)

I searched through the forum, through squid and through the Internet, but couldn't find a solution for my problem.

When the client (tried two different) tries to connect via pfSense, I always receive an error after authentication.
I also did logging with Wireshark, and it shows a different behavior connecting through pfSense and doing it without the Firewall (what of course works).

For the moment the party who delivers data using the server with the FTPS connection insists on this way to deliver the data.

So if anyone could give me an idea how this problem can be solved, I would be very grateful !

Thank you in advance for you input !

[johnpoz]

Are you active or passive connection.  When doing ftps - the control channels is encrypted.  So using the package helper for active connections can not open the port for the data side..  You would have to use passive.

This is never ending topic… Die already FTP!!  Why are you not using SFTP which is just 1 (ssh 22) port total secure not just the control channel..

[knatterton]

Hi  !

Thank you for your comment (and sorry for my delayed answer, did not receive an information about your reply) !

Connection is passive, so this does not change the situation.
I already thougt about asking for another transfer protocol, but before I am not sure that this should work with pfSense I didn't want to do this. When pfSense is able to handle SFTP in version 2.3.4 then I will try this - I hope they don't tell me that other customers can handle FTPS …

Thank you, but other solutions and informations are still welcome :-)

[johnpoz]

If your going to use passive - then the package doesn't do anything anyway and not needed.  Do you allow the passive ports out?

The server behind firewall that is being connected to from passive clients would have to have those ports forwarded to the server..

Pfsense since version 1 has allowed sftp to work.. Its 1 port!  Nobody today should still be using ftp… You can run it FREE on any OS out there..

If your going to troubleshoot ftp you need to understand are you active passive and what that means for the data channel..
http://slacksite.com/other/ftp.html
Active FTP vs. Passive FTP, a Definitive Explanation

For active client behind pfsense you would need to use the package and can not be ftps since control is encrypted.  For passive client behind pfsense - pretty much any has to be open if you limit what ports outbound then it will not work.  For server behind with passive clients coming from the internet you would have to forward the correct ports of the passive ports the server will tell the client to use.  And the server also needs to hand out your public IP not its rfc1918 behind the firewall..

[knatterton]

Hello again !

Thank you for the detailed answer !!

Necessary ports are opened as the old version 2.0 - which we used before - supported FTPS, maybe also because of the usage of Squid.
But after the upgrade to 2.3.4 it did not work any longer, and I can't find any hint on the usage of Squid to change this …

Anyway, as SFTP is an option I try to change the communication with the customer, hopefully he can support this.

I will let you know !

[johnpoz]

ftps has never been supported by any helper/proxy - you would have to manually forward the ports..  Squid has zero to do with it..

How about you look at the ftp logs and or firewall logs and or sniff to figure out what the problem is?  But moving to sftp is a much better solution that is for sure!!

[knatterton]

I swear, it worked before :-)
And we haven't changed any settings on the rules. And of course I checked the firewall log for denied connections, that's why I used Wireshark to find out what the difference is between a connection behind pfSense and without, but as I am no specialist for TCP and FTP I only can tell, that pfSense changes the data flow, but not why and how …

Anyway, we try to change the communication, that seems to be the easiest way to solve the Problem.

[johnpoz]

So your ftps server is outside pfsense on the public internet.. In ftps the control channel is encrypted there is no way for pfsense to see inside this control channel to see what IPs need to be changed or opened..

So the ports have to be allowed on pfsense by manual forwarding..  If your client is using active, then the ftp server would get an IP from the client and try and connect to this port from source port 20.  If passive the server would tell the client what IP and port to connect too.  Are you blocking outbound traffic from the client?  Or is your rule any any on the lan?

You sure your client supports passive?  For example the ftp cmd line client in windows does not support passive only active.

There is no helper/proxy that can help with ftps since the control channel is encrypted - so it can not change rfc1918 to the correct public nor can it open any ports on the firewall.

Lets make sure we are on the same page.. Is the server out on the public internet and your connecting to it from a client behind pfsense, or is the server behind pfsense and the client is coming from the internet?  Do you have the client logging?  You can see if there is a port command or pasv command given  From that command you can work out the IP and port that is to be used for the data connection.

[knatterton]

To be clear:

We are in a LAN behind the pfSense 2.3.4, and connect to a FTP Server outside in the Internet, using PASSIVE FTPS - with no success.
And we didn't change rules which worked before the upgrade from 2.0 - which we did by exporting the settings, made a new, fresh install, and imported the settings.
Squid and FTP proxy were active on the old installation, but Squid is not any more, ONLY the FTP Proxy package is installed as we also have customers who want us to download their data without encryption. But I also tried FTPS with FTP Proxy deactivated what didn't change the result.

Hope this makes it exactly clear.

Ah, and yes, I tried the log of Filezilla (which is the client), but this does not log anything helpful as the connection fails after the AUTH command - that's why I used Wireshark to get more information.

[johnpoz]

dude if your rule is any any out.. Then pfsense has ZERO to do with your ftp problem.. Again I suggest you read the link I gave you for how ftp works be it active or passive.

And filezilla will for sure give you info.. What was the port or pasv command?? After you authed.. Make sure in filezilla you click on show detailed log..

see the difference

Normal log:
Status: Selected port usually in use by a different protocol.
Status: Resolving address of ftp.sophos.com
Status: Connecting to 195.171.192.29:990…
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing...
Status: Directory listing of "/" successful
Status: Disconnected from server

Detailed log:
Status: Selected port usually in use by a different protocol.
Status: Resolving address of ftp.sophos.com
Status: Connecting to 195.171.192.29:990...
Status: Connection established, waiting for welcome message...
Response: 220-Sophos FTP service
Response: 220 This is a private system - No anonymous login
Command: AUTH TLS
Response: 234 AUTH TLS OK.
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Command: USER <snipped>Response: 331 User <snipped>OK. Password required
Command: PASS **********
Response: 230-User <snipped>has group access to:  domain use
Response: 230 OK. Current restricted directory is /
Command: SYST
Response: 215 UNIX Type: L8
Command: FEAT
Response: 211-Extensions supported:
Response: EPRT
Response: IDLE
Response: MDTM
Response: SIZE
Response: REST STREAM
Response: MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
Response: MLSD
Response: AUTH TLS
Response: PBSZ
Response: PROT
Response: UTF8
Response: ESTA
Response: PASV
Response: EPSV
Response: SPSV
Response: 211 End.
Command: OPTS UTF8 ON
Response: 200 OK, UTF-8 enabled
Command: PBSZ 0
Response: 200 PBSZ=0
Command: PROT P
Response: 200 Data protection level set to "private"
Status: Logged in
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is your current location
Command: TYPE I
Response: 200 TYPE is now 8-bit binary
Command: PASV
Response: 227 Entering Passive Mode (195,171,192,29,195,212)
Command: MLSD
Response: 150 Accepted data connection
Response: 226-Options: -a -l
Response: 226 12 matches total
Status: Directory listing of "/" successful

See the PASV command and the response..  So this tells me the IP for client to connect too.. 195.171.192.29 and then the port (195*256)+212 or port 50132

So every time I hit dir refresh it makes a pasv command and the port changes.. And I can see that in pfsense state table..  See attached image..  Now when I try and use active I get different command.. PORT

Now when I use active mode..

Response: 257 "/" is your current location
Command: TYPE I
Response: 200 TYPE is now 8-bit binary
Command: PORT 192,168,9,100,193,127
Response: 500 I won't open a connection to 192.168.9.100 (only to 24.13.x.x)
Error: Failed to retrieve directory listing

What is are problem here!!! My client sent its rfc1918 address.. And since this is a ftps connection the ftp active package can not fix that..  So how would the ftps server connect back to my private address..  And is port (193*256)+127 = 49535 forwarded to me??

So I can change filezilla to use my public IP.. See attached.  I then tell it to use specific ports in active, and setup pfsense to forward those to my client.. And bing it works!!  If does not then you need to troubleshoot your port forwarding, is it even getting to your firewall etc..  This is via a ftps connect as you can see from above where the control channel is encrypted and pfsense can not help with any package..

</snipped></snipped></snipped>