RC3 : Passive FTP disconnects after 127 "PASV" commands !

thewild

Hi all !

I have an FTP server behind a pfSense box in NAT.
The FTP server is FileZilla, running on Windows 2008 R2.

I have an alias IP address that is used for the FTP connection. Connections on this address on port 21 are forwarded to my server.
It works, so I believe that the FTP helper has done it's job.

Now, I have a FTP-sync application that makes a lot of PORT commands to the server in a short time.
This application hangs every 100-200 transfer (= every 100-200 PORT command), saying that the connection was aborted by the server.

On my FTP server, I see a log that the data connection could not be opened right after the MLSD command (sent to get the directory listing).

I checked the states table and I can see a lot of states left in FIN_WAIT_2:FIN_WAIT_2 state here, but not so many as too block my firewall (I have 1000 states just between my client and server).

Edit : I just tried to narrow the range of passive ports on the server, but it seemed to make the problem even worse.

Any idea about what is going on here ?

Thanks a lot !

rancor

Have you tried to do the same thing on the LAN without the firewall? How do you know if this a problem with pfsense and not the server/client?

// rancor

thewild

@rancor:

Have you tried to do the same thing on the LAN without the firewall? How do you know if this a problem with pfsense and not the server/client?

// rancor

No I haven't tried this, but I will.

I assume that the problem is with pfSense because we have moved from an Astaro configuration 2-3 months ago.
I am fairly confident that this problem did not exist then.
I just noticed this problem, but I believe this is happening since then, even though I have no evidence of that.

thewild

I just gave it a try through an OpenVPN connection and it worked just fine.

Am I right to assume that I have a problem with the FTP Proxy ?

eri--

Are you just opening port 21 on you firewall or you have opened even the passive ports?
Does pfSense log any blocked traffic?

thewild

@ermal:

Are you just opening port 21 on you firewall or you have opened even the passive ports?
Does pfSense log any blocked traffic?

I have only opened port 21, that's why I assume that the ftp proxy is doing its job. I have not even changed the advertised passive IP address of my FTP server, but this address is correctly translated to the public address by pfSense.

Firewall logging to RAM was disabled. I just enabled it and checked. I see 3 lines in the firewall log, and they look like passive ftp connections.
The 3 lines are from the same source and destination ports, so I believe they are the 3 connection attemps from the client before reconnecting to the server ?
Now, the ftp script has been running for about 5 minutes, it had to reconnect at least 20 times but I only see 5 lines (3 for the first couple of src-dest ports, 2 for a second couple of ports).
So all failures are not logged.

thewild

The script has been running for some time now. I don't really understand what I see in the firewall log.
Some PASV connection failures are shown, some are not.
I also see some connections logged that were in fact successfull, according to the client's log !
I must admitn this is way beyond my understanding.

thewild

All those packets are blocked by the default deny rule apparently :

The rule that triggered this action is:

@1 scrub in on em1 all fragment reassemble
@1 block drop in log all label "Default deny rule"

Notre also that I have a lot of those logs. I am not worried because I have read this article, but is it normal that all packets have to be reassembled to be passed on to the firewall ?

rancor

@thewild:

All those packets are blocked by the default deny rule apparently :

The rule that triggered this action is:

@1 scrub in on em1 all fragment reassemble
@1 block drop in log all label "Default deny rule"

Notre also that I have a lot of those logs. I am not worried because I have read this article, but is it normal that all packets have to be reassembled to be passed on to the firewall ?

Yes, if one of the routers along the path to destination requires smaller packet size when the originating MTU the firewall needs to reassemble the packets. Often seen on tunnelled connection like GRE or VPN.

thewild

OK. That seems to be the case for most if not all connections, but that is not connected to my problem anyway.

Now, I am not sure that what I see in the firewall log is related to the passive connection problem…

thewild

Sorry for "upping" this thread, but this problem is quite problematic for us.

Is there something else I should check or some parameter I could try to change ?

Thanks a lot for your help !

pinoyboy

1 - Are you using port redirect or static mapping for ip of FTP?
2 - Attached small screenshot of the WAN / LAN rule pertaining to FTP only.
3 - Have you set static port range for PASSIVE FTP on FTP Server itself, if so what are exact numbers / ranges?

thewild

Thanks for your answer pinoyboy !

@pinoyboy:

1 - Are you using port redirect or static mapping for ip of FTP?

I don't understand your question. I have a simple NAT rule to forward port 21 to the private address of the FTP server, the passive part is handled by the FTP Proxy I guess ?

2 - Attached small screenshot of the WAN / LAN rule pertaining to FTP only.

As I said, very basic rule.
http://i54.tinypic.com/ih3x2p.png

3 - Have you set static port range for PASSIVE FTP on FTP Server itself, if so what are exact numbers / ranges?

No, I tried but it did not seem to help. I think I tried with a 1000 port range, from 21000 to 21999.

thewild

Back on the case. I upgraded to the latest RC3 (25th July) and did some more testing.

It appears that the connection error comes exactly every 127 PASV commands (the 128th fails).
I changed the subject of my post because I had written every 100-200 PORT command.

That is of course not a coincidence.

My client FTP software does a directory tree scanning, and it issues a "PASV" command for every directory in the tree (afterwards it issues an MLSD command).
Since the command is very fast to run, my client issues a lot of "PASV" commands in a very short time.

The the sequence is :
-> PASV
<- 227 entering passive mode
-> MLSD
<- directory information
-> PASV

…. 127 times

-> PASV
<- 227 entering passive mode
-> MLSD
Error, server closed connection.

I hope this helps to understand where the problem is ?

danswartz

Boy, that sounds like someone is using a signed byte where they shouldn't be…

thewild

@danswartz:

Boy, that sounds like someone is using a signed byte where they shouldn't be…

I guess I should submit this issue in redmine then ?

eri--

Please try a snapshot from tomorrow and check if is ok.

thewild

@ermal:

Please try a snapshot from tomorrow and check if is ok.

OK, I'll try it ASAP.
Server is in production, so I have to do this outside of business hours.

thewild

@ermal:

Please try a snapshot from tomorrow and check if is ok.

Just tried with the latest snapshot, and it works great !
~28.000 folders scanned by the FTP client in less than 10 minutes, no deconnection !
That's perfect, thanks a lot !