Passive local FTP issue

[lp]

I have a pfsense running with 4 LANs & 2 WANs.
client, C, is on LAN 1. Server S, is on LAN 2.

There's a firewall rule which allows ALL traffic from C to S.

When I open a connection from C to S, no problem. (FTP-control). The file transfer starts… so C sends the "PASV", S answers and send port information.
So C connects to S... It sends its SYN packet, then S replies with the SYN-ACK packet and boom. pfSense doesn't forward the packet, and replies to S with a ICMP packet "Destionation unreachable (host unreachable)".

I don't think it matters, but there's another firewall rule which give full from S to C.

I cannot understand it at all.

[lp]

Thanks to GeekGod who pointed me to this great website…
Here's a network capture:

http://www.cloudshark.org/captures/6d0fa1f1b55a

[jnorell]

A pfsense version would probably help, and more fw rules details.  I had a problem (http://redmine.pfsense.org/issues/863) using outbound floating rules in 2.0 with ftp connections, which may be addressed in the future, but the workaround is easy enough - don't use outbound floating rules  :).  I think both passive and active data connections were working for me (ie. with floating rules removed) last I'd checked.

[lp]

Strange, I've replied yesterday but can't see my reply. Hope it's user error and that I'm not answering a 2nd time.

I use 2.0, usually upgraded daily.

Here's the rule that allows the connection:
pass in log quick on re0_vlan5 inet proto tcp from 10.12.3.161 to <giada>flags S/SA keep state (tcp.established 300) allow-opts label "USER_RULE: Printer to Giada"

(so it allows everything, 10.12.3.161 is the FTP client, Giada is the FTP server)</giada>

[lp]

One more information:
pfctl -sa | grep "block return"

returns nothing.

So I would tend to think there's no (firewall) reason why an ICMP Host unreachable packet should be issued. And since the host exists, and is perfectly reachable from pfSense router, I'm starting to wonder what the issue can be…

[brcisna]

lp,

This really does not help you with a resolve to your problem,but what version and distro are you using as your ftp server? In about a years worth of using pfSense(-1.2.3-RELEASE) I found that 64-bit OS Centos 5  vsftp does not work were 32 bit vsftp works fine with pfSense. Must be something ground into the kernel of 64-bit that gets wonky with pfSense.
I am thinking it may be possibly some of the 'conntrak_x' stuff built into 64-bit versus 32-bit,but just a hunch,,and probably something else entirely.
I got the exact same results as you are explaining with the packet not sent back to xyz,.etc.
I tried enabling, disabling the ftp helper etc,in pfSense FYI.
Have 4 vsftp (32-bit) servers so I gave up and tracking down the 64-bit vsftp problem.
Just passing this along.

Barry

[lp]

Sorry, some more details…
Forgot to mention it's 2.0 amd64. I'll consider trying with 2.0 x86.

FTP server is vsftpd 2.0.6, running on Ubuntu 8 x86.

I'm suspecting the conntrak too (there's no FTP Helper in 2.0, and GeekGod told me it's now included at kernel level).

Thanks for the info, I'll if maybe 32 bits pfSense helps.

[lp]

I now have 2 boxes… same config, but one runs amd64, the other runs x86.
Result is the same on both.

Additional note: I have no rule allowing ICMP to go out from the pfSense box, but over SSH, I can ping 10.12.3.161 and get replies.
However, if I do a "telnet 10.12.3.161 80", it returns immediatly with:
"
Trying 10.12.3.161...
telnet: connect to address 10.12.3.161: Operation not permitted
telnet: Unable to connect to remote host
"

Connecting to 10.12.3.161 from, for example, 10.12.8.129, works fine.

If I add a floating rule:

• • • • • • none
            applying it to all interfaces, I still can't telnet from the pfSense box.

Now if I turn off the firewall, no problem, telnet goes thru.

[brcisna]

lp,

try to do a telnet on port 21 from a client machine to your ftp server's ip address

telnet 1.2.3.4 21      # server's ip address
If you do get a connect, try doing the same telnet to the machine name itself (not ip address)
See what happens.

B

[lp]

Thanks for your reply, but I'm not getting your point.
I can of course open a connection to port 21 with the IP address.
The machine has no name.
If it would have one, in best case, it'd work as well, worst case, it wouldn't work because name resolution would fail.

[lp]

I have now more interesting details…
http://www.cloudshark.org/captures/9ebde8d4aac7

This is a partial capture of a FTP session, fully done over telnet.
Partial because greetings and login are missing, and also several previous succesful attemps. I'll explain later.

Client is 10.12.3.11, server 10.12.8.136, connected over 10.12.8.253, pfSense box.

On packet 21, you can see passive mode is asked. Then on packet 129, LF is sent, to validate a "list /backup/".

At that point, I issued a "telnet 10.12.8.136 50065".

Packet 133 & 134 are the SYN packet from 10.12.3.11 relayed over 10.12.8.253 to 10.12.8.136.
Packet 136 shows the reply (SYN ACK) from 10.12.8.136, and packet 135 shows that 10.12.8.253 tells 10.12.8.136 that 10.12.3.11 is not reachable. Nothing new, was visible in previous capture. (Don't mind that packets 136 & 135 should be inversed).
3 seconds later, then SYN ACK is sent again and ignored. And again 3 seconds later (packet 140)

Some seconds later, I issued again a "telnet 10.12.8.136 50065".
And what do you see? 146 & 147 are the SYN, and miracle, 148 & 149 are the SYN ACK being routed!
And then you can see everything works fine, directory listing is tranfered.

Above, I said I had several succesfull attemps, because as long as I try to connect twice to the passive port, the 2nd attemps always succeeds. Interesting, isn't it?