Possible FTP helper bug…



  • I see a similar thread, but I think this is slightly different.

    It's more related to the bug noted in the closed ticket #1100 http://cvstrac.pfsense.org/tktview?tn=1100

    Something has definitely changed although my settings have not. Running latest snapshot 1.0-SNAPSHOT-09-07-06 (which is actually 09-06-06…) Now unfortunately the FTP server I'm connecting to at the office is also behind the same version of pfSense so I'm not completely sure which direction has the problem now. But ACTIVE mode definitely doesn't work. Actually I just tested from the other end to my FTP server on this end, and the same problem exists. PASSIVE mode is fine but ACTIVE fails. This has worked fine as far as I've noticed right up to this snapshot, and I've kept up-to-date as much as possible.

    I just tried it from another location where I have also installed pfSense, but it's still at 1.0-SNAPSHOT-09-04-06, outbound FTP in ACTIVE mode works fine. However, FTP to the FTP server behind that pfSense from here or the office with the 1.0-SNAPSHOT-09-06-06 does not work in ACTIVE mode, only PASSIVE. So it does seem that something bug has crept into the latest snapshot.

    Hope this helps track it down. Thanks for all your hard and EXCELLENT work!  ;D  8)

    JC (Superman)



  • This doesn't really help.  We need to know more information.  We need to know if this is being published to the internet, what site in question, is this a vpn connection, etc.

    Unfortunately your bug report just results in me being more confused.  Please try to outline it a little more.



  • Here's a few snippets. It's not much more than what I explained earlier. I'm not quite sure what other information you need and how to get it. I tried looking in the logs, but there's just nothing there. I uses ps on both and determined that pftpx is running for sure. What else can I do to get you more information?

    Examples.
    Behind 1.0-SNAPSHOT-09-06-06 (This connection is running Cable Internet using DHCP):

    # ftp ftp.freebsd.org
    Trying 204.152.184.73...
    Connected to ftp.freebsd.org (204.152.184.73).
    220 Welcome to freebsd.isc.org.
    Name (ftp.freebsd.org:root): anonymous
    530 Please login with USER and PASS.
    SSL not available
    331 Please specify the password.
    Password:
    230-
    230-You have reached the freebsd.isc.org FTP server, serving the
    230-full FreeBSD FTP archive over IPv4 (204.152.184.73) and IPv6
    230-(2001:4f8:0:2::e) networks.  This server is also known as:
    230-
    230-    ftp.freebsd.org
    230-    ftp4.freebsd.org
    230-    ftp4.us.freebsd.org
    230-
    230-This server is operated by Internet Systems Consortium (ISC),
    230-on behalf of the FreeBSD Project, with hardware donations from
    230-Apple, Intel and Iron Systems.
    230-
    230-Questions about this service can be sent to: freebsd@isc.org.
    230-
    230 Login successful.
    Remote system type is UNIX.
    Using binary mode to transfer files.
    ftp> ls
    500 Illegal PORT command.
    ftp: bind: Address already in use
    ftp> passive
    Passive mode on.
    ftp> ls
    227 Entering Passive Mode (204,152,184,73,228,157)
    150 Here comes the directory listing.
    drwxrwxr-x    3 0        0             512 Apr 17  2003 pub
    226 Directory send OK.
    ftp> 
    
    # ps wwaux | grep pftpx
    proxy    558  0.0  0.3   656   412  ??  Ss   Thu09PM   0:01.45 /usr/local/sbin/pftpx -c 8021 -g 8021 192.168.1.1
    

    Behind 1.0-SNAPSHOT-09-04-06 (This connection is running ADSL using PPPoE):

    # ftp ftp.freebsd.org
    Connected to ftp.freebsd.org.
    220 Welcome to freebsd.isc.org.
    Name (ftp.freebsd.org:root): anonymous
    331 Please specify the password.
    Password:
    230-
    230-You have reached the freebsd.isc.org FTP server, serving the
    230-full FreeBSD FTP archive over IPv4 (204.152.184.73) and IPv6
    230-(2001:4f8:0:2::e) networks.  This server is also known as:
    230-
    230-    ftp.freebsd.org
    230-    ftp4.freebsd.org
    230-    ftp4.us.freebsd.org
    230-
    230-This server is operated by Internet Systems Consortium (ISC),
    230-on behalf of the FreeBSD Project, with hardware donations from
    230-Apple, Intel and Iron Systems.
    230-
    230-Questions about this service can be sent to: freebsd@isc.org.
    230-
    230 Login successful.
    Remote system type is UNIX.
    Using binary mode to transfer files.
    ftp> ls
    200 PORT command successful. Consider using PASV.
    150 Here comes the directory listing.
    drwxrwxr-x    3 0        0             512 Apr 17  2003 pub
    226 Directory send OK.
    ftp> 
    
    # ps wwaux | grep pftpx
    proxy    500  0.0  0.2   656   420  ??  Ss   Wed06PM   0:02.92 /usr/local/sbin/pftpx -c 8021 -g 8021 192.168.1.1
    

    Both system have two interfaces, one WAN, one LAN. LAN is 192.168.1.x/24 on both as well. The other FTP servers I was talking about are private servers connected to the internet through each respective pfSense firewall. All are running PureFTPd, one on FreeBSD, one on Gentoo Linux and the other on Debian Linux. Previous to updating to this snapshot  (1.0-SNAPSHOT-09-06-06) everything "just worked," no matter what mode.

    Both are using these settings:








  • I just "reflashed" my local pfSense that was running 1.0-SNAPSHOT-09-06-06 to 1.0-SNAPSHOT-09-04-06 (or is that a "flashback" ;) ) and now FTP works in ACTIVE mode.

    # ftp ftp.freebsd.org
    Trying 204.152.184.73...
    Connected to ftp.freebsd.org (204.152.184.73).
    220 Welcome to freebsd.isc.org.
    Name (ftp.freebsd.org:root): anonymous
    530 Please login with USER and PASS.
    SSL not available
    331 Please specify the password.
    Password:
    230-
    230-You have reached the freebsd.isc.org FTP server, serving the
    230-full FreeBSD FTP archive over IPv4 (204.152.184.73) and IPv6
    230-(2001:4f8:0:2::e) networks.  This server is also known as:
    230-
    230-    ftp.freebsd.org
    230-    ftp4.freebsd.org
    230-    ftp4.us.freebsd.org
    230-
    230-This server is operated by Internet Systems Consortium (ISC),
    230-on behalf of the FreeBSD Project, with hardware donations from
    230-Apple, Intel and Iron Systems.
    230-
    230-Questions about this service can be sent to: freebsd@isc.org.
    230-
    230 Login successful.
    Remote system type is UNIX.
    Using binary mode to transfer files.
    ftp> ls
    200 PORT command successful. Consider using PASV.
    150 Here comes the directory listing.
    drwxrwxr-x    3 0        0             512 Apr 17  2003 pub
    226 Directory send OK.
    ftp> 
    

    And this now shows in the logs:

    Sep 9 07:30:34 	pftpx[558]: #1 client reset connection
    Sep 9 07:30:34 	pftpx[558]: #1 client reset connection
    

    Beyond changing back to the previous snapshot no other settings were changed.



  • So now everything is okay?



  • Yes, it all works properly with 1.0-SNAPSHOT-09-04-06.

    Interestingly, as I'm sure you're well aware, in the cvstrac timeline, I count about 4 commits that all reference the FTP helper between the two snapshots. Something  in those changes must be affecting it's operation.

    Thanks again!!



  • So you are having trouble with VPN + FTP basically?



  • Nope, no VPN involved, just FTP.

    I did notice that those changes did reference VPN, so it does seem odd that they are affecting just regular FTP over the Internet, so maybe it's not those. But some change between those 2 snapshots is certainly affecting FTP. Sorry I know that is a little vague, but I can't point to anything in particular that could be causing it.

    Thanks again!



  • Then I really don't know.  We only changed code related to incoming FTP + reflection recently.  Nothing else, unfortunately.

    Trouble is FTP is working perfectly fine for me outgoing.



  • Not sure what client you're using, but I noticed with SmartFTP which I was using it automatically falls back to PASSIVE if ACTIVE doesn't work. So I didn't notice the problem at first. But when I added items to the Queue they wouldn't download, because the queue section doesn't have the fallback feature, and was just trying ACTIVE mode. So then I tested with CLI ftp on Linux, so I could limit the mode to one or the other and then I noticed that for sure ACTIVE mode wasn't working. I confirmed this behind 2 different pfSense firewalls both running the 09-06-06, and that behind a third pfSense box @ 09-04-06 that it worked.

    All these firewalls have very "stock" settings. Pretty much simply using the stock settings with a few NATed services.



  • Using a FTP client (FreeBSD's):

    226 Directory send OK.
    ftp> passive off
    Passive mode: off; fallback to active mode: off.
    ftp> get rawrite.exe
    local: rawrite.exe remote: rawrite.exe
    200 EPRT command successful. Consider using EPSV.
    150 Opening BINARY mode data connection for rawrite.exe (36064 bytes).
    100% || 36064      57.35 KB/s    00:00 ETA
    226 File send OK.
    36064 bytes received in 00:00 (47.58 KB/s)
    ftp> passive on
    Passive mode: on; fallback to active mode: off.
    ftp> get rawrite.exe
    local: rawrite.exe remote: rawrite.exe
    229 Entering Extended Passive Mode (|||53759|)
    150 Opening BINARY mode data connection for rawrite.exe (36064 bytes).
    100% |
    | 36064      71.84 KB/s    00:00 ETA
    226 File send OK.
    36064 bytes received in 00:00 (58.50 KB/s)
    ftp> passive auto
    Passive mode: on; fallback to active mode: on.
    ftp> get rawrite.exe
    local: rawrite.exe remote: rawrite.exe
    229 Entering Extended Passive Mode (|||59392|)
    150 Opening BINARY mode data connection for rawrite.exe (36064 bytes).
    100% |*************************************| 36064      71.25 KB/s    00:00 ETA
    226 File send OK.
    36064 bytes received in 00:00 (58.18 KB/s)
    ftp>



  • Well, I don't know what to say. It was certainly a problem on my systems until I went back to the aforementioned snapshot. Strange. I'll try re-updating to the latest snapshot to see if something funny happened the last time. But it seems odd that the same "funny" thing would happen on 2 different machines.

    Thanks for your attention to this anyway! 8)



  • Okay, I reapplied the 09-06-06 snapshot and like clockwork the problem resurfaced. I don't know what it is, but for me FTP ACTIVE mode really doesn't work with this snapshot. :( I'll just go back to 09-04-06 for now…

    Thanks!



  • Indeed there was a bug, if you did not have a vpn defined, it would not have installed the rule.

    Issue these commands from a shell to test:

    /etc/rc.conf_mount_rw
    fetch -o /etc/inc/ http://www.pfsense.com/~sullrich/filter.inc
    /etc/rc.filter_configure



  • Should I reapply 09-06-06 first??

    • Oops, duh, since it's not a problem in 04 I guess I should…sorry for the stupid question...off to update and apply! *


  • Yep.



  • Oops, missed your reply before my edit…

    Anyway, reapplied 09-06-06 and then followed your directions and voila!! It works!! Awesome!! You rock!! :D

    Thanks!! 8)


Locked