Outbound FTP Problems
-
Eugene,
OK, I did that. I can get to the FreeBSD FTP site without a problem, but I still can't get to the FTP site they need to access. The site is an EDI system at PubNet.org. The way you log into it is to access an FTP site with a username and password over the command line. That site sends back a response:
230 User authenticated to proxy
I then have to issue a command:
user "xxxxxx@ <host>xxxxx" SITE
That command results in a response that says:
230-User logged in, proceed.
Current Default Relationship - Recv: xxxxxx APRF: *BINARY
Get option: singleI am supposed to be able to send an ls command from that point and see a list of files, but when I issue the ls command, it hangs and then terminates the connection.
I am running m0n0wall at my house and I can access this system from there no problem, with no special setup, from my LAN connection.
Note: If I try to access the first site via a browser, after I authenticate it sends back:
500 user must be connected to remote site
and terminates the connection.</host>
-
Well, my friend 'tcpdump' helps me a lot if I do not understand why something does not work.
Please do simultaneous traces on LAN and WAN
tcpdump -i <lan>-n -s0 host x.x.x.x
tcpdump -i <wan>-n -s0 host x.x.x.xreplace <lan>and <wan>with you interfaces names and x.x.x.x with real IP of your FTP site.</wan></lan></wan></lan>
-
Hmm. OK, I did tcpdump on the LAN interface and in that process I see that the second ftp server has no external DNS and I'm guessing no external IP address, which is why they are using an FTP proxy server to get to it. Again, this works in m0n0wall without a hitch.
When you say do simultaneous traces, you mean using SSH to get into the shell onn the pfsense appliance? And pardon me for ignorance, but how would I run simultaneous traces, using two different shells? I will give that a try from my own system and from inside the LAN there and see what I can see.
-
I use PuTTY to connect to my firewalls. You can open many sessions at the same moment. In one session you can trace traffic on LAN, in the second one - traffic on WAN.
Can you give your traces here please? -
OK, sorry this took me so long to get back, I've been on other stuff. I set the dumps and did the same process as before. This is on a machine on the LAN subnet that has an internal only IP address and no port forwarding or other firewall stuff associated with it.
These dumps end after the ls command, which returns nothing.
Here is the WAN dump:
~# tcpdump -i <wan>-n -s0 host 204.90.130.189
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vr0, link-type EN10MB (Ethernet), capture size 65535 bytes
19:38:54.936637 IP 69.198.163.10.22159 > 204.90.130.189.21: S 1975913199:1975913199(0) win 65228
<mss 0="" 218625506="" 1460,nop,wscale="" 4,sackok,timestamp="">19:38:55.016243 IP 204.90.130.189.21 > 69.198.163.10.22159: S 1024348918:1024348918(0) ack
1975913200 win 24616 <nop,nop,timestamp 1596847992="" 218625506,nop,wscale="" 0,nop,nop,sackok,mss="" <br="">1460>
19:38:55.016356 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 1 win 4163 <nop,nop,timestamp <br="">218625514 1596847992>
19:38:55.112753 IP 204.90.130.189.21 > 69.198.163.10.22159: P 1:8(7) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.112868 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 8 win 4162 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.113709 IP 204.90.130.189.21 > 69.198.163.10.22159: P 8:83(75) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.113787 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 83 win 4158 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.113899 IP 204.90.130.189.21 > 69.198.163.10.22159: P 83:146(63) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.113967 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 146 win 4159 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.114780 IP 204.90.130.189.21 > 69.198.163.10.22159: P 146:221(75) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.114856 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 221 win 4158 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.114918 IP 204.90.130.189.21 > 69.198.163.10.22159: P 221:296(75) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.114980 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 296 win 4153 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.115046 IP 204.90.130.189.21 > 69.198.163.10.22159: P 296:374(78) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.115109 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 374 win 4148 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.115221 IP 204.90.130.189.21 > 69.198.163.10.22159: P 374:452(78) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.115288 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 452 win 4158 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.116601 IP 204.90.130.189.21 > 69.198.163.10.22159: P 452:511(59) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.116676 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 511 win 4159 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.116734 IP 204.90.130.189.21 > 69.198.163.10.22159: P 511:514(3) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.116796 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 514 win 4159 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.116997 IP 204.90.130.189.21 > 69.198.163.10.22159: P 514:593(79) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.117073 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 593 win 4158 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.117128 IP 204.90.130.189.21 > 69.198.163.10.22159: P 593:669(76) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.117214 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 669 win 4153 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.117249 IP 204.90.130.189.21 > 69.198.163.10.22159: P 669:742(73) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.117312 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 742 win 4148 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.117542 IP 204.90.130.189.21 > 69.198.163.10.22159: P 742:792(50) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.117616 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 792 win 4159 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.117655 IP 204.90.130.189.21 > 69.198.163.10.22159: P 792:795(3) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.117719 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 795 win 4159 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.120038 IP 204.90.130.189.21 > 69.198.163.10.22159: P 795:928(133) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.120114 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 928 win 4154 <nop,nop,timestamp <br="">218625524 1596848002>
19:38:55.120151 IP 204.90.130.189.21 > 69.198.163.10.22159: P 928:969(41) ack 1 win 24616
<nop,nop,timestamp 218625514="" 1596848002="">19:38:55.120231 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 969 win 4152 <nop,nop,timestamp <br="">218625524 1596848002>
19:39:00.397566 IP 69.198.163.10.22159 > 204.90.130.189.21: P 1:14(13) ack 969 win 4163
<nop,nop,timestamp 218626052="" 1596848002="">19:39:00.476959 IP 204.90.130.189.21 > 69.198.163.10.22159: . ack 14 win 24616 <nop,nop,timestamp <br="">1596848538 218626052>
19:39:00.685828 IP 204.90.130.189.21 > 69.198.163.10.22159: P 969:1015(46) ack 14 win 24616
<nop,nop,timestamp 218626052="" 1596848560="">19:39:00.685968 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 1015 win 4160
<nop,nop,timestamp 218626081="" 1596848560="">19:39:04.983553 IP 69.198.163.10.22159 > 204.90.130.189.21: P 14:29(15) ack 1015 win 4163
<nop,nop,timestamp 218626511="" 1596848560="">19:39:05.160085 IP 204.90.130.189.21 > 69.198.163.10.22159: . ack 29 win 24616 <nop,nop,timestamp <br="">1596849007 218626511>
19:39:05.160275 IP 204.90.130.189.21 > 69.198.163.10.22159: P 1015:1048(33) ack 29 win 24616
<nop,nop,timestamp 218626511="" 1596849007="">19:39:05.160357 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 1048 win 4160
<nop,nop,timestamp 218626528="" 1596849007="">19:40:02.484009 IP 69.198.163.10.22159 > 204.90.130.189.21: P 29:64(35) ack 1048 win 4163
<nop,nop,timestamp 218632261="" 1596849007="">19:40:02.652695 IP 204.90.130.189.21 > 69.198.163.10.22159: . ack 64 win 24616 <nop,nop,timestamp <br="">1596854757 218632261>
19:40:02.693315 IP 204.90.130.189.21 > 69.198.163.10.22159: P 1048:1099(51) ack 64 win 24616
<nop,nop,timestamp 218632261="" 1596854760="">19:40:02.693420 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 1099 win 4159
<nop,nop,timestamp 218632282="" 1596854760="">19:40:02.693496 IP 204.90.130.189.21 > 69.198.163.10.22159: P 1099:1154(55) ack 64 win 24616
<nop,nop,timestamp 218632261="" 1596854760="">19:40:02.693562 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 1154 win 4156
<nop,nop,timestamp 218632282="" 1596854760="">19:40:02.694337 IP 204.90.130.189.21 > 69.198.163.10.22159: P 1154:1190(36) ack 64 win 24616
<nop,nop,timestamp 218632261="" 1596854760="">19:40:02.694410 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 1190 win 4160
<nop,nop,timestamp 218632282="" 1596854760="">19:40:02.888632 IP 69.198.163.10.22159 > 204.90.130.189.21: P 64:79(15) ack 1190 win 4163
<nop,nop,timestamp 218632301="" 1596854760="">19:40:02.993462 IP 204.90.130.189.21 > 69.198.163.10.22159: P 1190:1308(118) ack 79 win 24616
<nop,nop,timestamp 218632301="" 1596854790="">19:40:02.993554 IP 69.198.163.10.22159 > 204.90.130.189.21: . ack 1308 win 4155
<nop,nop,timestamp 218632312="" 1596854790="">19:40:06.705700 IP 69.198.163.10.22159 > 204.90.130.189.21: P 79:107(28) ack 1308 win 4163
<nop,nop,timestamp 218632683="" 1596854790="">19:40:06.784891 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:40:06.874576 IP 204.90.130.189.21 > 69.198.163.10.22159: . ack 107 win 24616
<nop,nop,timestamp 218632683="" 1596855179="">19:40:10.158748 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:40:16.903210 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:40:30.406938 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:40:48.760118 IP 69.198.163.10.22159 > 204.90.130.189.21: F 107:107(0) ack 1308 win 4163
<nop,nop,timestamp 218636888="" 1596855179="">19:40:48.840269 IP 204.90.130.189.21 > 69.198.163.10.22159: . ack 108 win 24616
<nop,nop,timestamp 218636888="" 1596859375="">19:40:57.404067 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:41:51.399305 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">And here is the LAN dump:tcpdump -i vr1 -n -s0 host 204.90.130.189
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vr1, link-type EN10MB (Ethernet), capture size 65535 bytes
19:38:54.935640 IP 192.168.1.61.49988 > 204.90.130.189.21: S 3223142984:3223142984(0) win 8192 <mss 1460,nop,wscale="" 0,nop,nop,sackok="">19:38:54.935937 IP 204.90.130.189.21 > 192.168.1.61.49988: S 3843978849:3843978849(0) ack 3223142985 win 65228 <mss 1460,nop,wscale="" 4,sackok,eol="">19:38:54.936096 IP 192.168.1.61.49988 > 204.90.130.189.21: . ack 1 win 8192
19:38:55.113112 IP 204.90.130.189.21 > 192.168.1.61.49988: P 1:8(7) ack 1 win 4106
19:38:55.310178 IP 192.168.1.61.49988 > 204.90.130.189.21: . ack 8 win 8185
19:38:55.310278 IP 204.90.130.189.21 > 192.168.1.61.49988: P 8:969(961) ack 1 win 4106
19:38:55.510188 IP 192.168.1.61.49988 > 204.90.130.189.21: . ack 969 win 7224
19:39:00.397169 IP 192.168.1.61.49988 > 204.90.130.189.21: P 1:14(13) ack 969 win 7224
19:39:00.397292 IP 204.90.130.189.21 > 192.168.1.61.49988: . ack 14 win 4105
19:39:00.686198 IP 204.90.130.189.21 > 192.168.1.61.49988: P 969:1015(46) ack 14 win 4106
19:39:00.881619 IP 192.168.1.61.49988 > 204.90.130.189.21: . ack 1015 win 7178
19:39:04.983185 IP 192.168.1.61.49988 > 204.90.130.189.21: P 14:29(15) ack 1015 win 7178
19:39:04.983296 IP 204.90.130.189.21 > 192.168.1.61.49988: . ack 29 win 4105
19:39:05.160580 IP 204.90.130.189.21 > 192.168.1.61.49988: P 1015:1048(33) ack 29 win 4106
19:39:05.350973 IP 192.168.1.61.49988 > 204.90.130.189.21: . ack 1048 win 7145
19:40:02.483643 IP 192.168.1.61.49988 > 204.90.130.189.21: P 29:64(35) ack 1048 win 7145
19:40:02.483758 IP 204.90.130.189.21 > 192.168.1.61.49988: . ack 64 win 4104
19:40:02.693788 IP 204.90.130.189.21 > 192.168.1.61.49988: P 1048:1154(106) ack 64 win 4106
19:40:02.885550 IP 192.168.1.61.49988 > 204.90.130.189.21: . ack 1154 win 7039
19:40:02.885641 IP 204.90.130.189.21 > 192.168.1.61.49988: P 1154:1190(36) ack 64 win 4106
19:40:02.888309 IP 192.168.1.61.49988 > 204.90.130.189.21: P 64:79(15) ack 1190 win 7003
19:40:02.888388 IP 204.90.130.189.21 > 192.168.1.61.49988: . ack 79 win 4105
19:40:02.993737 IP 204.90.130.189.21 > 192.168.1.61.49988: P 1190:1308(118) ack 79 win 4106
19:40:03.186569 IP 192.168.1.61.49988 > 204.90.130.189.21: . ack 1308 win 6885
19:40:06.705236 IP 192.168.1.61.49988 > 204.90.130.189.21: P 79:105(26) ack 1308 win 6885
19:40:06.705360 IP 204.90.130.189.21 > 192.168.1.61.49988: . ack 105 win 4104
19:40:48.759724 IP 192.168.1.61.49988 > 204.90.130.189.21: F 105:105(0) ack 1308 win 6885
19:40:48.759846 IP 204.90.130.189.21 > 192.168.1.61.49988: . ack 106 win 4106
19:40:48.760018 IP 204.90.130.189.21 > 192.168.1.61.49988: F 1308:1308(0) ack 106 win 4106
19:40:48.760169 IP 192.168.1.61.49988 > 204.90.130.189.21: . ack 1309 win 6885</mss></mss></nop,nop,sackok,mss></nop,nop,sackok,mss></nop,nop,timestamp></nop,nop,timestamp></nop,nop,sackok,mss></nop,nop,sackok,mss></nop,nop,sackok,mss></nop,nop,timestamp></nop,nop,sackok,mss></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></nop,nop,timestamp></mss></wan> -
Out of curiosity, are you running Snort on this system?
I've recently hit a couple roadblocks with a client getting to an FTP server, and it was because the snort FTP preprocessor drops what it believes to be malformed FTP commands, even when Snort itself is not configured to block. The behavior I saw is identical to what you describe.
The two commands I had trouble with are EPSV, and SITE. I fixed EPSV (it was just missing from the list of allowed commands) but haven't had a chance to look further at SITE.
In that case, it may just be a matter of correcting Snort's interpretation of the SITE command and what is good/bad.
-
Also, from the console, you may find the tcpflow package more helpful when diagnosing FTP (And other plaintext protocols):
Add tcpflow and make it show up like so:
pkg_add -r tcpflow rehash
And then execute it much like tcpdump. The -c parameter makes it output to console instead of to a log file.
tcpflow -c -i <wan> host 204.90.130.189</wan>
-
From here:
19:40:06.784891 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:40:10.158748 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:40:16.903210 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:40:30.406938 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:40:57.404067 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">19:41:51.399305 IP 204.90.130.189.20 > 69.198.163.10.60146: S 1275924435:1275924435(0) win 24820
<nop,nop,sackok,mss 1460="">we can clearly see that you are trying to work in passive FTP mode but the server can not establish connection to your PC. According to your dump on LAN side these SYN packets do not go through pfSense, it is weird, I think ftp-proxy should handle this, double check please that you did not put check-mark on 'Disable the userland FTP-Proxy application' for LAN interface and make sure it is checked on WAN.
Try to use active FTP mode please. If you give us dumps for this situation it will be great.EDITED: sorry, never played with snort, can't say anything.</nop,nop,sackok,mss></nop,nop,sackok,mss></nop,nop,sackok,mss></nop,nop,sackok,mss></nop,nop,sackok,mss></nop,nop,sackok,mss>
-
Thanks for the continued help, I appreciate it.
First, the system is running embedded on a CF card, so no snort (or any other packages) are running.
The ftp-proxy is disabled on WAN and active on LAN.
I can't use pkg_add to get tcpflow because the ftp site has moved from where pfsense thinks the package resides (7.1 instead of 7.0). Is there something like 'apt-get update' to refresh the locations? Sorry, I'm not that familiar with BSD. I'd rather not monkey with ftp and remounting the filesystem read-write manually as this system is in production during a busy season for my clients.
According to my understanding, ftp.exe on Windows does not support passive mode FTP. I am trying to use ncftp, but I'm not exactly clear on what syntax to substitute for the user command on ftp.exe.
-
can you give us
pfctl -sr | grep ftp