Strange FTP issue going from behind pfsense and out
-
Not sure if this is the correct forum but here goes.
Running pfSense 2.1-RELEASE
We have pfSense setup in bridged mode, so no NAT whatsoever.
For our setup we followed this guide -> http://goo.gl/8rBXrSEverything works alright except for a couple of our Windows servers having trouble with FTP sessions and webservice sessions with one of our partners.
The FTP problem is very strange. No matter what GUI FTP client i try (Filezilla, CoreFTP, WS FTP) i cannot connect to the external FTP of our partner. If i use the commandline FTP however it works. This is only to that specific partners FTP, others work just fine.
A similar problem exist when we run webservice queries with this partner, sometimes it works but basically 50% of the traffic is lost somewhere.
We think this problem started when we installed pfSense 2.1 but we are not sure.
We were running 1.2.3 before that.If we had problems with all FTP connections and webservice calls this would be so much easier but now its only with this one specific partner.
Im not sure if its pfSense, Windows or even the partner thats to blame.
Does anyone have any idea where to start? -
Do a packet capture for an FTP session that fails, and compare that to an FTP session that successfully logs in. See if you can determine what's being lost or blocked.
-
I did some capturing of both a session succeeding and a failing.
I did the capturing on the WAN interface since our firewall is running in bridge mode.
(attached)Also i have noticed incoming FTP being affected, but not as much.
Could it be a problem with ftp proxy handler? Like i said i am running 2.0 RELEASE.
I did however disable the handler in tunables but i could not see any change.Failed FTP session
No. Time Source Destination Protocol Length Info 1 0.000000 81.92.76.37 193.13.207.1 TCP 60 50061→21 [FIN, ACK] Seq=1 Ack=1 Win=63978 Len=0 Frame 1: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 50061 (50061), Dst Port: 21 (21), Seq: 1, Ack: 1, Len: 0 No. Time Source Destination Protocol Length Info 2 0.003970 81.92.76.37 193.13.207.1 TCP 66 50068→21 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 Frame 2: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 50068 (50068), Dst Port: 21 (21), Seq: 0, Len: 0 No. Time Source Destination Protocol Length Info 3 0.004030 193.13.207.1 81.92.76.37 TCP 60 21→50061 [ACK] Seq=1 Ack=2 Win=64141 Len=0 Frame 3: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Ethernet II, Src: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01), Dst: 00:50:56:a1:00:0b (00:50:56:a1:00:0b) Internet Protocol Version 4, Src: 193.13.207.1 (193.13.207.1), Dst: 81.92.76.37 (81.92.76.37) Transmission Control Protocol, Src Port: 21 (21), Dst Port: 50061 (50061), Seq: 1, Ack: 2, Len: 0 No. Time Source Destination Protocol Length Info 4 0.013061 193.13.207.1 81.92.76.37 TCP 62 21→50068 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 SACK_PERM=1 Frame 4: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) Ethernet II, Src: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01), Dst: 00:50:56:a1:00:0b (00:50:56:a1:00:0b) Internet Protocol Version 4, Src: 193.13.207.1 (193.13.207.1), Dst: 81.92.76.37 (81.92.76.37) Transmission Control Protocol, Src Port: 21 (21), Dst Port: 50068 (50068), Seq: 0, Ack: 1, Len: 0 No. Time Source Destination Protocol Length Info 5 0.041903 193.13.207.1 81.92.76.37 TCP 60 21→50061 [FIN, ACK] Seq=1 Ack=2 Win=64141 Len=0 Frame 5: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Ethernet II, Src: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01), Dst: 00:50:56:a1:00:0b (00:50:56:a1:00:0b) Internet Protocol Version 4, Src: 193.13.207.1 (193.13.207.1), Dst: 81.92.76.37 (81.92.76.37) Transmission Control Protocol, Src Port: 21 (21), Dst Port: 50061 (50061), Seq: 1, Ack: 2, Len: 0 No. Time Source Destination Protocol Length Info 6 0.042081 81.92.76.37 193.13.207.1 TCP 60 50061→21 [ACK] Seq=2 Ack=2 Win=63978 Len=0 Frame 6: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 50061 (50061), Dst Port: 21 (21), Seq: 2, Ack: 2, Len: 0 No. Time Source Destination Protocol Length Info 7 3.005397 81.92.76.37 193.13.207.1 TCP 66 [TCP Spurious Retransmission] 50068→21 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 Frame 7: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 50068 (50068), Dst Port: 21 (21), Seq: 0, Len: 0 No. Time Source Destination Protocol Length Info 8 3.008199 193.13.207.1 81.92.76.37 TCP 62 [TCP Retransmission] 21→50068 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 SACK_PERM=1 Frame 8: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) Ethernet II, Src: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01), Dst: 00:50:56:a1:00:0b (00:50:56:a1:00:0b) Internet Protocol Version 4, Src: 193.13.207.1 (193.13.207.1), Dst: 81.92.76.37 (81.92.76.37) Transmission Control Protocol, Src Port: 21 (21), Dst Port: 50068 (50068), Seq: 0, Ack: 1, Len: 0 No. Time Source Destination Protocol Length Info 9 9.011196 193.13.207.1 81.92.76.37 TCP 62 [TCP Retransmission] 21→50068 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 SACK_PERM=1 Frame 9: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) Ethernet II, Src: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01), Dst: 00:50:56:a1:00:0b (00:50:56:a1:00:0b) Internet Protocol Version 4, Src: 193.13.207.1 (193.13.207.1), Dst: 81.92.76.37 (81.92.76.37) Transmission Control Protocol, Src Port: 21 (21), Dst Port: 50068 (50068), Seq: 0, Ack: 1, Len: 0 No. Time Source Destination Protocol Length Info 10 9.020499 81.92.76.37 193.13.207.1 TCP 62 [TCP Spurious Retransmission] 50068→21 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 SACK_PERM=1 Frame 10: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 50068 (50068), Dst Port: 21 (21), Seq: 0, Len: 0 No. Time Source Destination Protocol Length Info 11 25.021070 81.92.76.37 193.13.207.1 TCP 66 50072→21 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 Frame 11: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 50072 (50072), Dst Port: 21 (21), Seq: 0, Len: 0 No. Time Source Destination Protocol Length Info 12 25.025333 193.13.207.1 81.92.76.37 TCP 62 21→50072 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 SACK_PERM=1 Frame 12: 62 bytes on wire (496 bits), 62 bytes captured (496 bits) Ethernet II, Src: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01), Dst: 00:50:56:a1:00:0b (00:50:56:a1:00:0b) Internet Protocol Version 4, Src: 193.13.207.1 (193.13.207.1), Dst: 81.92.76.37 (81.92.76.37) Transmission Control Protocol, Src Port: 21 (21), Dst Port: 50072 (50072), Seq: 0, Ack: 1, Len: 0
Success
No. Time Source Destination Protocol Length Info 1 0.000000 81.92.76.37 193.13.207.1 TCP 60 57729→21 [FIN, ACK] Seq=1 Ack=1 Win=512 Len=0 Frame 1: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 57729 (57729), Dst Port: 21 (21), Seq: 1, Ack: 1, Len: 0 No. Time Source Destination Protocol Length Info 2 0.004110 193.13.207.1 81.92.76.37 TCP 60 21→57729 [ACK] Seq=1 Ack=2 Win=256 Len=0 Frame 2: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Ethernet II, Src: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01), Dst: 00:50:56:a1:00:0b (00:50:56:a1:00:0b) Internet Protocol Version 4, Src: 193.13.207.1 (193.13.207.1), Dst: 81.92.76.37 (81.92.76.37) Transmission Control Protocol, Src Port: 21 (21), Dst Port: 57729 (57729), Seq: 1, Ack: 2, Len: 0 No. Time Source Destination Protocol Length Info 3 0.005226 81.92.76.37 193.13.207.1 TCP 66 57731→21 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 Frame 3: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 57731 (57731), Dst Port: 21 (21), Seq: 0, Len: 0 No. Time Source Destination Protocol Length Info 4 0.008355 193.13.207.1 81.92.76.37 TCP 66 21→57731 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 Frame 4: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) Ethernet II, Src: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01), Dst: 00:50:56:a1:00:0b (00:50:56:a1:00:0b) Internet Protocol Version 4, Src: 193.13.207.1 (193.13.207.1), Dst: 81.92.76.37 (81.92.76.37) Transmission Control Protocol, Src Port: 21 (21), Dst Port: 57731 (57731), Seq: 0, Ack: 1, Len: 0 No. Time Source Destination Protocol Length Info 5 0.008516 81.92.76.37 193.13.207.1 TCP 60 57731→21 [ACK] Seq=1 Ack=1 Win=131328 Len=0 Frame 5: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 57731 (57731), Dst Port: 21 (21), Seq: 1, Ack: 1, Len: 0 No. Time Source Destination Protocol Length Info 6 0.032917 193.13.207.1 81.92.76.37 TCP 60 21→57729 [FIN, ACK] Seq=1 Ack=2 Win=256 Len=0 Frame 6: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Ethernet II, Src: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01), Dst: 00:50:56:a1:00:0b (00:50:56:a1:00:0b) Internet Protocol Version 4, Src: 193.13.207.1 (193.13.207.1), Dst: 81.92.76.37 (81.92.76.37) Transmission Control Protocol, Src Port: 21 (21), Dst Port: 57729 (57729), Seq: 1, Ack: 2, Len: 0 No. Time Source Destination Protocol Length Info 7 0.033079 81.92.76.37 193.13.207.1 TCP 60 57729→21 [ACK] Seq=2 Ack=2 Win=512 Len=0 Frame 7: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) Ethernet II, Src: 00:50:56:a1:00:0b (00:50:56:a1:00:0b), Dst: 5c:5e:ab:76:9c:01 (5c:5e:ab:76:9c:01) Internet Protocol Version 4, Src: 81.92.76.37 (81.92.76.37), Dst: 193.13.207.1 (193.13.207.1) Transmission Control Protocol, Src Port: 57729 (57729), Dst Port: 21 (21), Seq: 2, Ack: 2, Len: 0 No. Time Source Destination Protocol Length Info 8 0.050170 193.13.207.1 81.92.76.37 FTP 140 Response: 220-FTP Server ready...
-
Sorry, I meant a capture via pfSense so you could see if it was blocking anything (Diagnostics - Packet Capture). I've had trouble in the past configuring FTP in for servers, but I've never had an issue with LAN clients going out. I'm wondering if it's an active-passive issue. For these sites that give problems, have you tried changing the default mode?
-
That packet capture was from the WAN port of pfsense. Nothing is blocked, it just drops the package somehow. It works from other servers behind the firwall as well so its not consistent.
If i setup a SOCKS proxy on another Windows server and run FTP through that it works flawlessly.Going active or passive does not do anything. It super strange and so damn hard to troubleshoot.
Thats why im wondering if..
1. its a pfsense bug
or
2. our bridged setup is misconfigured. -
I have many users behind our pfSense instance and nobody has any problems with FTP to anywhere. FTP in gave me trouble whereas FTP out was simple. Is it always the same systems that fail to connect while others connect ever time?
-
It looks like its the same servers since we only have a few that actively use FTP.
But from more testing the FTP problem seems to be everywhere.If i enable the proxy helper on the WAN interface in pfsense i can make a external -> internal FTP connection in passive mode alright, but when i start to transfer lots of files it suddenly stops accepting connections.
If i switch off the proxy (debug.pfftpproxy=1) and connect in Direct mode everything works fine.
So yes it seems to be a passive/direct problem… somehow.
Im not even sure why the pfftpproxy is in use at all since im in transparent bridge mode. -
I'm running 2.1.4 and I don't have the FTP Proxy enabled. Do you have any restrictions on LAN going out? When you're in the middle of a failing FTP session, is there anything in the WAN firewall logs about blocking anything from the destination IP address?
-
LAN is setup to allow everything going out, nothing in the logs at all when packets are lost.
The only stuff i see on the firewall is what the packetcapture managed to snap up.I will try to upgrade pfsense to see if the problem disappears.