Outbound and inbound FTP stopped working

kcpoole

pfsense under diag, packet capture can do all the packet capture you need to do..

If you need a host outside to confirm – just PM the info, happy to to do a test connection to it and tell you what I see active and passive what is being sent to the client, etc.

Ok thanks
Will let you know

Ken

kcpoole

@johnpoz:

pfsense under diag, packet capture can do all the packet capture you need to do..

If you need a host outside to confirm – just PM the info, happy to to do a test connection to it and tell you what I see active and passive what is being sent to the client, etc.

I Did some testing and the IP is being rewritten as it is supposed to.
Some of the Connection attempts i have made work for a moment or two but most just fail at the data connection and times out

ken

Screenshot.png_thumb

kejianshi

I'm confused. FTP is dirt simple.
So my first thought is what might you have changed that is breaking it?

kcpoole

@kejianshi:

I'm confused. FTP is dirt simple.
So my first thought is what might you have changed that is breaking it?

As far as I can tell, Nothing has changed at my servers, The remote server, or the firewall. The Firewall was rebooted after power failure and then next time the users could not connect sucessfully and upload files:-(

just trying an inbound connection to my own FTP server while capture the traffic and this is the capture once already connected.

the 114 IP is my firewall wan address and 60.240 is my client here that I am trying to connect from.

it does not matter it seems whether I am trying to connect into my ftp server or out to an external one, the ftp connection is setup correctly but once data is to be tranferred it fails.

Ken

Screenshot-1.png_thumb

kejianshi

Are you using a static public IP or dynamic one? Do you use a dynamic dns updater?

I'm wondering if its as simple as your IP changed.

Also thinking if its not, wipe the pfsense and reinstall then restore your config.

johnpoz

I don't see any data attempts in your post.. And where are you sniffing, see send fin,ack and then retrans it twice - I assume that is your wan.. So what does pfsense have to do with not seeing a a response and sending retrans? Not like the answer is seen at the wan at pfsense?

In the first looks like see the syn from 182 to 114, and then 114 sending back syn,ack - and then sending it 2 more times.. because no answer? That is on your wan is in not - so what does pfsense have to do with no answer from 182?

can you post up the sniff, so we can look at the details? To me looks like you have problem outside pfsense. If these sniffs are taken on the pfsense wan?

kcpoole

@johnpoz:

I don't see any data attempts in your post.. And where are you sniffing, see send fin,ack and then retrans it twice - I assume that is your wan.. So what does pfsense have to do with not seeing a a response and sending retrans? Not like the answer is seen at the wan at pfsense?

In the first looks like see the syn from 182 to 114, and then 114 sending back syn,ack - and then sending it 2 more times.. because no answer? That is on your wan is in not - so what does pfsense have to do with no answer from 182?

can you post up the sniff, so we can look at the details? To me looks like you have problem outside pfsense. If these sniffs are taken on the pfsense wan?

Yes the sniffs are on the outside interface using pfsense to take them.
the last one posted was limited to a single IP and port 21 as there is much other traffic to that host.

Happy to post the full sniff but how can i remove the password from it? i do nt really want to post the ftp details on the web

Ken

kcpoole

@johnpoz:

To me looks like you have problem outside pfsense.

I am wondering that myself. how can I prove or test that thought?
I am setting up another host on my network that is outside the pfsense and hope I can find the issue there.

Ken

kcpoole

Does anyone have an ftp server I can test uploading to?
The one my client is trying to send to does not work for me, I have my own that i use at my home and that works fine :-(

TIA
Ken

johnpoz

Valid point with the passwords - which is why there are anon tools for sniffs ;)

You could use http://www.tracewrangler.com/ to remove the passwords..

So see my first sniff where there billy password sent to ftp.microsoft – You add an anon task, set everything to passthrough except the text part. You put in the original and what you want to replace it with. See 1st attachment

You run the task, and then the new pcap it creates doesn't have your password in there ;) But all the other info is in tact for looking at what could be wrong..

You can anon other stuff as well like IPs, etc. Don't go all crazy on it, it can make it difficult to spot issues if too much manipulation is done.

replacedcopy.png_thumb

anonpasswordsout.png_thumb

kcpoole

Thanks John for the link to the anonymizer

I have rerun the capture and attached it below
Any help to work out why it is failing will be great.

note: I have appended .txt to the end of the file so i can upload.

Thanks
Ken

[Capture - FailFTP_anon.pcapng.txt](/public/imported_attachments/1/Capture - FailFTP_anon.pcapng.txt)

johnpoz

well clearly this is wrong.

So see the port command to IP 114 port 52030

Why is it trying to go to IP 170?

And where is this address coming from - see second image.

wrongip.png_thumb

wrongaddress.png_thumb

kcpoole

Ok i did not see them at all.
the Address 114 is my own server and the server on 182.50.153.244 is the host I am trying to get to
I have no idea what the other IP addresses are and what they are doing in the converstaion. thanks I will have to investigate where they fit inot it.

Ken

johnpoz

well that is not the capture you gave then..

This is 170 talking to a 88, that seems to report its IP address is 182 in the banner.

The port command yes tells this server 88 hey come talk to 114 on port 52030 for the data connection, but the SYN from 20 source port to 52030 is going to to 170. Which yeah never answers.. You see the retrans, and you see it tell it hey come talk to more ports as well where the port commands are 114, but traffic is to 170.x.x.x

You have dual wan connection on this pfsense?

kcpoole

Nope only single wan network connected

is there any possibility that somehow the port 20 traffic redirected or hacked?

What does the handling of the traffic on port 20 ?

Ken

kcpoole

@johnpoz:

well that is not the capture you gave then..

When i compare the non anoymised capture it show the correct IP addresses wher the one I posted has different IPs in it !

Sorry but the process must have changed them all. I will repost the other.

Ken

kcpoole

Okay sorry but In the original file i uploaded i anonymized the IP addresses too by mistake
Redid the process and here is the correct file

Ken

[Capture - FailFTP_anon.pcapng.txt](/public/imported_attachments/1/Capture - FailFTP_anon.pcapng.txt)

kcpoole

I note that it is the traffic returning on Port 20 that fails.
As I am using outbound NAT, how does the firewall know that the returning traffic is part of the original outbound conversation on port 21?

I assume the ftp proxy on pfsense does that magic, how do i test whether that is working properly?

Ken

johnpoz

ok 114 is the client, and 182 is the server..

your 114 box tells 182 hey come talk to me on Port X with the port command.. You see the connections from 182 to the port, but never see an answer back. Yes if this on sniffed on wan of pfsense, and 114 is behind pfsense we can see the traffic get to wan of pfsense.

Can you see the traffic at the lan interface get sent to 114?

Do you see the states get opened? What if you try a passive connection vs active. Where the client will make the connection to the server, ie so 114 will make a connection to 182 on the port 182 says to connect too.

As to the IPs being changed - remember in my post where I said to set everything else in the task to passthru ;) "You add an anon task, set everything to passthrough except the text part. " ;)

Why would you be doing an outbound nat, other than automatic? Do you have some strange outbound net setup? I would assume all traffic from lan of pfsense going to wan network would get natted to the pfsense wan IP. Ie the 114 address.

Normally in an active connection the ftp helper would change the lan IP of the server to the public, and no that hey that server is going to be coming in from source IP 20 to port X in the port command, so will send that on to the lan IP of the ftp client.

You can run into problems if that port is already in use, sure. Try a passive connection, set on 114 when you talk to the 182 server. What does the sniff show then?

Just to clarify - your the client in this conversation right, your 114 is behind pfsense. and 182 is some public ftp server? Or is 182 behind pfsense?

kcpoole

@johnpoz:

ok 114 is the client, and 182 is the server..

your 114 box tells 182 hey come talk to me on Port X with the port command.. You see the connections from 182 to the port, but never see an answer back. Yes if this on sniffed on wan of pfsense, and 114 is behind pfsense we can see the traffic get to wan of pfsense.

Can you see the traffic at the lan interface get sent to 114?
Yes, See the new sniff on the LAN Side. I see the same traffic there too

Do you see the states get opened? What if you try a passive connection vs active. Where the client will make the connection to the server, ie so 114 will make a connection to 182 on the port 182 says to connect too.
this trace was PASv and failed as wwell.
yes the states are opened ( at least I think they are correct , See the attached shot.

As to the IPs being changed - remember in my post where I said to set everything else in the task to passthru ;) "You add an anon task, set everything to passthrough except the text part. " ;)
I am little tired these day or is it just dumb! :-)

Why would you be doing an outbound nat, other than automatic? Do you have some strange outbound net setup? I would assume all traffic from lan of pfsense going to wan network would get natted to the pfsense wan IP. Ie the 114 address.

Outbound NAT is automatic. no rules have been changed for many months and has only just stopped working :-(

You can run into problems if that port is already in use, sure. Try a passive connection, set on 114 when you talk to the 182 server. What does the sniff show then?
See attached Sniff from inside using PASV, This connection failed too

Just to clarify - your the client in this conversation right, your 114 is behind pfsense. and 182 is some public ftp server? Or is 182 behind pfsense?
yes that is correct, Teh remote site is not behind an firewall ( as far as I know, I d not have anythig to do with that host)

Screenshot-4.png_thumb
[Capture LAN- FailFTP_anon.pcapng.txt](/public/imported_attachments/1/Capture LAN- FailFTP_anon.pcapng.txt)