Ports, rules, NAT
-
The packet capture basic output display is pretty simple. You start a capture via Diagnostics - Packet Capture and you filter on the LAN IP address of your server. The only packets you should see during your test would be coming from your client trying your NAT. I would expect to see no packets, confirming that pfSense isn't forwarding them on to LAN. If you do see packets going out from LAN then the problem is something else.
However, before I do anything else I would reboot your ESXi host just to remove any doubt about hardware in a weird state or the pfSense VM acting up.
-
I really dont want to have to reboot the whole host, given that port 25 NAT is working and all other VMs are running as expected.
I do appreciate the help though, I wouldn't be able to do this on my own without a magic wand.
Here is a piece of the packet capture - hopefully this means something
16:03:10.026412 IP 192.168.1.35.443 > 95.216.36.80.32944: tcp 0
16:03:11.935289 IP 192.168.1.35.443 > 95.216.36.80.53295: tcp 0
16:03:12.081959 IP 192.168.1.35.443 > 95.216.36.80.60995: tcp 0
16:03:13.074664 IP 95.216.36.80.48521 > 192.168.1.35.443: tcp 0
16:03:13.074938 IP 192.168.1.35.443 > 95.216.36.80.48521: tcp 0
16:03:13.128209 IP 95.216.36.80.48521 > 192.168.1.35.443: tcp 0
16:03:15.484324 IP 192.168.1.35.443 > 95.216.36.80.37319: tcp 0
16:03:15.593657 IP 192.168.1.35.443 > 138.201.87.102.60223: tcp 0
16:03:15.940938 IP 192.168.1.35.443 > 138.201.87.102.45092: tcp 0 -
Addition - the IIS box is a reverse proxy, proxying Exchange, even if I update the NAT to go to Exchange directly it still does not work.
I am happy to negate the need for a NAT and setup Squid / HA as a reverse proxy, however at this point I would just like it working again, then I can tackle additional challenges later.
-
means your webserver is answering - my guess would be its sending back RST.
16:03:13.074664 IP 95.216.36.80.48521 > 192.168.1.35.443: tcp 0
16:03:13.074938 IP 192.168.1.35.443 > 95.216.36.80.48521: tcp 0You see that 95.x from source port 48521 source port got an answer from your web server 443 back to the 48521 port.
-
Thank you @johnpoz
That confirms what I thought, that it is working, however, even having redirected the traffic directly to exchange I get the same results.
I know the ports are open on the internal side, and I see the port as open externally, but content is never collected and returned.
I dont know how to fix this quirk, hence my post.
I have rebooted both the IIS proxy and exchange, this does not help. I have not rebooted the PfSense box, I was hoping to fix it without that, but I guess if we believe this would help I can bounce it. It's a VM and should only take moments - just an annoyance to do so.
-
Here is the initial handshake and reply from my test with a web server listening on tcp/80:
After the initial handshake, the payloads should be non-zero. As John said, the NAT itself is working so your problem must be elsewhere. Do you have any security thingies that might have been updated and are now getting in the way? Antivirus or antimalware apps, etc?
-
Best thing to do would be to open your sniff in say wireshark so you can actually see what is getting sent back, or up the verbosity of the sniff. so you can see if your seeing syn, and then syn,ack back or just RST.
-
indeed as i say you must have something on 192.168.1.35 or whatever is your web server ip blocking it/not accepting it for some reason
set this option and do a capture again -
I have nothing auto-updating, I have checked and rebooted most systems except PF at this point.
I do see 'some' logs in IIS about activesync, but nothing else gets back to the client or the web-browser in terms of webmail.
I predict it will be something silly and/or trivial, but it's not clearly obvious at this point.
I do really appreciate your help, but I dont understand wireshark or have it installed at this point.
Learning more about networking generically is on my list of to-do.
Just re-running that packet capture now...
-
Does this help anyone?
16:27:43.348163 IP (tos 0x28, ttl 67, id 58574, offset 0, flags [DF], proto TCP (6), length 40)
138.201.87.102.42429 > 192.168.1.35.443: Flags [S], cksum 0xae6a (correct), seq 3568136006, win 29200, length 0
16:27:43.348722 IP (tos 0x0, ttl 128, id 14187, offset 0, flags [DF], proto TCP (6), length 44)
192.168.1.35.443 > 138.201.87.102.42429: Flags [S.], cksum 0x7a39 (correct), seq 1036996916, ack 3568136007, win 65392, options [mss 1460], length 0
16:27:43.388858 IP (tos 0x0, ttl 54, id 43753, offset 0, flags [none], proto TCP (6), length 40)
138.201.87.102.42429 > 192.168.1.35.443: Flags [R], cksum 0xe077 (correct), seq 3568136007, win 16384, length 0
16:27:44.407074 IP (tos 0x0, ttl 128, id 14188, offset 0, flags [DF], proto TCP (6), length 44)
192.168.1.35.443 > 138.201.87.102.34989: Flags [S.], cksum 0x3ef9 (correct), seq 564559956, ack 585524840, win 65392, options [mss 1460], length 0
16:27:44.445433 IP (tos 0x0, ttl 55, id 51013, offset 0, flags [none], proto TCP (6), length 40)
138.201.87.102.34989 > 192.168.1.35.443: Flags [R], cksum 0xb82e (correct), seq 585524840, win 16384, length 0
16:27:45.025532 IP (tos 0x0, ttl 128, id 1751, offset 0, flags [DF], proto TCP (6), length 44)I will re-re-boot my guests (email ones)
-
@Rod-It said in Ports, rules, NAT:
138.201.87.102.34989 > 192.168.1.35.443: Flags [R],
Well that looks like the 138.x box sent a RST to the web server.
-
@Rod-It said in Ports, rules, NAT:
16:27:43.348163 IP (tos 0x28, ttl 67, id 58574, offset 0, flags [DF], proto TCP (6), length 40)
138.201.87.102.42429 > 192.168.1.35.443: Flags [S], cksum 0xae6a (correct), seq 3568136006, win 29200, length 0
16:27:43.348722 IP (tos 0x0, ttl 128, id 14187, offset 0, flags [DF], proto TCP (6), length 44)
192.168.1.35.443 > 138.201.87.102.42429: Flags [S.], cksum 0x7a39 (correct), seq 1036996916, ack 3568136007, win 65392, options [mss 1460], length 0
16:27:43.388858 IP (tos 0x0, ttl 54, id 43753, offset 0, flags [none], proto TCP (6), length 40)
138.201.87.102.42429 > 192.168.1.35.443: Flags [R], cksum 0xe077 (correct), seq 3568136007, win 16384, length 0S (SYN), F (FIN), P (PUSH), R (RST), U (URG), W (ECN CWR), E (ECN-Echo) or . (ACK)
Flags [R] -> reset
-
So looks to me the client sent syn S and then server sent back syn,ack S.
But then the client says no thanks with a R
-
Sorry, I am not a network person.
Are we saying this is backend server dropping/rejecting or that things look ok here?
Both IIS box and Exchange will reboot shortly for completeness.
-
Apologies, didn't see your latter reply
Ok so it's IIS still saying it's not happy - at least I have a focus area.
-
Im not sure who the 138.x box is, but he is the client as he sends the syn... The server at 192.x box answers with syn,ack - this is normal start of a conversation.. But then the 138.x box sends R (RST) which is basically a F Off sort of thing..
-
Strange that the client is sending the reset. There are very limited circumstances for that to happen, and I don't see any that I know of applying here.
-
Just to be clear, this happens if I send the NAT to the IIS relay OR the Exchange box directly.
This does NOT affect clients on the LAN or any other VLAN, only requests that come in through the PfSense box.
The IIS box has it's firewall disabled for a moment, but this should not be it as internal clients work.
Both IIS and Exchange have been rebooted.
While I want to rule out AV because internal clients are working, I am happy to remove/disable it for now.
As an FYI - this is my home lab, not a production system so this is not massively critical, but a PITA nonetheless.
Something else that may or may not be relevant (I dont think it is, but i'l include it anyway).
About 2 days ago (probably some 6-8 hours before this started) I moved my domains DNS from DynDNS to CloudFlare - I have tried with and without proxied DNS, neither matter, however because the port shows as open on my public IP directly, I dont believe this to be related, though I am more than happy to be proven wrong.
-
If you have time, I'd love to see one more test run with the Level of Detail set to high so I can see the packet timestamp values.
-
17:16:46.428834 IP 192.168.1.35.443 > 138.201.87.102.48424: tcp 0
17:16:46.469427 IP 138.201.87.102.48424 > 192.168.1.35.443: tcp 0
17:16:46.522427 IP 192.168.1.35.443 > 138.201.87.102.61203: tcp 0
17:16:47.487447 IP 192.168.1.35.443 > 138.201.87.102.61529: tcp 0
17:16:47.886778 IP 192.168.1.35.443 > 138.201.87.102.61068: tcp 0
17:16:48.538018 IP 95.216.36.80.35970 > 192.168.1.35.443: tcp 0
17:16:48.538399 IP 192.168.1.35.443 > 95.216.36.80.35970: tcp 0
17:16:48.642150 IP 82.132.247.36.52308 > 192.168.1.35.443: tcp 55
17:16:48.645073 IP 192.168.1.35.443 > 82.132.247.36.52308: tcp 109
17:16:48.649273 IP 82.132.247.36.52308 > 192.168.1.35.443: tcp 46Guys - thank you for the help, really, it means a lot to know there are people out there willing to spend their weekends trying to help others.
I do believe 'CloudFlare' are to blame, I removed all of the DNS proxy connections once more and it's working again.
Does the above negate you wanting a further run?
