pfSense not responding to any ports
-
Alright so this is a weird issue probably related to AWS but if anyone has any ideas off the top their head it would really help me out.
I've got a pfSense server sitting on a subnet at the IP 10.60.66.30.
I can reach it fine via curl from 10.60.66.9 (same subnet), getting a response back as shown:
I cannot get a response from a box 192.168.3.20 (different subnet):
HOWEVER, here's the kicker, the pfSense box is getting the http requests, just not responding to them:
It does respond to requests from the same subnet... why?
If I try to connect back from the pfSense instance to my other box, same thing, but opposite direction:
However, this doesn't happen from another box on the same subnet (non-pfSense 10.60.66.9)
I've checked the rules and the logs.. nothing apparent is popping out at me as to why it would be doing this. Why does this pfSense instance hate me?
-
Someone might be able to help you once you give them a network map to know what's where, and a screenshot of your firewall rules.
-
Here's the sketch of the network map:
Note: the two pfsense instances can talk to each other just fine. The two ubuntu boxes can also talk to each other just fine.
rules on 10.60.66.30 pfsense instance:
WAN:
Floating:
advanced setting: statetype sloppy state:
IPsec:
-
OK the network map was the first clue I had that IPSec was involved. I don't know much about IPSec, but have you already gone through the troubleshooting docs?
https://docs.netgate.com/pfsense/en/latest/book/ipsec/ipsec-troubleshooting.html
https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/ipsec-troubleshooting.html
-
Right.. so I thought about that.. but wouldn't I still see an outgoing packet from the firewall if it were an IPSec issue? From the troubleshooting there seems to be routing enabled both ways - pfSense can send packets to Ubuntu, Ubuntu can send packets to pfSense, but neither are responding to the packets for some reason?
-
@thund3rsh0ck said in pfSense not responding to any ports:
Right.. so I thought about that.. but wouldn't I still see an outgoing packet from the firewall if it were an IPSec issue? From the troubleshooting there seems to be routing enabled both ways - pfSense can send packets to Ubuntu, Ubuntu can send packets to pfSense, but neither are responding to the packets for some reason?
I am confused, how can both be sending and neither responding? Are you doing PCAPs to determine this? Can 10.60.66.9 ping 10.60.66.30? Can you log into the pfSense webUI?
-
@tim-mcmanus said in pfSense not responding to any ports:
@thund3rsh0ck said in pfSense not responding to any ports:
Right.. so I thought about that.. but wouldn't I still see an outgoing packet from the firewall if it were an IPSec issue? From the troubleshooting there seems to be routing enabled both ways - pfSense can send packets to Ubuntu, Ubuntu can send packets to pfSense, but neither are responding to the packets for some reason?
I am confused, how can both be sending and neither responding? Are you doing PCAPs to determine this? Can 10.60.66.9 ping 10.60.66.30? Can you log into the pfSense webUI?
Yes - from the initial post there are tcpdumps.
Well I know both can send, receive, but the question is why neither is responding.
Yes 10.60.66.9 can ping 10.60.66.30
yes I can login to the pfsense webui at 10.60.66.30 from 10.60.66.9, but not from 192.168.3.20 as shown from the curl attempts.
192.168.3.20 can't get a response from ping either, even though the pfsense instance is getting them:
-
Machines on the same subnet will always be able to talk to each other because they aren't routed anywhere -- they're all local.
-
What network are you using for your IPSec tunnel? Has this ever worked before or did you just install it now?
-
Yes the tunnel functions as expected, mostly to NAT internet traffic out and from the corporate firewall. I'm not even sure all the traffic we're talking about here is being routed through those tunnels, probably via the VPC to VPC peering connection instead (which is what the ipsec tunnel is using as well).
-
@kom said in pfSense not responding to any ports:
Machines on the same subnet will always be able to talk to each other because they aren't routed anywhere -- they're all local.
I just had to ask because I didn't completely understand where we were with troubleshooting. Had to start with the basics.
My guess is that one or both of the tunnels does not have routing configured properly. I don't have IPsec configured on my router anymore, but I believe you need to set the routing up there as well as creating firewall rules.
-
I don't have the time to dig deep into this and I'm not really an IPSec guy, but my first random guess would be asynchronous routing.
-
@kom said in pfSense not responding to any ports:
I don't have the time to dig deep into this and I'm not really an IPSec guy, but my first random guess would be asynchronous routing.
yeah I've had some trouble with packets going back and forth via different routes due to the complex routing config here... which is why I had to mess with some of the sloppy state firewall rules. However, those were all caught by the firewall and logged. The puzzler here is nothing is showing in the firewall logs this time.. so I don't even know where to start to try and fix it.
The part I can't figure out is why there is no response caught by tcpdump. Even if the packet is lost in routing, shouldn't there still be an outbound packet? Also.. one-way connections work both ways which is also odd. Argh.. what a headscratcher.