No response to outgoing packets through VPN
-
I've set up pfSense on AWS using the official AMI. The goal of my server is to operate an OpenVPN server to authenticate users and put them behind a common IP address for those scenarios where our team needs to connect to systems that require an explicit allowlisted IP. The VPN does not directly provide access to any internal systems: it merely proxies internet traffic.
I can connect to the VPN successfully, and I can reach the web GUI while connected. I can also reach the internet from the pfSense server proper. However, all internet traffic coming over the VPN gets to the WAN but receives no response. This can be seen via the packet logger, as demonstrated below.
Can anybody help point out why this is happening? Even just knowing if this seems like a bizarre issue to anyone else would be helpful, since I'm new to pfSense.
First for reference, the packet log for a ping of 8.8.8.8 done from the web GUI, captured on the WAN interface. Pings are successful, and you can see the requests and replies:
However, when pinging from the connected VPN client, the outgoing pings reach the WAN interface, but no replies arrive:
Here's the matching log on the VPN interface:
For reference here is other relevant config:
-
Here's a screenshot of the relevant state, which looks good, but still no response traffic getting back:
-
Not sure what the forum etiquette is around here, but bumping this to hopefully get some advice! I'm about to abandon pfSense otherwise, which is disappointing.
-
Well your behind a double nat, pfsense is sending on the icmp from its wan source 10.0.0.107 to your gateway right 10.0.0.1..
So psfense did what it was suppose to do.. Your going to have to look upstream on why you didn't get a response.
-
What I don't understand is why I do get a ping response sending from the WAN interface when initiated on the pfSense device itself, but I don't get a ping response sending from the WAN interface when routed from the VPN client (i.e., the difference in my first two screenshots)?
As far as the upstream network is concerned, shouldn't those be identical?
-
Yup they would be.. you can ping from a client on pfsense lan right..
I am not behind a double nat.. But I can ping stuff through the vpn.. But my nat gets switched to pfsense public IP.
That is my phone coming in via vpn, and pinging 8.8.8.8
I would suggest you sniff on ping that works, and one that doesn't work and see if you can see how they would be different.. What do you have upstream? Another router that you can view such info on as its state table, etc.
-
TLDR: Resolved. It was a packet loss issue caused by this.
This is a pfSense Amazon EC2 instance originally launched into a subnet & VPC all by itself. Upstream is the AWS internet gateway.
Since your last message I tried launching another temporary EC2 instance into the same subnet to see if I could reach it from the VPN client, and no luck. Same behavior: I could sniff the outgoing packets and see them getting through on pfSense, but with no response. So then I tried sniffing them from this new destination server, and I was surprised to see no traffic received...until all of a sudden there was one packet that did get through.
That changed the whole game, and I ran some more exhaustive tests and found traffic was getting through, but with a 98-99% packet loss. One more Google search and I found the documentation page about hardware checksum offloading.
Given that I am using the official pfSense AMI, it would seem like this should ideally be preconfigured to be disabled if it's problematic on AWS. :-/
Theoretically this might be resolved by using AWS enhanced networking, but unfortunately Amazon documents this only for Linux, not FreeBSD, even though allegedly there is FreeBSD support.
-
@sbuchanan Thank you so much. That was an exceptionally frustrating 2 days I just spent on this. It's almost more upsetting that it was just a single checkbox for an optional feature.
