@viragomann , thank you.

It was indeed a floating firewall rule that was causing the problem.

After disabling it, all is working as expected again.

@viragomann
Cool! Thanks for the suggestion.

@3Texans By any chance, did you get it to work? I moved from Ubi Edge router Lite to PfSense and Obi200 GV config is broken for me. Would really appreciate if you could throw some lights in case if you were able to fix it.

Lets hope it is for a good cause.

Update, I Was never able to get this working properly, but Now that the 2.7.0 update has been released, once I updated, everything is working as expected. not sure if it was some sort of Hyper-V Driver issue, or some other bug that was fixed in this release.... just glad I can utilize my secondary internet connection better now. thanks for all the help!

@SteveITS Thank you! It was not. One other thing I forgot was I had DNS over TLS and some off these settings weren't properly configured. (https://docs.netgate.com/pfsense/en/latest/recipes/dns-redirect.html) with this properly configured even my work PC which tries to leverage a cooperate DNS server is forced back to my resolver (which properly resolves to my LAN address inside the network). At some point I will try your option which is also a great solution.
Thanks for your reply!
-b

@DirectRaw If that packet capture was on the VTI, it means your routes on pfSense1 are correct.

What about pfSense2? Do you have a route to send traffic to destination 172.19.0.1 through the VTI?

@ivarh
The outbound NAT rules are applied to interfaces. So they have nothing to do with gateway groups at all.
If you want them to specify only once for multiple interfaces, you can create an interface group and apply the rules to this.

@GameHoundsDev said in allow access from internal device to another internal device:

I am trying to allow internal VM to communicate with another VM

You need both

Ethernet level 2 connection. This is most easily done by having them on the same LAN. Within Proxmox that is done most easily by having them on the same bridge (the virtual equivalent of a physical Ethernet switch)

IP routing (if you intend to use a WAN IP to access a local LAN device). Look at NAT reflection

@fejzulla-neziri said in advanced configuration:

also services dns resolver
Host Overrides added domains but nithing

This is the preferred method to go, presumed your local computers use the DNS Resolver to resolve host names.

So ensure that they do conventional DNS requests, not DoH.

Consider to redirect all DNS requests to the localhost on all internal interface and to block DoH with pfBlockerNG.

Also ensure that you firewall rules allow access to the web servers.

@keyser thanks again

Bluntly, no. Not without a much better documented use case for this patch, along with tests and some sort of indications that the author (or someone...) will maintain it. Right now it is abandoned, and doesn't even apply any more. This patch makes fairly deep changes to the NAT code, changes which I currently do not understand and do not have the motivation or energy to study. If it gets committed and breaks something I'm going to be the one who has to fix it, so ... no, not unless someone can present a compelling case that this actually improves anything, that it is correct and that if there are issues they will work on them.

From the freebsd forum,I guess the pfSense guys can make it ?

@Matt_Sharpe said in NAT over IPSEC to private network:

It is not PFsense on both sides. However considering the NAT required is happening on the target side which is a PFsense. I assume this is the best place to ask :)

But the other site doesn't accept the multiple phase 2, as it knows only one, I guess.

Again, check the logs to find out, what's wrong.

@JonathanLee Thank you.

@emc

This issue has been fixed. NAT is working. It was a firewall issue in the PBX. I've whitelisted the IPs on the PBX's firewall and it works. Thank you everyone for your help.

@uz890ed so you disabled the 80 redirect on pfsense?

Validate that pfsense is not listening on 80, simple sockstat

[23.05.1-RELEASE][admin@sg4860.local.lan]/var/unbound: sockstat -l | grep :80 root nginx 90402 9 tcp4 *:80 *:* root nginx 90402 10 tcp6 *:80 *:* root nginx 90166 9 tcp4 *:80 *:* root nginx 90166 10 tcp6 *:80 *:* root nginx 90115 9 tcp4 *:80 *:* root nginx 90115 10 tcp6 *:80 *:* [23.05.1-RELEASE][admin@sg4860.local.lan]/var/unbound:

I then turn off the redirection..
redirect.jpg

[23.05.1-RELEASE][admin@sg4860.local.lan]/var/unbound: sockstat -l | grep :80 [23.05.1-RELEASE][admin@sg4860.local.lan]/var/unbound:

Adding a network diagram, which I hope helps better describe the problem.

Dual WAN Issue-Page-2.drawio.png

@johnpoz thanks for the tip and i did the same test.
Window on top is WAN and on the bottom is LAN. I just captured 10 packets from each interface and seems it is pretty fast so the culprit is not the NAT.

a243489b-bc55-49e5-87b2-747bd73a304f-image.png

Found though two solutions but still not why it is happening.

Remove Accept-Encoding header from the http request - result is very fast.

Using a reverse proxy with https is still fast with and without the Accept-Encoding header