Thanks! Exactly what I wanted. :D (Now I'll have to duplicate some firewall rules to outgoing NAT module)

That is entirely up to the client side in these cases. If you can configure the client to only send traffic for your far-side VPN subnets across, then it should be closer to what you want. But with IKEv2 in this sort of setup the client makes all those decisions. If you only have your server network(s) in the P2 list and have checked "Provide a list of networks…" on the mobile clients tab, clients might respect that and stop sending all traffic over IKEv2, but support varies by client.

Depends on what the data center is giving you. If it is two different interfaces then no problem. If it is two different gateways on the same interface I think you can still do Multi-WAN but it'd be an uncommon scenario. And it would be a single point of failure on your side. I don't see any reason you can't put multiple gateways on one interface, make a gateway group, and policy route to it though I have never seen it done (which doesn't mean it hasn't). Only one of them can be the default gateway.

Thank you for your answer. It is what currently happening … reaching the GUI from the inside on any of the Virtual IP's that are meant to be NAT to some LAN server (common way of NAT'ing) The only things I set up are : NAT on those 3 Virtual IPs some rules to have it accessed from the outside To get it maybe much more clear, here is some weird behavior linked : Situation : Ping from my laptop connected onto the VPN (L2TP/IPSec) with some 192.168.x.x address to a public IP that can be reached perfectly from the outside 91.134.183.74 (em0) which is NAT'ed to 10.0.10.4 in the LAN (em1). Here is the output : taken on the if l2tp0 : tcpdump -n -i l2tp0 host 192.168.x.x and port not 22 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on l2tp0, link-type NULL (BSD loopback), capture size 65535 bytes 10:10:46.586013 IP 192.168.x.x > 91.134.183.74: ICMP echo request, id 41134, seq 77, length 64 10:10:47.590574 IP 192.168.x.x > 91.134.183.74: ICMP echo request, id 41134, seq 78, length 64 10:10:48.594678 IP 192.168.x.x > 91.134.183.74: ICMP echo request, id 41134, seq 79, length 64 taken on the LAN (to see if it reaches the targeted server) : tcpdump -n -i em1 host 192.168.x.x and port not 22 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on em1, link-type EN10MB (Ethernet), capture size 65535 bytes (…. NOTHING here) taken on the WAN interface : tcpdump -n -i em0 host 192.168.x.x and port not 22 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on em0, link-type EN10MB (Ethernet), capture size 65535 bytes 10:09:58.470654 IP 91.134.183.74 > 192.168.x.x: ICMP echo reply, id 41134, seq 29, length 64 10:09:59.472724 IP 91.134.183.74 > 192.168.x.x: ICMP echo reply, id 41134, seq 30, length 64 10:10:00.475666 IP 91.134.183.74 > 192.168.x.x: ICMP echo reply, id 41134, seq 31, length 64 10:10:01.476897 IP 91.134.183.74 > 192.168.x.x: ICMP echo reply, id 41134, seq 32, length 64 10:10:02.474787 IP 91.134.183.74 > 192.168.x.x: ICMP echo reply, id 41134, seq 33, length 64 10:10:03.476701 IP 91.134.183.74 > 192.168.x.x: ICMP echo reply, id 41134, seq 34, length 64 10:10:04.480793 IP 91.134.183.74 > 192.168.x.x: ICMP echo reply, id 41134, seq 35, length 64 Hereby the routing table of the FW : Internet: Destination        Gateway            Flags      Netif Expire default            91.134.183.254    UGS        em0 10.0.10.0/24      link#2            U          em1 10.0.10.1          link#2            UHS        lo0 91.134.183.0/24    link#1            U          em0 91.134.183.72      link#1            UHS        lo0 91.134.183.73/32  link#1            U          em0 91.134.183.74/32  link#1            U          em0 91.134.183.75/32  link#1            U          em0 91.134.210.0/24    link#1            U          em0 91.134.210.4      link#1            UHS        lo0 127.0.0.1          link#6            UH          lo0 192.168.x.x        link#7            UHS        lo0 192.168.x.x      link#7            UH        l2tp0 Seems like there is at least an issue here as the FW seems to send that traffic to the internet … which could be acceptable if at least they came back ... .. or, better, to have the NAT rules applied on those addresses. I did not add those routes at any point and cannot have them deleted neither. Here the routing entries on the laptop : 91.134.183.72      10.0.0.1          UGHS            5    13449    en0 91.134.183.74      link#10            UHWIi          1    1199    ppp0 10.0.0.1 is the default GW of my CheckPoint here at home which is correct and works fine. But for the other WAN IP, it sends it to the tunnel. Same story, I did not add those records. The VPN connection propagated it here for me. So the first issue is pretty clear : the FW is sending its traffic onto the wrong interface (question #1 : how to get rid of that strange routing entries) But the second one is at least as weird : the FW seems to answer to a request to 91.134.183.74 instead of transferring it to the according server on the LAN. He definitively things he is the guy that own that IP (which is technically correct , except the NAT part) I already re-installed the entire pfSense twice from scratch and always the same issues … depression is watching me ... Thanks /O. [edit] Hmmmmm …. sounds like Reflection had not been enabled on that 1:1 NAT entry ... So from the LAN, it is working now. But still not from the VPN. It is allowed by the rules but a ping to some public VIP entering the l2tp0 if ig getting out to the WAN ... :-/

You have to assign the OpenVPN interface under Interfaces>assign in order for it to have gateways available.

The gateways' monitor IPs are basically the equivalent of IOS's tracking. If your backup path is going to be a VPN, you can use OpenVPN for that, which will give you an additional gateway. @ipodgorny: Thanks, that clears it up. I thought that Non-Local was both layer 2 and 3, now it all makes sense. That's just the nature of IP routing, you can't tell a system to use a gateway that isn't on the same layer 2 network (or otherwise directly-connected, like a VPN).

Did you ever find a solution to your problem?

start by testing with a clean install (=no packages whatsoever) if the problem is the same with clean install, check system logs for error, check interface configuration, check wiring, check cpu usage if the problem is fixed, add packages, one at a time & try again until you find the culprit.

That never would have worked. Interface group rules don't have reply-to because they can't by their nature. One rule applies to multiple interfaces. So your non-default WAN wouldn't have gotten replies out, unless it was sourced from the IP facing you on on the PPPoE concentrator, where it'd know how to reply. It might have changed to a diff source IP on the ISP side that needed reply-to to route back.

Cool thanks.  That's what I thought.  How about the "Outgoing Network Interfaces" setting for Unbound … what's the current best practice on that?  I notice it defaults to "All" but I usually change it to "LAN + Localhost" otherwise DNS queries forwarded over IPSEC tunnels do not function.  Seems to work well enough but I don't know if that's something I should be doing differently?

Go to the routing gateway page (System / Routing / Gateways) and select the edit action (looks like a pencil) for the gateway you want to configure. Scroll down and click on the "Display Advanced" button and look for "Latency thresholds"

In general you want to start a new thread rather than using old threads. Old threads don't receive as much attention as new threads. Give that you are using a satellite connection, you need to adjust the latency thresholds to higher values. Check the latency under load using ping, and then adjust the threshold values for the connection on the gateway edit page (System/Routing/Gateways/Edit) in the advanced section.

Pretty sure same as here https://forum.pfsense.org/index.php?topic=111108.0

I have tried pinging from the console and it was fine. DHCP was fine. Everything is up on the server. Internet on my LAN is not working., it can't connect to the internet.

That's "Alert interval".

Thanks for your answers @heper: your openvpn is a transit-network …. packets go THROUGH it instead of TO. Yes I understand this. It was kind of a "shortcut" : it was shorter to talk about "OpenVPN" rather than about "the machine connected through OpenVPN" @johnpoz: So looks to me you have this - see attached. Your drawing is really better than mine (except I do not see Internet as such a dark cloud)  ;)  Yes it is my network config. The reason I have such a config is because pfSense1 and server1 are virtual machines hosted on host1, while pfSense2 and server2 are virtual machines hosted on host2. Host2 acts as a backup of host1, and I wanted the settings of server2 (and all the other servers, configured that way), to be ready and operationnal. @johnpoz: I would not suggest trying to create a route on pfsense 2 point to the tunnel network 10.0.100/24 to pfsense 1 lan IP So is this the reason why the static route I set on pfSense2 (as described before - adding a "green arrow" on your drawing from pfSense2 to pfSense1) does not work ? Is there a (short) explanation why a "simple" static route will not do the trick ? I was expecting that if there is a "sign" in pfSense2 saying "to go to OpenVPN : follow the direction to pfSense1", and when you're in pfSense1, ask someone…