Hello,

thanks for your reply and suggestion. Yes, I supposed to do an experiment with a less important Lan to try if it works; my setup is with a vip (because I have 2 firewalls) and NAT rules on WAN connection.

I supposed that, even without any rule on the opt, I can change the firewall default gateway and for example navigate to internet from a pc inside the network, because on the lan interface I have the default rule that you said and left the default gateway.

Sincerely my worry is for ipsec VPN where the endpoint is with the old ip and I don't know, even if I suppose that it is so, if the traffic is routed correctly.

I'll try.

okay, that is some info i can use for something.

use one IP address as WAN IP address for example 78.X.X.74/29 and GW 78.X.X.73
and assign other IPs as VIP on wan interface

https://docs.netgate.com/pfsense/en/latest/book/firewall/virtual-ip-addresses.html

https://docs.netgate.com/pfsense/en/latest/book/firewall/methods-of-using-additional-public-ip-addresses.html

Rip? Really? You have needs for a dynamic internal routing protocol?

Apart from that, smells like nat configuration and/or routing on the second box.
Post a network diagram with subnet ip's and your routing and nat settings

@Daniel1972 I am trying to implement the same configuration, not because I'm running out of ports but because I'd like to have separate machines handling the WAN and internal LANs as well as having an internal LAN for monitoring. I just set up a(nother) 192.168.x.x/24 LAN between the boxes and used RIP to share the routes. Unfortunately, I'm running into an issue with routing to the Internet:
https://forum.netgate.com/topic/153989/1-pfsense-router-works-2-pfsense-doesn-t

Yeah...I wasn't looking at a router at the time and I hadn't looked at this one in a month. Oops.

Although that did light a bulb for me. Loss Interval says "Time interval in milliseconds before packets are treated as lost. Default is 2000." Do "treated as" packets actually get marked in the percentage lost? With an average of 1300 perhaps a few are taking longer than 2000ms and are considered "lost" although they arrive in, say, 2100ms and thus the 0% loss shown? I think I'll try using 120s for the time interval to see if that "provides smoother results."

Overall the goal was just to not have the connection drop/failover now and again, with 0% loss shown. High latency isn't great but moving the traffic from cable to DSL isn't generally going to improve that if it's due to traffic.

@cobrahead No, you are good to go. Everything else is mainly for carp and high availability setups.

Check out the great hangout done by jimp: https://www.netgate.com/resources/videos/openvpn-as-a-wan-on-pfsense.html
I'm sure this will answer all your questions. :-)

-Rico

Hi all,

I made another test. I configured another lab with these appliances:

FastGate Router -> Netgate SG-1100 (PfSense) -> Netgear Switch -> Laptops

Everything is going OK. No offline WAN_GW.

So, I'm thinking that the problem is Cisco Router.

Any ideas?

@Rico thanks, that makes sense. What information do you need?

@Derelict
Hi, now everything so fine and load balancing over to WAN function as desired. Meaning I get the bandwidth as sum of both WAN. Now I have to questions:

If I check "sticky connections" I no longer have the sum of both bandwidth. Its rather randomly once WAN1 or WAN2. Eeven mixed for Upload/Downlaod meaning it may occur that for download it takes WAN1 and for upload WAN2 but never both. It this the expected behavior?

How can I load balance the two WAN get both bandwidth added but still redirect all the traffic over VPN-Server (like mullvad, one or more sever)?

Best regards
Santo

@gwaitsi hmmm. so i set the default gateway to automatic instead of the gateway pool and it seems to have solved the problem. The pf box now defaults over the wan, and the policies are correctly working. so i am happy.

There may be a problem with your setup(?), as it would be quite a problem, if the known (trusted) DNS servers did not respond to the ping and would the provider's CPE restrict you from using ping ???
(this is just an idea why you can stop pinging from a known DNS server, for example, make sure the gateway IP, DNS severs, WAN IP, etc. are in your HOME_NET list / IPS/IDS)

We have been using Cloudflare DNS servers (1.1.1.1 / 1.0.0.1) for many - many years for monitor IP purposes, we have never experienced the problem you outlined.
Many ISP gateways really do not respond to ping, so a known DNS server is a good solution.
Test the best DNS server for you, starting with:

https://www.grc.com/dns/benchmark.htm

Or use this and try to PING the selected DNS server from a desktop machine for a long time and analyze the values obtained:

https://emcosoftware.com/ping-monitor

I don't think the multiple - gateway monitor IP is the solution, it would only bring more measurement tasks and results to the system, this is irrelevant here.
PS:
We have had the experience that sometimes on a self-made (from internet) blocklist, 1.1.1.1 is added to the list of banned IPs, the list is periodicaly updated on the firewall and 1.1.1.1 no longer works.

What did your own ISP answer this question? (FRITZ!Box vs. PING issue)

I would examine the rules on your OpenVPN tab and make them explicit otherwise traffic can get matched and sent down a different interface than you're expecting.

Post the remote access config and the site-to-site config for site 2... both located in /var/etc/openvpn