If it makes multiple outbound connections and the protocol doesn't like it coming from two different addresses you will have problems.

If it only makes one connection it should be fine.

Try it and see?

If it gives you issues you can policy route just that traffic out one WAN. You might also try sticky connections.

https://docs.netgate.com/pfsense/en/latest/book/multiwan/load-balancing-and-failover.html#problems-with-load-balancing

You have to use the tag because, as that blog describes, traffic heading out WAN has already had outbound NAT applied by the time the outbound floating rule is checked so you lose the ability to match on the hosts' inside IP addresses.

@Milan-M said in LAN Routes just disappear:

LANs 3-5 have been created by going to "Firewall -> Rules -> LAN" and creating the rules there.

That is not how you create anything.. Creating other lan would be done via interface assignments, be it a physical interface or a vlan you assign.

If you have other networks that are downstream that you want to get to via some other downstream router, then you wuld need to create a gateway in routing, and then the route(s) telling the networks at are available via that gateway.

Yes you would need to create rules to allow them access.. But that is not what "creates" them or routes to them.

Btw your rule there for "lan" isn't going to do anything - the source is set for the lan address, not the network.. So that says hey pfsense if you see traffic from your own lan address allow it ) Never going to work that way..

@dukynuky said in Problem changing gateway through rules:

pass in log quick on $OpenVPN reply-to ( ovpnc1 10.10.10.1 ) inet from any to any tracker 1560717223 keep state label "USER_RULE"
now it is working.. seems pfsense isnt creating the reply to rules.. :(

That will not survive filter rule rewrites.

The traffic coming into the lower pfsense MUST NOT MATCH rules on the OpenVPN tab or the state will not get reply-to.

The traffic must match the rules on the Assigned interface tab. If it matches a rule on the OpenVPN tab, those are processed first so the assigned interface rules will never be reached, and therefore no reply-to.

All of this works. It just has to be configured correctly.

I would just remove all rules from the OpenVPN tab and put the necessary rules on the appropriate assigned interface tab and never worry about it again. Some topologies support this method, some don't.

@Derelict said in Multiple public IP multiple routers...:

If all of this is jibber-jabber to you

My money is on this statement ;)

I still got this issue, now I can replicate it easily at 2 completely sites, all 2.4.4_p3 and both using;

FRR and OSPF

list itemHA pair

list itemIPSEC VTI tunnels bound to a CARP IP address

list itemFRR set to fllow the lan CARP address (so FRR off on the backup firewall)

Here's a continuous ping across the VPN from site A to site B.

Reply from 10.10.40.1: bytes=32 time=4ms TTL=253
Reply from 10.10.40.1: bytes=32 time=7ms TTL=253
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Reply from 10.10.40.1: bytes=32 time=4ms TTL=253
Reply from 10.10.40.1: bytes=32 time=3ms TTL=253
Reply from 10.10.40.1: bytes=32 time=4ms TTL=253
Reply from 10.10.40.1: bytes=32 time=3ms TTL=253

First timeed out, that's the primary firewall being rebooted, 4 pings lost and the backup completely takes over. Very acceptable. Excellent.

Now the slow bit... The primary comes up, CARP takes over and takes ages for things to settle and go online.

Reply from 10.10.40.1: bytes=32 time=3ms TTL=253
Reply from 10.10.40.1: bytes=32 time=17ms TTL=253
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Reply from 10.10.40.1: bytes=32 time=3ms TTL=253
Reply from 10.10.40.1: bytes=32 time=4ms TTL=253

After digging, I think the cause is the VPN, IPSEC, it's just not getting released from the backup firewall in a timely manner, it seems to hold on and on and on and keeps running IPSEC VPN tunnels. I can speed up the fail back by logging onto the backup firewall and in IPSEC status stopping the IPSEC tunnels.

I wonder if the issue is because my IPSEC tunnels are using a CARP IP address?

Ask the provider if they have an alternate subnet they can assign to the other subscription.

Many thanks for your help, it works fine.
You help me a lots.

@techanalyst NM solved

@Nick-Sharp said in Multiwan failover between two sites via P2P Leased line.:

Static routes
192.168.2.0/24 GW_OPT1 – 10.10.100.2 Interface WSP2PHH

This should read...
192.168.1.0/24 GW_OPT1 - 10.10.100.1 Interface HHP2PWS

Thanks for your response. You know, sometimes you need to be told something three times before it sinks in. Every time I've seen this recommendation, I've read the settings as "Pull Routes" not as "Don't Pull Routes". I thought having the box unchecked was accomplishing this. After more careful examination I see that I had it backward. I checked this box and voila! It's now working as expected. Thank you!

Help me understand the DNS leak concern and how to avoid it?

Well then you changing the cache default time makes no sense how it could fix anything..

Have your isp explain what that setting "fixes" If the mac doesn't change then your cache could be for 10 years ;)

Seems like your isp wants to see arps more often than every 20 minutes for whatever reason?

@guido_neumann said in Access to Web Gui over ISP WAN Gateway - Rules,NAT?:

Destination WAN Orbis1 and now i can ping and HTTPS.

Destination would be "WAN_ORBIS1 Addr" or "This Firewall". Source should be any because of - you get it - the internet. Or even better, if you access that from a static IP (company etc.) then only allow this or another trusted IP. Much better than just allowing all.

Thank you, Chris. I was able to download and install the image. All is good!

@viragomann Thanks, I may just try that.