High Availability with Multi-WAN and Multi-LAN

bitrot

Hi,

I currently have a setup similar to the example configuration listed here
https://docs.netgate.com/pfsense/en/latest/book/highavailability/multi-wan-with-ha.html

The gateway group is setup for failover so that if WAN1 goes down outbound traffic goes over WAN2 instead. I don't have a DMZ setup as per the example. All of this is setup and working properly.

However, instead of the DMZ described in the example, I would like to have a second LAN interface except I would like devices on this LAN2 to use WAN2 by default and failover to WAN1.

However, the instructions specifically state:

//
With Multi-WAN a firewall rule must be in place to pass traffic to local networks using the default gateway. Otherwise, when traffic attempts to reach the CARP address or from LAN to DMZ it will instead go out a WAN connection.

A rule must be added at the top of the firewall rules for all internal interfaces which will direct traffic for all local networks to the default gateway. The important part is the gateway needs to be default for this rule and not one of the failover or load balance gateway groups. The destination for this rule would be the local LAN network, or an alias containing any locally reachable networks.
//

How would I accomplish this since I'm explicitly told not to use the failover gateway group?

jimp

You can use a gateway group for other traffic, you just need the rule without the gateway group above that, so that traffic between the LANs can reach where it needs to go. Assuming you want the LANs to be able to reach each other.

bitrot

Thanks for the clarification. I actually don't need or even want the two LAN segments to be able to reach each other. So in that case, does this mean then that in each LAN segment the firewall rule can use the respective gateway group in the firewall rule?

jimp

In that case you'll want rules at the top of each tab to block traffic from reaching the other LAN (also without a gateway because block rules don't need gateways).

Don't rely on the gateway being there as the only thing keeping the LAN(s) isolated, since if the gateway/group is down, it will be omitted from the rule by default, so then traffic could flow between them. There is an option to change that behavior, but it's still a bad security practice.

bitrot

Great. That makes sense and agree with the reasoning.
Just to make sure I understand correctly then:
In the Firewall rules, I will create two rules in each LAN segment. The top rule explicitly blocking all traffic from reaching the other LAN segment (without a gateway). Below that the allow rule using the respective gateway group. Is that right?

jimp

At a minimum, yes. You can get much fancier/more fine-grained than that if you like.

reberhar

So if there is only one LAN connection and no DMZ, there would be no need for any such rules, right?

There would be no cross Lan traffic, or DMZ traffic to direct to or block. The example is just included incase such conditions exist.

Although I have done several HA's, the one I am doing now is my first multi-wan attempt.

Thanks

reberhar

This post is deleted!

senseivita

@reberhar When you add the third interface to pfSense (WAN, LAN, OPT1-third) it changes its behavior from ultra-permissive to default block, no rules on an interfaces mean no traffic.

Without a gateway set on the rules, it uses itself as the gateway (connected routes), then the default gateway (or gateway groups). If you set gateways on all rules other than the default, you bypass pfSense altogether — if traffic matches the rule. If you use the firewalls' services, e.g; DNS, a rule without gateway should come before the policy routing rules.

reberhar

This post is deleted!

reberhar

@senseivita

Of course there are other rules that were in place when pfSense was not HA. There are the rollover rules and the rule in the LAN set that directs traffic to them.

I made a copy of the original system and brought up two pfSense instances, doctoring them to the place of HA.

... However, if you could give me an example of such a rule that would be helpful. I am using 4 wans, there were 5 but there was a problem with one of the wans having to pass through the switch connected to the lan and then to the wan port. With multicasting that was a disaster.

This concept with the default wan is a little unclear. If I send my traffic to the default wan, how does it get the the gateway groups?

Thanks for your patience.

I am trying to figure out how to past a graphic ...

Floating
WAN1
LAN
WAN2
WAN3
SYNC
WAN5
OpenVPN
Rules (Drag to Change Order)
States Protocol Source Port Destination Port Gateway Queue Schedule Description Actions
19 /904 KiB

- - LAN Address 4443
    80
    2244 * * Anti-Lockout Rule
    0 /0 B
    IPv4 TCP * * 10.2.0.48 4443 * none LAN 48 admin
    48 /55 KiB
    IPv4 UDP * * LAN net 53 (DNS) * none Allow DNS to pfsense
    0 /1 KiB
    IPv4 UDP LAN net * * 53 (DNS) * none Block and DNS requist outside of pfsense
    0 /0 B
    IPv4 * 10.2.1.178 * * * * none Block unknown client for abuse
    0 /0 B
    IPv4 * 10.2.0.171 * * * * none Block unknown client for abuse
    0 /4 KiB
    IPv4 * 10.2.0.206 * * * * none Block unknown client for abuse
    0 /0 B
    IPv4 * 10.2.1.74 * * * * none Block unknown client for abuse
    299 /16.70 MiB
    IPv4 * LAN net * * * Infinitumn none Default allow LAN to any rule
    0 /0 B
    IPv6 * LAN net * * * * none Default allow LAN IPv6 to any rule

Infinitumn is the gateway group

reberhar

Ok - I think I've got it or am getting it. I knew about the third interface being blocked by default, but I had not much thought about actually deflecting the flow before the end of the rule set and what that really meant. I will be inspecting my installs and relearning some firewall rule functions I thought I understood. It is quite true that as soon as you have more than one WAN things get more complicated.

Thanks

reberhar

@reberhar Yes of course. Why would you want to choose a gateway for every rule.

I was just caught in the verbage.