Issue Routing Between Subnets With Multi-WAN
-
I'm having a strange issue when trying to setup policy based routing for a multi-wan setup. Below is a quick overview of our network setup;
Interface Assignments:
WAN (igb0)
LAN (igb1)
VOIP (VLAN 50 on igb1)
Guest_WiFi (VLAN 70 on igb1)
ATT_WAN (igb2)The issue I'm having is when I create a firewall rule in the LAN rules to load balance between both WAN's I lose access to the VOIP subnet from the LAN subnet. I've checked the firewall rules and don't see anything to prevent this from working. So I feel like it's some sort of routing issue but I can't seem to figure out how to resolve it. I thought I might need to add a static route but the system will not let me do that. What's strange is it does not matter where I put the firewall rule on the LAN, the second I add a rule that specifies a gateway group instead of "using the system routing table" I can no longer access the VOIP subnet (after reloading state table). Has anyone seen this issue before and know how to resolve it?
-
@snewby attached is my LAN rule for the load balancing.
-
@snewby my VOIP interface currently has an any/any rule to allow all traffic but the gateway is still specified as the default. Maybe this needs to match the LAN rule create earlier? i.e select the same gateway group? The thing is though I don't really want the VOIP subnet load balancing at this point.
-
If your going to force traffic out a gateway, you need a rule ABOVE it to allow access to other local networks.
https://docs.netgate.com/pfsense/en/latest/multiwan/policy-route.html#bypassing-policy-routing
-
@johnpoz thanks for that kb, I will try this out. I was not aware that was necessary. Would I need to specify only subnets connected directly to this router or also subnets in branch offices connected by site to site VPN's?
-
You would want a rule that allows that you want to allow access to, that is not reachable via the gateway your forcing traffic out. Anything that is reachable via pfsense normal route table, be it that some downstream router, etc. That you want to allow.
I personally would put in a rule above the gateway that blocks stuff you don't want to allow, so that stuff isn't routed out the gateway - that is not going to be reachable down that gateway anyway..
Say for example you had some other local vlan 192.168.X, that your vlan 192.168.Y shouldn't be able to get to.. Put a block/reject for that above before you route that out some wan, since your wan gateway sure isn't going to be able to get to 192.168.X anyway.
-
@johnpoz just wanted to follow up to say I implemented your fix and it resolved my issue. I can now reach the local subnets I need to with the load balancing GW group enabled. Thanks so much, I really appreciate the quick response. And that is a good suggestion about also adding some block rules as well. That is likely something I'll do at a later time. My rules are pretty basic at the moment but functional. I think I may need to brush up on my networking knowledge so I don't understand why these rules are needed but I guess if you select a gateway group that using a different routing table than when you use the default gateway.
-
@snewby said in Issue Routing Between Subnets With Multi-WAN:
f you select a gateway group that using a different routing table than when you use the default gateway.
Pretty much - when you set a gateway like that, you take the normal routing table that pfsense would use to know where to send traffic X, and just shoves it down that gateway.. Be it can get to where it wants to go or not.
Rules are evaluated top down, first rule to trigger wins, no other rules are allowed. If you have a rule before you shove it out a gateway that allows said traffic - pfsense will then route that traffic per its normal routing table. if the traffic is also attached, it knows exactly where to send it. Or if say you had another gateway for a downstream router via typical routing then it would know to send it to that gateway.
Glad you got it sorted - here to help, even if just a general sort of networking question. Happy to help when I can, even if not some specific to pfsense.. Just ask it in the off topic section if has zero to do with pfsense.
