Good question. Yes, it's not optimal. Very very little traffic though. I do have a host override, so anything that goes through pfsense for DNS will get the local IP. I haven't figured out how to do this on the VPN. It's basically a wifi VPN, so the hosts are things like iphones and androids. Since they don't go to pfsense for dns, they don't get the host override. And, I haven't figured out how to override it locally on the devices themselves.

Yeah, probably not. You can do things like have multiple gateway groups and policy route different traffic across different groups but I do not see the algorithm changing. As far as I know what would have to be done upstream in pf anyway.

Your problem would be related to the rule you have on your lan that forces all traffic out this MGW_MAIN gateway. How is suppose to get to your 10.41.0.0/30 on igb0 when the allow rule that allows traffic will force all its traffic out this MGW_MAIN gateway. And your rule on your timenet_modem /30 network has your allowed source as your lan.. Your problem is not so much routing, but your firewall rules. Rules are evaluated top down as traffic enters an interface.. First rule to trigger wins, no other rules are evaluated.  So traffic trying to go to timenet from lan would be sent down the mgw_main gateway.  Which not sure what that is from the info you provided.  You need to put a rule above this rule that forces it out the gateway with no gateway set so pfsense can just use its routing table to get to the other network attached to pfsense. Your traffic coming from your timenet network trying to go to lan woudln't go anywhere, because the source of the traffic would be from the timenet /30 not lan - your rule on timenet inteface says the source has to be lan net - which never going to be the case. BTW - pfsense is not going to nat traffic between networks directly attached to it, unless you set it up to do so.  It would only create nats for traffic going out what it considers a wan, ie has a gateway set on the interface.

@Gentle: What 2.4 Ghz channel did you set your SonosNet to be? Give it a dedicated channel, for example channel 1. Then for your Unifi APs, use channel 6 or 11 (NO other) in HT20 width. I've converted all my Sonos devices over LAN.  I use Power Over Ethernet for each Sonos device. The Sonos Android app goes through my unify AP into my LAN to control the sonos controller. @Gentle: Only use band steering if you have tested all your clients are ok with it being enabled, it can cause some devices to never connect. Using a single SSID is easy and simplifies things, use for mobile devices. Then add a 5 Ghz only SSID. You can up to four SSIDs per WLAN group, and perhaps 8 in a future update. I just started reading & learning about band steering. Need to understand how it works…. I am more old school... I like to select a SSID and be 100% sure it will never use 2.4 GHZ frequency.  ( I have frequency analyser in my house....my next step would be reduce power to the minimum level) I still don't know how to create a dedicated 2.4GHZ and dedicated 5GHZ.  Do I go Setting - Wireless - 2G Data Rate Control keep and put each speed drop down to disabled and only keep 5G Data Rate Control? I don't understand yet all those drop down !  :-[

172.16.33.0**/16** ?? You need IPsec Phase 2 entries for both LANS to the OpenVPN tunnel network. And the OpenVPN clients need to know to pass traffic for those remote LANs to the OpenVPN tunnel. If you are using redirect gateway that should already be happening. If not you need to push those routes to the clients. As always, firewall rules have to pass the traffic as it enters pfSense. Too many inconsistencies in your description to be more specific.

@johnpoz: Oh you caught me never nothing bigger than a /24.. <rolleyes>Been managing global networks for years.. Actually mange a /16 with Arin - but sure just small networks for me ;)  I have personally migrated large plant networks with 1000's of nodes to new networks.. Not once have needed to run multiple layer 3 on the same layer 2.. Now if you had lots of static devices on a network that had to each be touched - ok might take you more than a few minutes.. But would be your own fault or the guy before you not thinking ahead to have large amounts of devices with statics in the first place. While it is "possible" suggesting it to new user even in a LAB is just Borked plain and simple..  There is ZERO reason to ever do this other then the time needed to migrate, which is planned correctly should be very short amount of time..  Minutes if done correctly!! Not in the smallest of labs or the smallest of home networks just starting out is this ever a good idea!  Production, not production, lab - whatever this is just plain Borked.. Period! BTW - are you having fun smiting me every time you login?  Seems odd my count just went up again, minutes if not exactly when you logged into the forum..</rolleyes> You are being a bit forceful and nasty in how you reply. Whatever. Yawn. Agree to disagree.

its not a modem if your pfsense wan is not public - its a gateway (modem/router combo) doing nat.  So you either need to put it in bridge mode so pfsense gets your public IP.  Or you would have to forward any ports you want to forward on pfsense - this would be double nat.  Your cox isp device might also support dmz host, where it forwards all ports to an inside IP.. ie your pfsense wan IP.

Hmm even disabling ipv6 on WAN on pfsense makes me lose access to it. I regain access after a reboot. There is no special configuration I am doing on pfsense's side and ipv6 worked fine before adding the openwrt router. It seems pfsense becomes quite buggy behind another router for ipv6. I don't need v6, but it would be nice. I clearly don't have much knowledge on the subject, but it seems weird to me that ipv6 is more complicated in these scenarios compared to double nat, when v6 was supposed to make this simpler.

That is on 10/100 and no does not do vlans..

@Fabio72: Every openvpn client is using the failover group as outbound interface. Every openvpn client has the pulled routes disabled. I configured outbound NAT rules for every VPN and disabled the IPV6 gateways for the VPN interfaces. This is most critical part of your configuration.

@Derelict: Bridging is incompatible with CARP/HA interfaces. Good to know, then switches are the only option anyway. Thanks for your help

a router (pfsense) is not a switch. switches are much cheaper ($80) & perform much better realtek nics are known for their bad behaviour 2Gbit/s wirespeed requires serverclass hardware, not low power (at this point in time)

You're still using GroupWise? That's your problem there… I'd guess you are using a load balanced gateway. Try putting a rule for GroupWise and ssl traffic on a failover gateway. (Before the balancer) Lots of secure sites don't like the IP changing.

Solved, it was indeed just a simple tweak. I unchecked the System -> Routing -> Miscellaneous -> Failover default gateway option I had checked, and updated any outgoing firewall rule to explicitly use the advanced settings. Under advanced settings I changed Gateway from default to the name of my Gateway group. Now, outgoing firewall rules have a black sprocket icon next to them to indicate advanced settings which in my case is the Gateway group. Internal traffic rules, any LAN -> LAN for example, do not need this updated setting. Only traffic using a Gateway should be updated to use a Gateway group. To answer my initial questions explicitly: yes, this has to apply to each firewall rule that allows WAN traffic. If you have an outgoing firewall rule with the Advanced Settings -> Gateway left on 'default' it will NOT allow traffic to leave via the secondary WAN. I had to update my SSH rule and HTTPS rules independently for both to work. This is all clear in retrospect, but this might help someone else in the future to learn from my confusion. Once again, thanks heper!