@kashs said in Added second WAN but no traffic:

Correct. I had to remove the bridge mode and set it to DHCP in the TOM box.

Ok got it..

WAN2_5G is setup as static IP 172.16.1.2. No way to avoid double NAT but so far no issues.

For simplicity, and for further testing, I'd keep pfsense as DHCP. It really doesn't matter what IP it gets from the TMO box, and you have already created a static entry in the box based on pfsense MAC.

Here is what the traceroute shows:
7dda35d3-ecc7-4425-95f1-6c69b2e1f76a-image.png

None of these are my static IP or the ISP Gateway IP.

When you log into the TMO box, you should be able to see the settings there, for "internet". So you would see what IP and Gateway it has received from TMO. Also, entry no 5 seems to start with 72.xx which is the same as the static IP you have been given by TMO?

The static IP is correctly assigned to the WAN2_5G interface, but the WAN2_5G_GW does not get an IP. If I set it manually to the ISP GW IP, no traffice and Offline status. When I tried the static IP in the GW, it shows it as Online, but no traffic.

What Inseego router is it that you have? I did some googling and found someone having similar problems on an FX2000and all that was required would be the following.

Unplug everything on LAN side of Inseego and reboot it Set pfsense WAN2_5G back to dhcp Connect to the Inseego

https://www.reddit.com/r/tmobileisp/comments/11x7mgy/how_fx2000_in_bridge_mode_with_5g_business/

@viragomann said in Multi ISP without failover:

pfSense routes incoming traffic just to the destination IP. If the packet is destined to a LAN2 IP it will be routed to it, no matter if both LANs are defined on the same NIC or on different ones, and no matter, on which WAN NIC the packet as entered.

Ah, now I understand. Thanks :)

@viragomann
It’s a Cisco Meraki the router Site A!
But, i’m thinking now:
The traffic should be routed to 192.168.100.222, not for the gateway 192.168.100.1 (this is the router with the VPN tunnel).
In the 100.1 router have static routes for route the traffic specified throught the 100.222
Is it the same solution (change phase 2 to 0.0.0.0/24)???
Thanks again

Another thing worth mentioning is that I've tested with an old Cisco RV320 router this same setup and it worked without any issues. The only things I did on that RV320 were configure the WAN with the same parameters as the pfSense, a static route and a resolver for the FQDN of the PBX server. Hope someone can give me a hint.

I found the solution:

ProtonVPN allows alternate gateways following the format: 10.x.0.2/32
I have tested 2 through 9 (10.2.0.2/32, 10.3.0.2/32, ... 10.9.0.2/32) and they work.

Similar problem in CE2.7.2 in AGO 2024

@wojciech__
https://docs.netgate.com/pfsense/en/latest/multiwan/index.html

@frog yeah that config ;) What do you have connected to all your ports? I can not think of a sane reason to have a setup like that..

pfsense can not even ping its own interface, on pfsense? Do you have it enabled?

So you have ports 2-8 all connected to the same usw-pro? Sounds like a loop to me..

You have 2-8 all in a lagg/port channel/etherchannel/lacp - whatever unifi calls it on their end.. Did you set that up on pfsense? Why would you have all 3 of your vlans tagged on every port unless they were connected to different switches or APs, or different vm hosts?

With the info provided that setup looks wrong to me.

@kunundrum0 put your block in floating on wan outward direction.. Your prob routing through a gateway failover group. This if traffic is leaving wan 1 it would be allowed, but leaving wan 2 it would not.

@viragomann

Sorry I never replied back with what I found out. It turns out that you can do this but it requires a setting in the OpenVPN client and Outbound NAT settings. I only figured this out through many iterations of trying every possible reasonable setting.

Create a firewall Host alias that contains the domains you want to force through a specific gateway. Enter the domains/URL's there. When you save it will fetch the URL's, so do this first or your list might not be updated with resolved IP's.

Add your firewall "host" alias (that contains the list of fqdn's you wish to force through the specific VPN Gateway) to the "IPv4 Remote Networks" field in the OpenVPN client settings. This is what updates the routing table. Note that the remote networks field does not auto-complete the alias when you type it (unlike most fields), but it'll fail if it's the wrong type or non-existent.

For the outbound NAT settings: set the interface to the VPN you want to force the domains through, IPV4, any protocol, source any, destination -> network or alias - use an alias created in pfblockerng OR the one you use above, [gateway] translation [your vpn client name] Address, and place it at the very top of the Outbound NAT rules

For pfblockerng alias, it can be used for the outbound NAT config but cannot be used for the OpenVPN client config (you can probably ignore pfblockerng and just use the built in URL alias option). In the pfblockerng settings create IPV4 alias and set the domains for Whois with state On and enter the domains. Under settings use Alias Native and pick your update settings (For local alias I'm pretty sure you can just leave it disabled, but it doesn't really matter).

It ended up not working for what I wanted to use it for because the domains in question end up forwarding, but at least there is a way. I did not find this documented anywhere.

@McMurphy
Remember that the routes have been added correctly on both VPN endpoints to work. So also check the remote site.

Also ensure the the respectively remote networks are entered in the Wireguard settings at allowed networks on both sites.

@Ximulate said in Routing rather than Gateway Group?:

but at least in my use case I think policy routing might be easier to manage

Why?

You can specify the failover group as the default gateway. So it is used by any device behind pfSense as well as by pfSense itself.
Policy routing rules have to be defined on each interface on the other hand.

The meaning of policy routing is to direct traffic from certain sources or to certain targets to a specific gateway.
If this is, what you want, you can go with it. Otherwise I'd prefer a gateway group as the default.

@oscar-pulgarin What VLAN ID's does your ISP say that you need?

If for example they use ID 100 for internet, I'm thinking you should do the following...

Create a VLAN with ID 100, using the physical interface used for WAN (igb0 for example). This is under Interfaces > VLAN's Under Interfaces / Assignments, click the drop down box for WAN and select the newly created VLAN.

That should take case of your internet traffic.

To pass through IPTV I suppose you have to add that VLAN ID to both WAN and LAN as well as any switches that sit between pfsense and your TV-box.

@fcostars Descobri!

O link da operadora ALGAR não deixa passar, mudei o link para a operadora da vivo para testar e funcionou!

Que loucura!

Obrigado pessoal!

@Gertjan Thanks for the tip @Gertjan!
I have done similar modifications of the config when changing NIC's. And it is as you say nothing more than search and replace. Didn't think about that for this type of change though, so this goes into my list of good things to remember...