First thing is make sure you not pulling routes from your vpn service.

Doesn't matter if the vlans are directly connected to pfsense or not, still just a simple policy route. Just set your firewall rules for your policies for your downstream vlans on your transit interface that connects to yoru downstream router.

BTW moved this to routing section, has zero to do with openvpn.. What your asking about is policy routing.

@rico This fix works for both IPv4 and IPv6. Thanks.

Came here to backup @candlerb. We're used to ECMP routing across two VTI tunnels on ASRs and such, but the ASA (due to the asymmetric path check) doesn't allow this.

This seems to be due to the ASA assigning an outbound VTI interface (E.g. VTI1) to the flow state table and mandating that return traffic also return on that external interface, when in realty BGP will load balance return flows to VTI2. It definitely presents a confusing issue at first.

Our way around this is to disable multi-pathing by decreasing outbound MED advertisements and increasing LOCAL_PREF for a designated 'primary' VTI interface.

Are you trying to setup a L2 site-to-site connection with your suggested VPS?

Hi

Actually VPC's DHCP server issued non-canonical interface address 10.162.0.10/32
with gateway 10.162.0.1 for network 10.162.0.0/20

I think the reason is that VM attached not to real (not to emulated) ethernet. and all communication should performed via GW

Routing table looks (look at vtnet1 routes):

Internet: Destination Gateway Flags Netif Expire default 10.200.0.1 UGS vtnet0 10.162.0.0/20 10.162.0.1 UGS vtnet1 10.162.0.1/32 42:01:0a:a2:00:0a US vtnet1 10.162.0.10 link#2 UHS lo0 10.162.0.10/32 link#2 U vtnet1 10.200.0.0/24 10.200.0.1 UGS vtnet0 10.200.0.1/32 42:01:0a:c8:00:0a US vtnet0 10.200.0.10 link#1 UHS lo0

On linux (another instance)

qq@vm-1:~$ ip r default via 10.162.0.1 dev ens4 proto dhcp metric 100 10.162.0.1 dev ens4 proto dhcp scope link metric 100 qq@vm-1:~$ ip n 10.162.0.1 dev ens4 lladdr 42:01:0a:a2:00:01 REACHABLE @vm-1:~$ ifconfig ens4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1460 inet 10.162.15.221 netmask 255.255.255.255 broadcast 0.0.0.0 inet6 fe80::4001:aff:fea2:fdd prefixlen 64 scopeid 0x20<link> ether 42:01:0a:a2:0f:dd txqueuelen 1000 (Ethernet) RX packets 383 bytes 502096 (502.0 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 365 bytes 49133 (49.1 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

You need a WAN CARP VIP on each WAN and set Outbound NAT to use that.

What you have is an invalid HA configuration. Both WANs should be on both firewalls in a Multi-WAN configuration.

"No address record" means one of two things:

It can't reach the Internet, typically because there is no default gateway in the routing table. Check your gateway settings, make sure the default is set as expected, then save/apply. If you are using a gateway group as default, try it with a single WAN gateway. Look under Diagnostics > Routes and see if you have a default listed. Your DNS settings are not correct or it otherwise cannot reach upstream DNS servers.

I read a bit more about it and I think I must use vSphere Hypervisor. Thank you

Fun fact, if I start pimd without a proper configuration (only interfaces are correct), kill it and then start again igmpproxy, it works. Multicast routes are correctly received.
Does anyone know the reason of this behaviour ?

See down below for screenshots of router1.

So I have done some more testing and have narrowed it down. When using a PC on this router1's LAN, downloading is using WIFILink1 and uploading is using WIFILink2. So I changed the Firewall rules not to use the gateway group but to use only the WIFI2_GW on both routers.
Router1:0_1550870097055_07a1e25f-edcb-43bc-8c74-1b156950e876-image.png
Router2:0_1550870864772_71e4ffd4-949d-495b-9292-45bdee09f186-image.png
Some traffic is still using WIFILink1. I am not sure how. See traffic graphs on router1 after I disabled the WIFILink1 interface and then enabled in on router2 with the above rules to use WIFI2_GW and WIFI_GW_2:
0_1550870647694_3d806ef3-a5bc-458e-93ad-9c6940e2d28e-image.png

Maybe I am missing something in my settings or my understanding.

Router1 screenshots:

0_1550790411939_4b984daa-0422-4924-a48c-a5262e12a007-image.png

0_1550790446656_ec6ded53-07eb-4fcd-b0e7-be4665ed1796-image.png

0_1550790756147_b40b524e-5191-470d-a5bf-3d3e9540cda2-image.png

0_1550790794079_2db95bef-d01a-433f-aab2-98f9fd59a9ed-image.png

@jaredadams

You are 100% correct here. Please accept my apologies. I won't try to make excuses or give explanations because there are none. Not really much more to say than that.

Glad rebooting the VM got you up and running.

Glad you got it sorted..

Mind sharing what specific device this was on - so future readers might learn from your experience.

Currently all the problems that I have are because of a misconfigurated appliance. Our case is a kind of special, because we need to work side by side with our old firewall and this is causing some troubles. For example, the public IP address that I was trying to use , was still used by the old firewall. This I noticed it when I went to Diagnostics/ARP Table and I found out that the IP address that I wanted to use is still in use.

Hey,
i got the new mechanic but until you wrote about browser refresh was difficult to understand due to double (default) labeled gateway while I use ipv4 only.

Thanks all for support

Thank you for responding Pat. I'll start with some background.

I have a 4 port NIC assigned as WAN1, WAN2, OPT1, and OPT2.
OPT1 and OPT2 are in a LAN bridge in case I needed another LAN port for whatever reason.
0_1550730710630_Interface Assignments.PNG

0_1550730780799_Bridges.PNG
Here are the interfaces in my dashboard.
0_1550731230613_Interfaces Edit.png

To answer your first two questions, both WANs are seen as a DHCP connection as shown.
0_1550730977974_Gateways Edit.png

This is the gateway group I have.
0_1550731101330_Gateway Group.PNG

The firewall rule for the LANBRIDGE interface (my LAN) is set to use the gateway group I named "Failover".
0_1550731055893_Firewall Rule.PNG This is where I think the problem may lie but I'm not sure.

Here are some of the thresholds for Gateway 1 (Comcast).
0_1550731185009_Gateway 1 Thresholds.PNG

Since WAN2 wasn't working as a failover, I instructed the client to simply use the SSID from the AT&T 4G gateway modem so they can have something. I haven't yet put the AT&T into bridge mode yet as there doesn't seem to be a "proper" way to do it. It seems the true WAN IP won't for this 4G modem won't be on the WAN2 interface unless we pay for a static IP, but either way, double-NATing shouldn't be a problem as all the client needs is a simple internet connection to function. If it was working correctly I would have disabled the WIFI on the AT&T device (actually not sure if it will even let me. This thing is pretty locked down.) Please let me know what you think of if there are any other pieces of information that would help in solving this issue. Thanks in advance!

@KOM Ok, thanks. Let me give that a try.

Jeff

@rookiee Of course. Not matter WAN assigned IP dinamically or static.
https://docs.netgate.com/pfsense/en/latest/routing/multi-wan.html

Gabriel