Thank you both very much!
I added the NIC and then rebooted the first time. Now I've followed your advice which worked like a charm.

If the DHCP server was giving you bogus DNS servers you can expect delays in anything on the firewall that was looking to resolve names. Totally normal.

@hsv said in Debugging static routes:

How do I pass the traffic into the transportvlan.

Pass rules on the transportvlan interface.

I have no idea what a static rule is.

You just made it make sense.

I had been going over and over about going around the default route but I kept coming up with nothing because traffic was going to the Internet (AKA the default route), therefore it had to take the default route to get there. It made no sense. ☹️

Thanks a million for your help!

@planedrop Yep, recognize you from the Unifi forums. Running pfSense in front of my UDMP has worked out great once I sorted out the Outbound NAT rules. For the last 30 days, I haven't had a need to touch anything in pfSsense, it just works.

So other than load balancing my 2 WANs, I don't do anything on pfSense, everything else is happening on the UDMP. I don't have any port forwarding in place right now as I don't really need it, but my VPN to my work machine on a corporate network (using Cisco Anyconnect) has been working flawlessly from my personal home workstation.

That said, I would imagine that in order to make port forwarding work properly, one would have to make entries on both the UDMP as well as in pfSense and I'd imagine pfSense will let you make port forwards sticky on one WAN or the other.

As I mentioned on the UniFi forum, once I get my dual symmetrical GigE WANs up and running, and be doing some benchmarks from machines behind the UDMP, as well as from a box hanging directly off the pfSense appliance.

Have you worked through https://docs.netgate.com/pfsense/en/latest/book/trafficshaper/limiters.html ?

-Rico

@johnpoz

This solves my problem.

Thank you two buddies!

It won't be "proper" HA as there is no way for the dynamic WANs to participate in CARP or to trigger an HA failover. There would be no seamless failover of clients if the primary node failed to the secondary while the dynamic WAN was in use.

So it can function in the most basic sense -- Multi-WAN could possibly work (gateways may be tricky, for one) but it wouldn't be a good experience and that's one of the reasons we say that type of configuration is unsupported.

If there is only a single shared CPE for the dynamic WAN, you could enable routing mode in the CPE (if it has one) and then setup HA w/CARP on pfSense in the private subnet behind it. Setup 1:1 NAT on the CPE to map all traffic on its public address to the private CARP VIP.

That won't work if each HA node has its own separate second WAN, though.

Post up screenshots of your firewall rules. And, like I said earlier, that extra router behind pfsense is probably causing the problems.

Jeff

Thanks.
This pfSense VM article is what I followed.

Bit confused with the interfaces, so that may be an issue.

Heh... I kinda thought that might be the case. At least that one's an easy fix. Thanks again for all of your help!

I got my issue resolved and feel quite relieved - but also kind of embarassed for taking so long to find the problem. In the hope that it might save someone else from digging around for days, here is what I found.

Problem was: private IPs will not be routed. All my 192.168.xx.yy/24 networks are private networks and I force-routed them a little way but could not get them through all the way.

Solution was: set an outgoing NAT rule:
c274d0d7-6f2c-4f73-8f12-75283e7ab6a9-grafik.png

Again: router A is the openVPN server, it has subnet 192.168.225.0/24. The above setting is for router B, which has subnet 192.168.245.0/24 for LAN. This permits a host in B's subnet to reach a host in A's subnet. A corresponding NAT rule will be required on A for the opposite direction.

I my case server A will assign an interface address to B, so the NAT address needs to be B's openVPN interface address.

What else did I learn?

For one thing, Apple's version of ping supports some really helpful options:

-A will make a sound for each outgoing packet -a will make a sound for each incoming response -f will flood the target with ICMP packets. On an otherwise quiet system, this permitted me to see where my packets were going just by looking at pfSense's traffic graphs on the dashboard.

Another thing is, it took me ages to get to the solution but I feel that all the failures I have been through taught me more than I ever wanted to know ☺ 🎓 Keep working on your problems, eventually you will master them!

🤦💆
nice !

Thanks for the replies. Here are some screenshots of my LAN rules, gateways and gateway groups.LAN_rules.png Gateways.png Gateway_groups.png