Well, I figured out the issue, so thought I should post what I found, even though I feel a bit stupid now. Seems the main problem was a lack of knowledge on my part and that of Comcast Tier 1 support. Basically I had my gateway IP and static IP reversed. Turns out that since we were originally using the Comcast CM as a modem/gateway without a firewall behind it, and then later set up the firewall in the CM's DMZ, the gateway IP was functioning as our public static IP. It didn't help that the person who set up the network had documented the gateway IP as our static and vice versa. And Tier 1 support apparently had no clue. It took Tier 2 support to point out my mistake and of course it seems fairly obvious to me now. I suppose my one remaining question is whether this is typical behavior of static IP implementations or specific to Comcast and/or other ISPs? Either way, lesson learned. I should note one thing. I am 99% sure I did try reversing the gateway/public IPs when I first failed in configuring the static WAN interface, and that it did not work. What I did differently this time, however, was power cycle both the CM and FW, as opposed to just rebooting each; a simple step, mentioned by others in various posts, that might have helped me solve this sooner. Another lesson learned.

@viragomann I give it a try tomorrow, thanks !

@incognito Were u able to make this work? Since WG has been disabled in 2.5

@luckyh_de said in Multi-WAN with Backup down: So i have to prevent any Packet to the LTE-router AS Long as primary ist okay Hi, The failover mechanism does not allow this, you definitely need something that, which tells the firewall that the connections are alive (minimum GW pinger ICMP traffic)

It turns out that I have to set up a bridge in Interfaces→Bridges. For mDNS bridging I also set up Avahi between the different subnets.

@cool_corona Yes I know it looks so - but that's not the case.

@ofloo This is not limited to IPsec this happens in wireguard also. Not sure why but sometimes reloading some settings makes it not filter maybe it's I'm just imaging it but it comes and goes and it's not limited to just IPsec. I have do not filter traffic on same interfaces, I have just allow all traffic on the interface so no any firewall rule is there just allow any from any to any and yet !!! It filters. Lately it happens to happen more on WIREGUARD Interface then it does on IPSec.

Also there is nothing in filtering rules to deny anything all the interfaces are allowed to pass through the traffic. Neither its showing anything on the system logs as well

I put my camera on the same subnet, but I am not very happy with that. My MQTT devices did work cross VLAN, but I had a lot of errors telling me the packet was too short, shorter than expected. I moved these too to the same subnet and the errors are gone. There seems to be a layer 3 routing issue in pfsense. Can anyone who understands what is happing comment on this? Thanks, Sebastian

@viragomann how would i do that over the same port as the dhcp etc etc

Wooow nice :) I was not aware that it might work this way :) I've never connect internet to the router from lan site :P (at least not with the intention that it will work) It's work perfect Thanks for help

Happy to say that this issue has been fixed in the latest 2.5.1 snapshots: https://redmine.pfsense.org/issues/11602 Before - 2.5.0: --- X.X.X.X ping statistics --- 500 packets transmitted, 500 received, 0% packet loss, time 701ms rtt min/avg/max/mdev = 0.175/25.373/109.791/27.343 ms, pipe 8 After - 2.5.1-RC: --- X.X.X.X ping statistics --- 500 packets transmitted, 500 received, 0% packet loss, time 96ms rtt min/avg/max/mdev = 0.181/0.267/0.365/0.040 ms A big thank you to entire Netgate / pfSense team for addressing this so quickly.

I've since updated this script to handle failover to a second VPN where required. #!/bin/sh # Variables # VPN_IFACE1 is the primary VPN interface, VPN_IFACE2 is the backup VPN interface VPN_IFACE1=ovpnc1 VPN_IFACE2=ovpnc2 SQUID_CONFIG_FILE=/usr/local/etc/squid/squid.conf # Check whether VPN interfaces are connected and assign connected interface to VPN_IFACE. Exit if both are down VPN_IFACE1_STAUS=$(ifconfig $VPN_IFACE1 | awk '{print $2}' | egrep -o UP) VPN_IFACE2_STAUS=$(ifconfig $VPN_IFACE2 | awk '{print $2}' | egrep -o UP) if [ -z "VPN_IFACE1_STATUS" ] then VPN_IFACE=$VPN_IFACE1 elif [ -z "VPN_IFACE2_STATUS" ] then VPN_IFACE=$VPN_IFACE2 else echo "Both VPN interfaces down" exit 1; fi # Get current IP address of VPN interface VPN_IFACE_IP=$(ifconfig $VPN_IFACE | awk '{print $2}' | egrep -o '([0-9]+\.){3}[0-9]+') # Check current IP for VPN interface in squid.conf file VPN_CONFIG_IP=$(grep -m 1 "tcp_outgoing_address" $SQUID_CONFIG_FILE | awk '{print $2}' | egrep -o '([0-9]+\.){3}[0-9]+') # Check if the config file matches the current VPN interface IP, and if so exit script if [ "$VPN_IFACE_IP" == "$VPN_CONFIG_IP" ] then exit 0; fi # Replace the previous IP address in the squid.conf file with the current VPN interface address sed -ie 's/'"$VPN_CONFIG_IP"'/'"$VPN_IFACE_IP"'/' $SQUID_CONFIG_FILE # Force reload of the new squid.conf file /usr/local/sbin/squid -k reconfigure

@viragomann Like I said, it was late and I was frustrated, so I didn't approach troubleshooting in a methodical way. But I did try pinging the google dns servers (can't remember if I tried the gateway address or not) and the only one that worked was pinging google dns from the WAN using IPv6. I can tell you that I manually configured my laptop ethernet adapter with the static IP, mask and gateway, along with the default Comcast DNS servers (75.75.75.75/75.75.76.76) and connected directly to the CM. When I did that I did I was able to connect to some websites, but not others. The one that didn't load gave me a DNS error (can't recall the exact wording). I couldn't connect to anything from my PC when going through pfsense.

@pentangle Just to say that last night's candidate release fixed the second WAN issue, but pfBlockerNG needed to be disabled and re-enabled after upgrade because I could only ping the upstream gateway and the monitor IP (1.1.1.1) until I did that. Other pingable IPs (e.g. 1.0.0.1) were timing out until I did that.