Found a problem with WANs showing a "Pending" status under Gateways when running pfSense version 23.09.1-RELEASE. Documented and I filed to bug report this week.

This appears to be caused by the process dpinger. Tied to using a DHCP from the ISP. Does not happen if ISP is using a static IP for the WAN.

Occurs if the cable (RJ-45) between the modem and router/firewall is unplugged/replugged or if the modem is power reset. The problem happens with a single WAN and is not related to using a failover configuration. Can stay stuck in Pending state and is hard to get out of. Try, disabling the interface, then be sure the modem is fully online and ready to issue the IP by DHCP, and then reactive the interface. I've also tried releasing the IP and reassigning it under the Interfaces page.

If your ISP is Comcast, I also found a bug in their lease time for DHCPs if their default value is changed. The problem I found is when the lease time is changed, the modem will fail at 1/2 the assigned lease time. Logs will show monitoring pings don't respond. Way to fix this was to reissue another DHCP lease time or release the IP and reassign it under the Interfaces page

Another problem I found is modems in bridge mode will randomly disconnect because they stop responding to ARPs. Think the phSense default is 1200. Found it was necessary to reduce under 5 mins. I am using 240 for net.link.ether.inet.max_age found under Advanced/System Tunables. So far, this change appears to help stop the disconnects.

@zaphanathpaneah said in pFsense cannot ping devices directly connected:

for example I want to prevent traffic from one subnet moving to another. I have 4.

Here is a simple example of locked down rules.

lockdown.jpg

Devices on this network can not talk to any of my other networks, because all of my other networks are rfc1918 space, and there is a rule that blocks that access.. While rules above it allow what I want.. Ping Pfsense IP, ask pfsense address on this network for dns, I also allow this network to talk to my pihole on another network for dns. I allow it to ask pfsense for ntp. But they can not talk to any other pfsense IP be it for dns or webgui or ssh or anything because of the specific this firewall reject rule. This also prevents them from access pfsense public wan IP for anything.. Because the last rule allows any any, that has not been block above it.

The rfc1918 alias contains all the rfc1918 space, so any of my current networks or future networks would all be in rfc1918 space... If I created a network outside rfc1918 space, then that any any rule at the bottom for internet access would end up allowing that traffic.

The Plus Version 23.05.1 presents the same symptoms described in https://redmine.pfsense.org/issues/14763. Please assist if there are any available clues to resolve the issue.

@rk4n3
If you check Status > Gateways what's the status?
If it's not shown up as online edit the gateway in System > Routing > Gateways and check "Disable Gateway Monitoring Action".

Is the gateway selected in System > Routing > Gateways > Default gateway, or is it Automatic?

If it still doesn't work, show your whole routing table, please?
Diagnostics > Routes

I found the problem. Appears to be related to this thread. I implemented the fix and both wans have been stable for 48 hours.

https://forum.netgate.com/topic/185334/pfsense-wan-gateway-randomly-goes-down

@Matt_Sharpe
I'm not aware of an issue in pfsense with this.
However, if there is switch or a vswitch (in case one is virtualized) in between both routers, you have possibly to allow MAC changes on these devices.

@viragomann Thanks, I got it working. To clarify it for anyone who may find this in the coming years by Google and be in need of a HOW-TO, the process consists of three steps:

Create a ProxyARP pool that covers the intended External IP subnet Add IP Aliases for each of the External IPs from that subnet you want in the outbound pool Add an Outbound NAT with the ProxyARP as its outgoing network

@viragomann - Great, thank you for the clarification :)

@planetinse That’d be hybrid or manual outbound NAT: https://docs.netgate.com/pfsense/en/latest/nat/outbound.html

@orphen76 Would an IP alias work ?

https://docs.netgate.com/pfsense/en/latest/firewall/virtual-ip-addresses.html

@viragomann thanks for the suggestions. Masquerading could work, I see.

I will not be trying this only because I found a workaround that suits what I needed, even if it's not a solution that would apply to everyone. For information to other forum members: I disabled the monitoring of the public IP gateway, so now it is considered always up. I made a gateway group with the private and public IP gateways, and configured private as tier 1 and public as tier 2. Then I made this gateway group the default gateway of the firewall. Now thing work as I had planned: if the upper level firewall is connected to LAN4, it becomes the gateway. If it is disconnected and the ISP router is connected to LAN4 instead, the router becomes the gateway.

The use of the "physical" IP only occurs for the monitoring ping. When routing packets to the public IP gateway, the firewall uses the virtual IP and everything works just fine.

@tmevent
Danke für das Angebot. 🙂 Könnte mal sein...

Gut möglich auch, dass die Chancen auf Hilfe in einem anderen Forum besser stehen. Bspw. reddit oder administrator.de.
Azure mit seinem SDN spielt da nach ganz anderen Regeln als ein Standard-Router wie pfSense, und ich glaube dort liegt das Problem.

Der aktuelle Stand ist folgender:

Ich habe auf AZURE eine Eigene Pfsense aufgesetzt und greife dort per IKEv2 drauf zu.
Leider erreiche ich aus dem Azure-LAN nur meine Pfsense. Weder das Internet noch die per site2site verbundenen Netze sind erreichbar

Setup:
Azure WAN 192.168.0.0/24, PfSense 192.168.0.4
auf dieser Seite gibt es keine Routing Tabelle, lediglich die Azure NSGs mit den Regeln für meine IPsec Tunnel

Azure LAN 10.3.0.0/24, Pfsense 10.3.0.5
Hier gibt es eine Routing Tabelle:

Unbenannt.PNG

Azure Server Windows Server 2019 10.3.0.4
2x Site2Site IPsec Verbindung 10.0.0.0/24 und 10.2.0.0/24 zur pfsense
mehrer Client IPsec Verbindungen zur Pfsense

Ich erreiche von der GUI der Pfsense sowohl meine Azure Server, als auch das Internet.
Ich erreiche über die Tunnel die Pfsense und den Server auf Azure.

Ich habe nun ein bisschen versucht die Pakete zu verfolgen(Diagnose>Paketverfolgung).
Wenn ich von meiner Azure-VM auf 8.8.8.8 pinge, sehe ich diesen Ping am LAN Port
20:41:46.674926 IP 10.3.0.4 > 8.8.8.8: ICMP echo request, id 1, seq 1054, length 40
und am WAN Port
20:45:53.162014 IP 8.8.8.8 > 10.3.0.4: ICMP echo reply, id 1, seq 1058, length 40

Was mir unklar ist warum sehe ich den Ping nur eingehend und nicht ausgehend? An welcher Stelle verlässt das Paket mein Netz und warum schickt die pfsense es nicht raus?

Wenn ich über Diagnose>Ping meine VM 10.3.0.4 pinge sehe ich eingehende und ausgehende Nachrichtem, so wie es sein soll.

WAN
20:48:47.679123 IP 192.168.0.4 > 10.3.0.4: ICMP echo request, id 39016, seq 0, length 64
20:48:47.680056 IP 10.3.0.4 > 192.168.0.4: ICMP echo reply, id 39016, seq 0, length 64

LAN
nichts

Ich stehe leider echt auf dem Schlauch. Mehr Infos schicke ich auch gerne, einfach fragen.
Ich freue mich auf jeden Hinweis ....