Well, i figured it out.

I was doing the logical thing by adding the remote network to each side (Site B) and to the OpenVPN service (hosted on Site A). And that wasn't working.

So I started messing around with the openvpn firewall. Turns out that you need an additional explicit route on the mobile client server config.

Source: openvpn mobile client subnet (192.168.1.0/24 in this example)
Destination: any

Now the traffic routes. I'm sure that is documented somewhere but i couldn't come up with the right search phrase. I only figured it out with lucky guesses.

Now those lucky bastards on OpenVPN Client can see the network resources on Site B. (And much more since this is a mesh setup.)

Based on my understanding (which might be wrong), set your Gateway Group as the default Gateway then you shouldn't need to modify the firewall rules.

In my case, I need to modify the firewall rules to block all but high priority devices from using the backup (Tier 2) WAN. But, if thats not a concern of yours then I think you should be fine.

Found the problem.

I swapped out SIM cards and everything worked fine. Therefore, I tracked it down to having an issue with the cellular provider.

@romal-amarkhail said in Can't ping/access internal network from LAN or WAN interface.:

Any idea what might be wrong here?

Do you know for a fact that the unit responds to pings? And if your LAN is 192.168.2.0, why is the wifi on 192.168.1.0? Are you using a /16 mask? Is your wifi in AP mode?

It's more interesting, i switch Trigger Level to High Latency, and some time later pfSense himself switch it to Packet Loss! I didn't understand, why it happens.

And for sure it's dpinger bugs related, case i have checked "Disable Gateway Monitoring Action" and then made reconnect on router 2 (3 gateway) and in "Gateway status" i get on 3 gateway "Danger, Packetloss: 100%" on 3 gateway, i have check - traffic still goes through 3 gateway (router 2) without any problems, but dpinger thinks that it's dead for sure forever, till i make "save and apply" in any gateway settings.

I didn't now how to make monitoring work in pfSense. :(

I guess this issue is then solved.
Thanks for the help.

VLANs, VPNs, anything virtual.

GRE Tunnel Bonding Protocol [https://tools.ietf.org/html/rfc8157](link url) - "Single flow may use the combined bandwidth of the two connections.
Can this be implemented in pfSense?

It seems Layer2 bonding is one solution. " since load balancing in bonding takes places in Ethernet frames, even a single TCP/IP connection will enjoy an increased band thanks to the presence of multiple links."
[https://zeroshell.org/load-balancing-failover/#vpn-bonding](link url)
Can this be implemented in pfSense?

I believe I am now sorted.

I have left don't pull roots unchecked and in firewall/rules/secondlan advanced options/gateway I chose WAN_PPPoE.

I now have internet connection via VPN on igb1 and igb2 and connection not through VPN on igb4 just as I wanted.

Many thanks for the help

@dimskraft said in Dpinger sendto error: 65 on one of identically configured WANs:

sendto error: 65

Maybe this is relevant?

https://forum.netgate.com/topic/98656/gateway-send-to-error-65

@hebein glad to help you! I think log analyzing will help to reach 100%)

Put the other members of the gateway group on a lower tier. If you have them all on tier 1 then it won't switch when the down member comes back.

Well, the only rule that has seen any traffic at all is the Default allow LAN to any rule, so nothing is being blocked. That's why I suggested you try looking at it from the Synology side.

Sure with the HA proxy you can do that. I do it now for a couple different fqdn.

But that is going to work with http protocols, not going to be able to work with say smtp.

Add each WAN IPs you want to use to the WAN interface. Firewall >Virtual IPs. Use type "IP Alias".

Go to Firewall >NAT >Outbound. Switch into the manual mode. pfSense should take over the automically generated rules for each of your subnets into the manual mode. Edit each one, go down to the translation address and select the outbound IP from the drop-town you want to assign the respective source network.