@jimp:
The problem lies in how FreeBSD and others source UDP replies from hosts with multiple interfaces in certain cases, and also how IPsec works with the FreeBSD kernel.
The SNMP daemon will always reply to a query from the interface IP "closest" to the client, from a routing perspective. If you have no direct connection and no static route, it will reply via the default route (WAN IP), and the WAN IP isn't a part of the IPsec tunnel, so the traffic doesn't get back to the client, and even if it did, the IP wouldn't match and it would be dropped.
Binding to the LAN IP forces it to only use the one single IP, which then matches IPsec and it works. Because it isn't bound to any other IP, it can't use the "wrong" one to reply. This is the best fix. If you need to reach it via another interface, use port forwards to nudge the traffic to the LAN IP.
The routing workaround fixes it because the route will make the firewall send the traffic out the LAN IP, and it is then grabbed by IPsec.
For the same reason you also can't query the SNMP daemon on an interface IP that is "far" from you. For example if you're in the LAN subnet, you can't query the WAN address and get a proper reply because the response will comes from the LAN IP, when the query went to the WAN IP, and it will be dropped.
Thank you!
Feeling too complicated. already more than I can solve
I decided to change the way
Because all I need from both WAN and LAN sides are receiving the data can either