To update this post:
I have upgraded Netgate SG-3100 to newly released v.21.02-p1 and also upgraded the unbound package to the unbound-1.13.1 with:
pkg upgrade -fy unbound; pfSsh.php playback svc restart unbound
Unfortunately, all the NAT issue came back again. HAProxy does not respond to HTTPS port from outside the LAN network, and Xbox NAT status changed to closed again even though both using the same port forwarding settings that I had in the previous version 2.4.5p1.
I have reloaded v.2.4.5p1 again with the same port forwarding setting, and everything started working again. Xbox NAT status is Open, and HAProxy correctly working from WAN.