Even if I do go to manual outbound NAT which really was not my intention and go back to Automatic Outbound NAT they are still listed.
When you switch back to Automatic then automatic does what it is supposed to do - it automatically puts these NAT rules in place. That does not create any security hole - they re just helper rules for client app. That is not where the pass/block decision is made.
If, for example, you do not want to allow clients to do anything to port 500 then you can use a firewall rule block that on LAN interfaces.