The primary address is assigned using a PPPoE connection, the other addresses are a block assigned on that subnet. If I setup OpenVPN on the primary, perfect, no issues. If I try and use one of the other addresses, they are VIP's then no go, won't work.
Now, I have see other references to this issue on the forum where the solution is to use localhost, assign the VPN to that. Doing that alone still did not work, assigning the VIP to localhost AND assigning OpenVPN to that DOES work.
OK, port forward works nicely, replaced the 1:1 rule with that.
Edit: Port forward does NOT work, I obviously did not wait long enough for the rules to reload. Setting it back to 1:1 and removing the port forward it's working again.