@stephenw10 thanks, it should definitely work as I was envisioning.
Yes, I'd still have the VIP answering to ARP requests on that IP, and it will use the mac address of the interface where it was created to do so, then the destination IP field on those packets will be rewritten, then the rules will be applied and packets routed accordingly to the routing table.
I will then use tcpdump to identify all the hosts that are still using those IP I assigned as VIP and modify the config on the fly little by little.
btw netmap is amazing, the BSD network stack and what you guys built on top of it it's absolutely stunning, I have some small feedback on the UI but overall pfsense is definitely comparable to the major players firewall solutions out there, for sure you beat sonicwall and fortinet in my mind and I'm pushing to buy the actual netgate hardware because of that.