Hi silfen,
I don't know if you already solved this, but we have a similar configuration, but virtualized.
And yes, we use VLANs to segment different traffics (operation LAN, WAN or different WANS, clients LANs). You must use a VLAN tagged dot1q cappable switch.
Then you can use just one port as minimun to interconnect pfSense with the switch and configure that port on the switch as VLAN tagged port with all VLANs available 1-4096. On the pfSense you set that interface as VLAN, and then you can create many VLANs as you want and assign as virtual interfaces on pfSense. You can create then each VLAN for each customer + internal traffic, management, WAN, etc. VLANs for internal purpouse.
In that way you isolate each customer from each other not just at layer 3 IP/routing network range, but layer 2 MAC address.
You can even have the ESX Ethernet port configured as VLAN tagged and assign each VM to each VLAN number.
We have a testing VM (windows or linux) too that we can switch to any VLAN number to test anything as we were connected at the same customer LAN.
For the WAN public IPs, you can create Virtual IPs on the pfSense to NAT by ports or 1:1 to the interal IPs or even you can create a VLAN with the some Public IPs if the customer require a public IP at their VM.
Regards
JP