Multi WAN Multi LAN special routing best practices. Virtual IP NAT one Server
So I have a new setup With 3 WANs and 3 LANS. I have a bunch of stuff load balancing and that's all fine. I have a special requirement for one of my servers however. I want its resources to be able to be accessed from a static ip, so I ringed my ISP and they gave me another IP, in addition to the one I have now for our phone system. In trying to decide\figure out how to do this, I did the following and it worked, however, I suspect there may be a better way to do it, but heres what I have done. The ISP had assigned us two individual IPs in a large network segment. a 255.255.255.252 mask to be specific. Both IPs are assigned to one Ethernet port.
Everything is virtualized here so adding an extra NICs and virtual switches to VMs effectively does not cost me anything.
I previously had this server on our old network that just had plain portforwarding to our little router and one modem. everything at that time was on the same network. Now I have segmented our network into Secure, Public, and VoIP, all on different subnets AND interfaces. I had initially put my server on the Secure network thinking that there would be a way with NAT to translate my one internal server IP to the new external IP for my server. I could not figure out how to make it work in that fashion so heres what I did instead.
I made a new virtual switch and connected my server and pfSense with another virtual NIC. So basically the server is on a completely separate network now. I was able to make the traffic from that network always translate to that special IP by binding a Virtual IP created in pfSense to a NAT rule for that interface. This works and all things are performing as expected.
I suspect there is a way to do it by keeping my server on the secure network by creating a NAT rule that just selects traffic based on the server's internal IP, but I could not get it to work. I suspect the problem lied in how I was specifying the network match rule. I set an outbound rule that was set to source was my internal server IP and the translated address was selected from the virtual interface created. One thing I was unclear on was how many bits to set the netmask to. because I am not trying to match a network but rather just one host, I tried 28-32 but it never worked. traffic just used the default route for secure network traffic.
So after being frustrated with that I configured it like I had previously mentioned because I knew I could at least make a rule that would match a whole network to translate. and indeed it does. I thought of using static routes would be right in this application, but I don't think so. let me know if you think otherwise.
So is there a better way? Let me know what you think! I left out other routing and network details that I didn't think were relevant, but let me know if light should be shed on something.
Sorry, there is no tl;dr. Its a complicated setup.