Routing issues with VIP's
-
We're running a PFSense VM in Hyper-V to do all of the routing for a silo'ed DRaaS environment.
There are 15 hosts in 5 subnets running in this Hyper-V enviornment. All are connected to the same virtual switch and on VLAN 900.
The PFSense box has 2 interfaces (WAN and LAN). The LAN interface has an IP of 10.50.90.254 with Virtual IP's assigned in each of the 5 subnets (10.50.2.1, 10.50.8.1, 10.50.15.1, etc). There are Gateways created for each subnet as well and set to .1 for each (10.50.2.1, 10.50.8.1, etc).
So the LAN interface has the following IP's:
10.50.90.254 - Actual IP
10.50.2.1 - Virtual IP
10.50.8.1 - Virtual IP
10.50.15.1 - Virtual IP
10.60.2.1 - Virtual IP
10.60.15.1 - Virtual IP
Most of the time everything works as expected. All of the VM's on different subnets can talk to each other and get out to the Internet. However, routing between the subnets seems to stop at random times. I can fix this by going into the Virtual IP settings, clicking "edit" on one them, clicking save (without changing anything) and clicking "apply". So no changes have been made.
This resolves the issue but obviously something is breaking and I'd like to figure out a permanent solution.
Here are some screen shots of the config:
VIP's:
Gateways:
-
Please do not take offense but that is a horrible, horrible design.
Why are they not on the same IP subnet?
Or why are they not on separate layer 2 segments (broadcast domains/VLANs/interfaces)?
Why do you need the gateways at all?
-
@derelict No offense taken.
The VM's are created by our backup alliance (Unitrends) and are basically just snapshots taken from our production environment. So the networking config matches the production environment and can't be changed in this silo'ed DRaaS testing environment.
I basically just need the host VM's in all 5 subnets to talk to each other and out to the internet via PFSense.
My thoughts were to create gateways for each of the subnets as the default gateway on each VM is configured to .1 (10.50.2.1, 10.50.8.1, etc). Virtual IP's assigned to the LAN interface in PFSense seemed the best way to do this.
Is there a better way? The routing works for the most part but randomly stops working until I apply the config in PFSense without changing anything.
-
Yeah. Make a host-only network switch for each one and a pfSense interface. Make a pfSense interface for each subnet. Put the firewall rules on each so they pass the traffic.
If you don't want to make a bunch of vmware interfaces, then "tag" VLAN 4095 to a pfSense interface and the VLAN tags will be there so you can make pfSense interfaces on them.
Beware that vmware scrambles your interface order after you add your 6th interface or something.
Bottom line is putting all those subnets on one broadcast domain and expecting them to communicate is simply not the way to go. Each one should be on separate router interfaces so those networks can, you know, be routed to each other.
Regarding the intermittent behavior, you are probably running into issues with different behaviors regarding ICMP redirects which is one of the main problems with that sort of design. I would never expect that to work 100%.
