Routing between 2 pfSense appliances
I am building a test environment so that we can trial replication and fail-over of Exchange. Using VMware ESXi I have built 2 networks and had one pfSense with a WAN, LAN and OPT1 interfaces. LAN was for site 1 and OPT1 for site 2. The WAN interface is connected to our physical network and has a static IP. Site 1 is using 192.168.1.0/24 subnet addresses and Site 2 192.168.2.0/24.
In the above scenario, everything worked perfectly. All servers could see Internet via WAN port and could ping each other.
I now need to setup external mail to the 2 sites. I wasn't sure if 1 pfSense appliance would be suitable so I setup a 2nd for Site 2.
Site 1 has a pfSense with WAN (static), LAN (192.168.1.0/24) and OPT1 (192.168.0.1).
Site 2 has a pfSense with WAN (static), LAN (192.168.2.0/24) and OPT1 (192.168.0.2).
Pinging between the 2 sites no longer works, and the rules I have tried don't work. A tracert still shows it going out the WAN of the pfSense and never reaching the target.
I'm a newbie at all this networking stuff, and it is confusing me. I have now removed all rules to start from scratch with some help :)
Can people advise on the rules I need please? Many thanks!
First, if I understand what you have written, you have a basic routing problem. Both sites cannot have have the same subnet on the opt interface unless you are using nat. Second, did you disable the blocking of private ips on the wan interface. Do you have 4 routing rules in the default gateway of the wan address to tell traffic for those network to use pfsense, you can also create those routes on pfsense. 2 for each of the 2 subsets on the other pfsense machine. Are you running rules wide open since you are testing exchange failover?
I used the IPs from the same subnet as they were directly connected. Don't really understand why this wouldn't work though, but I'm a newbie :P
Private networks aren't blocked on either of the pfSenses.
The WAN side routing on our physical network isn't configured yet, but will be. I'm just trying to get the 2 sites talking so that Exchange and AD sync (again).
Am I doing this the right way or is there a better/easier way to set this up?
If they are no longer directly connected, then the subnets need to change otherwise the computers will think its partner should be on the same network and not consult any gateway. If I understand you correctly, you don't need an OPT interface for this test. The way I am thinking you have this setup is like this:
Exchange1 -> LAN1 -> pfSense -> WAN1 -> WAN2 -> pfSense -> LAN2 -> Exchange2
So the traffic would flow like this:
192.168.1.10 -> 192.168.1.1 -pfSense1 Rules/NAT-> static1 -> static2 -pfSense Rules/NAT-> 192.168.2.1 -> 192.168.2.10
The only reason I can think of for the OPT interface is if you are trying to test having a DMZ.
Right so I need a rule in my network to bridge them…. think that makes sense. I was trying to avoid that and them talk on the OPT connections directly.
Could this be achieved with one pfSense but have 2 MX records externally for different external IPs all arriving at the one pfSense?
Sure, but how are the Exchange servers going to be setup in the long term. Seems that you had it running behind 1 FW and it was working, unless I misunderstood, one exchange behind LAN and the other behind OPT1.
You don't have to create a rule in your network if you don't want to. You can put a route on each of the 2 pfSense machines that points to the other for the remote subnet.
192.168.2.0/24 routes to static2 (WAN of pfsense2)
192.168.1.0/21 routes to static1 (WAN of pfsense1)
If you are going to use 1 pfsense machine, you will need at least 2 external IPs to handle 2 mx records to 2 different exchange servers.