Routing with no NAT
I am trying to route two Pfsense boxes with each other on the same WAN ip subnet with private IP's behind (Saves VPN).
10.0.0.1/24 –- WAN 193.100.1.x ------------ 193.100.1.x WAN --- 10.0.1.0/24
Both PFsense boxes on their WAN have the default GW with NAT on from the LAN ip range. I have enabled manual NAT but said do not NAT if you are going from LAN to LAN. Both PFsenses have a static route enabled and firewall rules in. The issue I have is that it doesn't work. If I change the default GW to each other PFsense boxes ping works no problem.
What could be wrong
You don't have routing rules to route 10.0.1.0/24 to 193.100.1.x (WAN on the second pfsense) and then a route on the second pfsense that points 10.0.0.1/24 to the WAN on the first pfsense firewall. This is why it works when you set the default gateway to each other. It routes correctly.
I do have a route in as well as the default route between both pfsense boxes. Looking at wireshark both ways I see the ICMP request and I see the ICMP reply but the remote end doesn't receive the reply. This is the same if I ping from ether side.
Could you post your routes and outbound NAT rules? Do you have the proper WAN rules in place to allow communication?
GW for both 193.100.1.x
Manual NAT PF 1:
WAN source LAN subnet > PF2LAN subnet NO NAT
WAN source LAN subnet > * NAT
Manual NAT PF 2:
WAN source LAN subnet > PF1 LAN subnet NO NAT
WAN source LAN subnet > * NAT
I have put a temp solution in but I can't think but there is a bug when you have 2 routes on the same interface
–------------ GW 193.100.1.x ------------
WAN 193.100.1.x 193.100.1.x WAN
10.0.0.1/24 LAN --- OPT1 ------ OSPF ------ OPT2 --- LAN 10.0.1.0/24
I see what you are talking about as far as the NAT is concerned. You need to setup the following and I think it might work for you.
Keep you NAT rules as they are.
Create a GW. The IP should be 193.100.1.PF2.
Create a route that points 10.0.1.0/24 to that new GW.
Create a GW. The IP should be 193.100.1.PF1.
Create a route that points 10.0.0.0/24 to the new GW.
Leave the default GW on both to 192.100.1.x.
Its either that or you must setup the routes in the main GW (192.100.1.x). According to your post, you only are setting up defaults routes an not the specialized routes you require. Something some where has to tell the traffic where to go. If the pfsense FW are no doing it then the global router has to.
This is not a bug. I do this in my test lab all the time. Except that mainly I use private IPs since its a lab. Concept is the same though.
The anoying thing is what you have explained is what I did. Very strange. It only worked once I used an interface with no default route on it. I created the no NAT rules and the static routes but when I used the WAN interface it didn't want to work.
Is your no-nat rules about your NAT rules?
They are outbound NAT rules at the top of the table.
The no-NAT rules should be at the top of the list.
Could you post a traceroute screen shot from one to another?
From another post where someone got it working, so I thought I would ask here.
Is the Windows firewall disabled?