Static route between 2 pfSense
-
Hello,
I've tried many things, and read a lot of topics, but I don't understand what I am doing wrong.
Probably an easy question, but I am totaly confused right now.I have the following:
I want to be able to access the ESXi server from the workstation. What should I do? The DHCP server and gateway for the 172.25.22.0/24 network is on pfSense 1
In my understanding, I don't need anything to do on pfSense 1, because the traffic will go directly from pfsense2 to the ESXi server. Is this correct?On the pfSense 2 I tried many different things, but it won't let me connect to the server from the workstation. On diagnostic > Ping I am able to ping the server. If I select the subnet from the workstation I am not able to ping the server. So I thought it might be a firewall problem. I created a pass all rule on the firewall, but it didn't work.
What should I do? And what can I try to troubleshoot the problem?
-
You need to add a route to the ESXi for the network 192.168.5.0/24 dericting to pfSense2 address 172.25.22.12.
However, if you also want to access VMs, also on these VMs a route will be needed.
-
Thanks for your reply.
So I need to add a static route on the ESXi host? Is it also possible to do it without adding static routes on the ESXi host and/or VM's?
Maybe it is better to use the following setup:
If I am right, I need the following static routes:
pfSense 1:
192.168.5.0/24 gateway: 10.0.0.2pfSense 2:
172.25.22.0/24 gateway: 10.0.0.1Is this correct?
-
Using a transit network between routers like that is always better.
Yes, your routing looks sane. Firewall rules on the transit interfaces will also need to pass the source traffic from the downstream networks.
-
Yes, that's the basic pattern for any situation where you have to route between two networks that are not directly connected to the same router. Applies to site to site VPNs and other tunnel solutions as well.
-
That just made me so happy to see someone come up with that. I feel real joy.
-
I added a pass all rule on all 4 interfaces. (to test)
I am not able to ping the workstation from pfSense 1. But I am able to access the workstation from the LAN 172.25.22.0/24 net (using RDP).
I am not able to ping the ESXi server from pfSense 2, and I am not able to access the ESXi server from the workstation.How can I troubleshoot this?
-
I think this might be a NAT problem.
pfSense 1 is a fresh install, so everything is default.
pfSense 2 is our main pfSense, with a 3CX PBX behind it. So NAT is set to 'Manual Outbound NAT rule generation. (AON - Advanced Outbound NAT)'Do I need to add some NAT mappings?
Edit: I found the problem: the ESXi host had a static IP. Apparently this isn't working. How can I make it work with a static IP?
-
There should be no reason to NAT between those two networks.
Do not set gateways on the transit interfaces themselves. They should just be enabled and have IP addresses and netmasks. You create a gateway in System > Routing but do not place it on the interfaces themselves.
If there is outbound NAT for your transit interfaces, disable or delete those rules.
Not exactly sure what you're talking about with the ESXi address. Routing doesn't care how the addresses are assigned. The ESXi host needs to allow traffic from the remote subnets and have 172.25.22.1 as its default gateway. Same with any VMs you place on the 172.25.22.0/24 vSwitch.
-
The ESXi address was set static on the host. So it was not assigned by pfSense. Once I changed it to DHCP, everything worked fine.
I will try to change it back to static (because the pfSense is a VM on this host), and see what happens -
There should be no reason to NAT between those two networks.
Do not set gateways on the transit interfaces themselves. They should just be enabled and have IP addresses and netmasks. You create a gateway in System > Routing but do not place it on the interfaces themselves.
If there is outbound NAT for your transit interfaces, disable or delete those rules.
Not exactly sure what you're talking about with the ESXi address. Routing doesn't care how the addresses are assigned. The ESXi host needs to allow traffic from the remote subnets and have 172.25.22.1 as its default gateway. Same with any VMs you place on the 172.25.22.0/24 vSwitch.
I have a similar setup with a transit network between two pfsense and the only way hosts from a LAN behind pfSense 1 can get to hosts on the LAN behind pfSense 2 is if I assign the gateway in the transit interfaces.
-
Then you are doing it wrong.
In the diagram above:
On pfSense 1:
Make an interface for TRANSIT with 10.0.0.1/29
Make sure firewall rules on TRANSIT pass desirable traffic from TRANSIT Net and 192.168.5.0/24
System > Routing Make a gateway TRANSIT_GW on TRANSIT for 10.0.0.2
System > Routing Make a static route for 192.168.5.0/24 to TRANSIT_GWOn pfSense 2:
Make an interface for TRANSIT with 10.0.0.2/29
Make sure firewall rules on TRANSIT pass desirable traffic from TRANSIT Net and 172.25.22.0/24
System > Routing Make a gateway TRANSIT_GW on TRANSIT for 10.0.0.1
System > Routing Make a static route for 172.25.22.0/24 to TRANSIT_GWAnd you're done.
If that does not work there is, perhaps, some policy routing on the LAN interfaces that needs to be bypassed.
-
Then you are doing it wrong.
In the diagram above:
On pfSense 1:
Make an interface for TRANSIT with 10.0.0.1/29
Make sure firewall rules on TRANSIT pass desirable traffic from TRANSIT Net and 192.168.5.0/24
System > Routing Make a gateway TRANSIT_GW on TRANSIT for 10.0.0.2
System > Routing Make a static route for 192.168.5.0/24 to TRANSIT_GWOn pfSense 2:
Make an interface for TRANSIT with 10.0.0.2/29
Make sure firewall rules on TRANSIT pass desirable traffic from TRANSIT Net and 172.25.22.0/24
System > Routing Make a gateway TRANSIT_GW on TRANSIT for 10.0.0.1
System > Routing Make a static route for 172.25.22.0/24 to TRANSIT_GWAnd you're done.
If that does not work there is, perhaps, some policy routing on the LAN interfaces that needs to be bypassed.
Thanks for the reply. That's exactly the way I had it, but I'll doublecheck the FW rules just in case. I'm using a /30 network for the transit network with 10.10.10.1 and 10.10.10.2 on each end, but that should not make a difference.
-
/30 is, of course, just fine.
Putting a gateway on an interface makes pfSense treat it as a WAN which is probably not what you want.
That would enable reply-to on inbound states which might mask if there was a static route in only one direction.
-
/30 is, of course, just fine.
Putting a gateway on an interface makes pfSense treat it as a WAN which is probably not what you want.
That would enable reply-to on inbound states which might mask if there was a static route in only one direction.
You were 100% right! On pfSense 2 the static route to the LAN behind pfSense 1 had a typo ::) . Fixed that, removed the gateways from both the transit interfaces' settings and everything is working.
Thanks!
-
Excellent!
-
I know this is super old, but I have a variation on this issue.
I have an internal IP that is hooked up to another system as well, however it is not PF Sense on that side. Also, routes have been learned via RIPv2 so I don't really know how to handle the Gateway static entries, as they are not static. Any advice?
I should also say, I can reach the RIP learned routes on all internal parts of the different router interfaces. The only problem is behind the other router, it cannot get beyond PF sense hence no Internet access. I did a traceroute from one of the end points on the RIP learned routes, and internet bound traffic dies upon reaching the PF Sense interface.
-
RIPv2? Really?
-
Well, not to go into too long of a story, this is an old TDMA iDirect satellite system in which they did some really stupid networking things like using RIPv2 and even proxy ARP. Not under my control.
I've attached a diagram. So behind the RHEL server, new subnets are created fairly frequently and in order to inform the world of their creation, they are updated via RIPv2 on the front end of the RHEL server. This is actually much more fairly involved thing involving some other moving parts, but I believe this is the relevant to PFsense portion.
So my issue is, once a new subnet is created, PF is picking it via RIP but for some reason hosts on the private subnets behind the RHEL can't get anywhere on the Internet; inside any of my private nets yes, it works fine. My workaround for now was to add a GW to PF indicating specific manually added static routes for the private nets that lay behind the GW (10.10.1.11) and the route to it. For example I've added the GW RHEL and then put routes behind it for example to 192.168.1.0/24. This works ok (although for some reason DNS is not passing, but one thing at time). Just want to see if it's possible to do this same thing via RIP. It doesn't seem to work the way it is now
-
You probably need to add outbound NAT for all of the private subnets on pfSense WAN.
Manually adding the static routes very likely enabled pfSense to know what networks were downstream so they were picked up by Automatic Outbound NAT.
Nothing like that is possible when pfSense doesn't have the routes in the configuration since they are dynamically-learned.
