Connecting Two Subnets with pfSense



  • Hi,

    I've set up a little VM test lab to try something I'd like to eventually replicate for real. The topology is like this:

    | | pfSense 1
    WAN: DHCP (internet)
    LAN: 192.168.1.1
    | |
    | pfSense 2
    WAN: DHCP (pfSense 1)
    LAN: 192.168.2.1

    Client 2 (Win10)
    LAN: 192.168.2.100 (DHCP pfSense 2) | | pfSense 3
    WAN: DHCP (pfSense 1)
    LAN: 192.168.3.1

    Client 3 (Win10)
    LAN: 192.168.3.100 (DHCP pfSense 3) |

    I hope this is clear enough. Essentially, I am trying to emulate two subnets with clients connecting to their own pfSense, each pfSense then connecting to pfSense 1 which acts as the internet gateway.

    So far, both Win10 VMs have internet access but cannot ping each other on separate subnets (as expected). I'd like to set up some routing so that clients from either subnet can communicate with each other. Unfortunately, I don't yet know enough about routing or firewall rules to set this up. It would be great if someone could point me in the right direction.

    Thanks!



  • You have to add static routes to both, pfSense2 and pfSense3 for the subnet behind the other one.

    At first you have to switch to static IPs on WAN interface of both instead DHCP.

    Then on pfSense2 go to System > Routing and add a gateway:
    Interface: WAN
    Name: pfSense3
    Gateway: pfSense3s WAN address

    If you want you can disable monitoring, enter a description and save it.

    Then go to "Static Routes" tab and add a new route:
    Destination network: 192.168.3.0/24
    AT Gateway select the pfSense3 GW you've added above and save it.

    Do the same on pfSense3 with the data of pfSense2s site.



  • Many thanks for your help.

    I tried what you suggested, but unfortunately I am still unable to ping clients on alternate subnets.

    Here's what I did:

    pfSense2

    • Change WAN to static (192.168.1.2), add pfSense1 as default gateway for internet (192.168.1.1)

    • Add pfSense3 WAN address as new gateway (192.168.1.3)

    • Create static route to 192.168.3.0/24 using pfSense3 gateway

    pfSense3

    • Change WAN to static (192.168.1.3), add pfSense1 as default gateway for internet (192.168.1.1)

    • Add pfSense2 WAN address as new gateway (192.168.1.2)

    • Create static route to 192.168.2.0/24 using pfSense2 gateway

    I am unable to ping Client 3 or pfSense3 from Client 2, and vice versa.

    Could I have missed something?



  • Have you added your allow to any rules for each interface used in Firewall:Rules?



  • Have you unchecked "Block private networks and loopback addresses" in the WAN interface setting of pfSense2+3?



  • @viragomann:

    Have you unchecked "Block private networks and loopback addresses" in the WAN interface setting of pfSense2+3?

    I hadn't, although it seems obvious now that I read the description. I've unchecked it now though, and rebooted both pfSense VMs just to be sure, but no luck.

    @elliotcater:

    Have you added your allow to any rules for each interface used in Firewall:Rules?

    No, I'll give that a go.

    Thanks guys



  • @elliotcater:

    Have you added your allow to any rules for each interface used in Firewall:Rules?

    I tried adding a rule on pfSense2 and pfSense3 for the WAN interface, passing packets from any source to any destination, using any protocol. Unfortunately, it didn't work. Did I apply the rule correctly?



  • It should work that way. You may restrict the source and protocol in this rule later.

    The Diagnostic > Packet capture helps you at troubleshooting.
    Go to pfSense3 and take a packet capture from WAN, select ICMP protocol an start it. Then do a ping at client2 to client3.
    Then switch to LAN and take a second capture. Post the outputs, please.



  • Okay, I performed the captures as suggested. The WAN outputs are attached (I hadn't enabled copy / paste in the VMs' configurations). The LAN captures for each pfSense installation showed nothing at all!






  • So from the PFSense1 Box, there are 2 LAN interfaces? Are they bridged interfaces or is there a hypothetical switch in-between.

    To be honest, it looks like you could do all of this with just 1 pfsense box?



  • I can't see a request to client3 (192.168.3.100).
    Requests at pfSense2 WAN go to only to 192.168.1.1!  :-\


  • LAYER 8 Global Moderator

    Yeah I don't get why you want to use 3 pfsense boxes/vms for just 2 subnets?

    In your setup out of the box pfsense nats, so you wouldn't want that, etc.

    To setup your 3 pfsense scenario your top pfsense would use 2 transit networks to connect to your 2 downstream pfsense boxes, and you would disable nat on them.  You would just setup routing on your top pfsense to your 2 downstream networks.  But again you could have hundreds of networks hanging off 1 pfsense.  The only reason for more than 1 would be a carp setup for redundancy or different geographic locations.



  • @elliotcater:

    So from the PFSense1 Box, there are 2 LAN interfaces? Are they bridged interfaces or is there a hypothetical switch in-between.

    To be honest, it looks like you could do all of this with just 1 pfsense box?

    Your image is almost correct. Please see below:

    I did think that perhaps this would be possible with a single pfSense box. I ended up with three by following a guide on connecting subnets with physical routers, of which there were three. I wanted to emulate that setup before delving further into the possibilities offered with pfSense.


  • LAYER 8 Global Moderator

    "following a guide on connecting subnets with physical routers,"

    Yeah that was some moron using some stupid off the shelf user wifi router.. Not even running some sort of 3rd party firmware that supports vlans.

    There is no reason at all do that sort of setup to have 2 networks.  You only need 1 pfsense to do this.  Your making a mess trying to do it the way your doing it.

    You need your 1 pfsense.  Add an opt interface, or as many opt interfaces you want for how ever many networks you want.  Connect these opt interfaces to different vswitches.  Setup your firewall rules on your opt interfaces = done.



  • @johnpoz:

    "following a guide on connecting subnets with physical routers,"

    Yeah that was some moron using some stupid off the shelf user wifi router.. Not even running some sort of 3rd party firmware that supports vlans.

    There is no reason at all do that sort of setup to have 2 networks.  You only need 1 pfsense to do this.  Your making a mess trying to do it the way your doing it.

    You need your 1 pfsense.  Add an opt interface, or as many opt interfaces you want for how ever many networks you want.  Connect these opt interfaces to different vswitches.  Setup your firewall rules on your opt interfaces = done.

    Okay, thank you; I will try with a single VM. Do I need to set up any static routing in addition to the firewall rules?



  • As long as you have your allow from any rules on the 2 lans they should talk to each other no dramas. Static routes just describe to your router, which gateway to use to get a hop closer to the destination network. Of course it all becomes moot if your router has all the gateways.


  • LAYER 8 Global Moderator

    No you do not need to setup any routes, pfsense knows how to route between its attached networks ;)



  • Thanks guys, I'll fire up a VM this morning and let you know how I get on.



  • Okay, I set up a new single pfSense VM. Please see below to see how it's set up:

    I also created a firewall rule, although it's probably configured incorrectly. There is a default rule to allow LAN to any, so I figured I needed one for OPT1. Following the default rule, I set interface to OPT1, protocol to any, source to OPT1 net, and destination to any.

    I can ping OPT1 (192.168.2.1) from Client 1 (192.168.1.100), but I cannot ping Client 2 (192.168.2.100) from Client 1. The same happens from Client 2 to Client 1.

    I'm sure the firewall rule is at fault. Can anyone point me in the right direction?

    Thanks



  • Are the client configured to use the correct gateway?
    Client1 - 192.168.1.1
    Client2 - 192.168.2.1

    Have you set a gateway in the pfSense LAN and OPT interface config? That must not be set.


  • LAYER 8 Global Moderator

    And is there a software firewall on these vms?  Windows for example blocks ping..

    Out of the box the lan rules on pfsense would allow you to ping opt2 network.  If its not answering points to firewall on that client..



  • @viragomann:

    Are the client configured to use the correct gateway?
    Client1 - 192.168.1.1
    Client2 - 192.168.2.1

    Have you set a gateway in the pfSense LAN and OPT interface config? That must not be set.

    The clients have the correct gateway set via DHCP, and neither LAN nor OPT have a gateway set. Both clients have internet access and can ping both LAN and OPT interfaces, but not each other.

    @johnpoz:

    And is there a software firewall on these vms?  Windows for example blocks ping..

    Out of the box the lan rules on pfsense would allow you to ping opt2 network.  If its not answering points to firewall on that client..

    I'll check, but I've never had the Windows Firewall block pings before. Unless that's because I'm normally pinging from the same subnet.



  • @johnpoz:

    And is there a software firewall on these vms?  Windows for example blocks ping..

    Out of the box the lan rules on pfsense would allow you to ping opt2 network.  If its not answering points to firewall on that client..

    That was it, thanks! I disabled the firewall on both clients, and they were able to ping each other.


  • LAYER 8 Global Moderator

    There you go see 1 pfsense and you have 2 network, you could have as many networks as you wanted that your VM host would be able to support ;)  Just using 1 pfsense vm.

    Now if you wanted you could start getting fun with it and use it to play with vlan tagging, etc. vs your actual physical network simulation you have going on now.  Using port groups on your vswitch and then setting up the vlans on the 1 vm nic you have connected to pfsense, etc.



  • @johnpoz:

    There you go see 1 pfsense and you have 2 network, you could have as many networks as you wanted that your VM host would be able to support ;)  Just using 1 pfsense vm.

    Now if you wanted you could start getting fun with it and use it to play with vlan tagging, etc. vs your actual physical network simulation you have going on now.  Using port groups on your vswitch and then setting up the vlans on the 1 vm nic you have connected to pfsense, etc.

    I am interested in VLANs and have no experience with them, so I think I will try setting something like that up next.


Log in to reply