Connecting Two Subnets with pfSense

tyh

Hi,

I've set up a little VM test lab to try something I'd like to eventually replicate for real. The topology is like this:

| | pfSense 1
WAN: DHCP (internet)
LAN: 192.168.1.1
| |
| pfSense 2
WAN: DHCP (pfSense 1)
LAN: 192.168.2.1

Client 2 (Win10)
LAN: 192.168.2.100 (DHCP pfSense 2) | | pfSense 3
WAN: DHCP (pfSense 1)
LAN: 192.168.3.1

Client 3 (Win10)
LAN: 192.168.3.100 (DHCP pfSense 3) |

I hope this is clear enough. Essentially, I am trying to emulate two subnets with clients connecting to their own pfSense, each pfSense then connecting to pfSense 1 which acts as the internet gateway.

So far, both Win10 VMs have internet access but cannot ping each other on separate subnets (as expected). I'd like to set up some routing so that clients from either subnet can communicate with each other. Unfortunately, I don't yet know enough about routing or firewall rules to set this up. It would be great if someone could point me in the right direction.

Thanks!

viragomann

You have to add static routes to both, pfSense2 and pfSense3 for the subnet behind the other one.

At first you have to switch to static IPs on WAN interface of both instead DHCP.

Then on pfSense2 go to System > Routing and add a gateway:
Interface: WAN
Name: pfSense3
Gateway: pfSense3s WAN address

If you want you can disable monitoring, enter a description and save it.

Then go to "Static Routes" tab and add a new route:
Destination network: 192.168.3.0/24
AT Gateway select the pfSense3 GW you've added above and save it.

Do the same on pfSense3 with the data of pfSense2s site.

tyh

Many thanks for your help.

I tried what you suggested, but unfortunately I am still unable to ping clients on alternate subnets.

Here's what I did:

pfSense2

Change WAN to static (192.168.1.2), add pfSense1 as default gateway for internet (192.168.1.1)
Add pfSense3 WAN address as new gateway (192.168.1.3)
Create static route to 192.168.3.0/24 using pfSense3 gateway

pfSense3

Change WAN to static (192.168.1.3), add pfSense1 as default gateway for internet (192.168.1.1)
Add pfSense2 WAN address as new gateway (192.168.1.2)
Create static route to 192.168.2.0/24 using pfSense2 gateway

I am unable to ping Client 3 or pfSense3 from Client 2, and vice versa.

Could I have missed something?

elliotcater

Have you added your allow to any rules for each interface used in Firewall:Rules?

viragomann

Have you unchecked "Block private networks and loopback addresses" in the WAN interface setting of pfSense2+3?

tyh

@viragomann:

Have you unchecked "Block private networks and loopback addresses" in the WAN interface setting of pfSense2+3?

I hadn't, although it seems obvious now that I read the description. I've unchecked it now though, and rebooted both pfSense VMs just to be sure, but no luck.

@elliotcater:

Have you added your allow to any rules for each interface used in Firewall:Rules?

No, I'll give that a go.

Thanks guys

tyh

@elliotcater:

Have you added your allow to any rules for each interface used in Firewall:Rules?

I tried adding a rule on pfSense2 and pfSense3 for the WAN interface, passing packets from any source to any destination, using any protocol. Unfortunately, it didn't work. Did I apply the rule correctly?

viragomann

It should work that way. You may restrict the source and protocol in this rule later.

The Diagnostic > Packet capture helps you at troubleshooting.
Go to pfSense3 and take a packet capture from WAN, select ICMP protocol an start it. Then do a ping at client2 to client3.
Then switch to LAN and take a second capture. Post the outputs, please.

tyh

Okay, I performed the captures as suggested. The WAN outputs are attached (I hadn't enabled copy / paste in the VMs' configurations). The LAN captures for each pfSense installation showed nothing at all!

pfSense-3-WAN-capture.png_thumb

pfSense-2-WAN-Capture.png_thumb

elliotcater

So from the PFSense1 Box, there are 2 LAN interfaces? Are they bridged interfaces or is there a hypothetical switch in-between.

To be honest, it looks like you could do all of this with just 1 pfsense box?

viragomann

I can't see a request to client3 (192.168.3.100).
Requests at pfSense2 WAN go to only to 192.168.1.1! :-\

johnpoz

Yeah I don't get why you want to use 3 pfsense boxes/vms for just 2 subnets?

In your setup out of the box pfsense nats, so you wouldn't want that, etc.

To setup your 3 pfsense scenario your top pfsense would use 2 transit networks to connect to your 2 downstream pfsense boxes, and you would disable nat on them. You would just setup routing on your top pfsense to your 2 downstream networks. But again you could have hundreds of networks hanging off 1 pfsense. The only reason for more than 1 would be a carp setup for redundancy or different geographic locations.

tyh

@elliotcater:

So from the PFSense1 Box, there are 2 LAN interfaces? Are they bridged interfaces or is there a hypothetical switch in-between.

To be honest, it looks like you could do all of this with just 1 pfsense box?

Your image is almost correct. Please see below:

I did think that perhaps this would be possible with a single pfSense box. I ended up with three by following a guide on connecting subnets with physical routers, of which there were three. I wanted to emulate that setup before delving further into the possibilities offered with pfSense.

johnpoz

"following a guide on connecting subnets with physical routers,"

Yeah that was some moron using some stupid off the shelf user wifi router.. Not even running some sort of 3rd party firmware that supports vlans.

There is no reason at all do that sort of setup to have 2 networks. You only need 1 pfsense to do this. Your making a mess trying to do it the way your doing it.

You need your 1 pfsense. Add an opt interface, or as many opt interfaces you want for how ever many networks you want. Connect these opt interfaces to different vswitches. Setup your firewall rules on your opt interfaces = done.

tyh

@johnpoz:

"following a guide on connecting subnets with physical routers,"

Yeah that was some moron using some stupid off the shelf user wifi router.. Not even running some sort of 3rd party firmware that supports vlans.

There is no reason at all do that sort of setup to have 2 networks. You only need 1 pfsense to do this. Your making a mess trying to do it the way your doing it.

You need your 1 pfsense. Add an opt interface, or as many opt interfaces you want for how ever many networks you want. Connect these opt interfaces to different vswitches. Setup your firewall rules on your opt interfaces = done.

Okay, thank you; I will try with a single VM. Do I need to set up any static routing in addition to the firewall rules?

elliotcater

As long as you have your allow from any rules on the 2 lans they should talk to each other no dramas. Static routes just describe to your router, which gateway to use to get a hop closer to the destination network. Of course it all becomes moot if your router has all the gateways.

johnpoz

No you do not need to setup any routes, pfsense knows how to route between its attached networks ;)

tyh

Thanks guys, I'll fire up a VM this morning and let you know how I get on.

tyh

Okay, I set up a new single pfSense VM. Please see below to see how it's set up:

I also created a firewall rule, although it's probably configured incorrectly. There is a default rule to allow LAN to any, so I figured I needed one for OPT1. Following the default rule, I set interface to OPT1, protocol to any, source to OPT1 net, and destination to any.

I can ping OPT1 (192.168.2.1) from Client 1 (192.168.1.100), but I cannot ping Client 2 (192.168.2.100) from Client 1. The same happens from Client 2 to Client 1.

I'm sure the firewall rule is at fault. Can anyone point me in the right direction?

Thanks

viragomann

Are the client configured to use the correct gateway?
Client1 - 192.168.1.1
Client2 - 192.168.2.1

Have you set a gateway in the pfSense LAN and OPT interface config? That must not be set.