Routing between subnets

ruben.rothermel · Feb 4, 2015, 2:05 PM

Hi,

I've been using PfSense for a year as a router (gateway, firewall, content filter, DHCP) in a simple INTERNET–--MODEM----WAN/LAN----SUBNET and it has been working just fine for an academic setting. Gateway is the PfSense box at LAN1 and IP 192.168.1.1 in my 192.168.1.0/24 subnet. Very simple setup.

What I'm trying to do now is access a few servers (but not internet) from a separate administrative SUBNET wich has its own internet, modem, Windows DS, gateway, firewall, DHCP, etc. So I've connected OPT1 with IP 192.168.2.5 to the administrative 192.168.2.0/24 subnet. The layout looks like:

Internet1----Modem----WAN1 (public ip, gateway) PfSense box LAN1 (192.168.1.1)----Academic subnet 192.168.1.x
OPT1 (192.168.2.5)-------------------¬
|
|
Internet2----Modem----WAN1 (public ip, gateway) Other server LAN1 (192.168.2.1)----Administrative subnet 192.168.2.x

On both LAN connections:

Automatic outbound NAT rule generation is enabled
Block private or bogon networks is disabled
IPv4 Upstream Gateway is off

General Setup

Only gateway is my WAN1 on the PfSense box
No static routing
Proxy filtering (SquidGuard) off

Firewall rules:

Pass, 192.168.1.1 interface, IPv4, any protocol, from 192.168.1.x net, to 192.168.2.x net, logging turned on.
Pass, 192.168.2.1 interface, IPv4, any protocol, from 192.168.2.x net, to 192.168.1.x net, logging turned on.

I can ping other 192.168.2.x subnet within PfSense>Diagnostics>Ping but not from Windows. Can't access the servers either.

What else do I need?

phil.davis · Feb 4, 2015, 2:32 PM

The devices in 192.168.2.0/24 subnet will have 192.168.2.1 as their default gateway. When they receive a packet from a 192.168.1.* device, they will reply to their default gateway 192.168.2.1 - that gateway knows nothing about the nearby 192.168.1.0/24 subnet - so it will send the packets out its WAN to be dropped as unroutable by the upstream ISP.
Devices in 192.168.2.0/24 subnet might have a firewall that is allowing the local subnet but blocking anything from outside the local subnet - Windows is a common offender.

Solve (2) by checking and disabling any firewalls

Solve (1) by:
a) (my preferred solution) Enable an OPT1 on Other Server (router - I guess it is also pfSense?). Connect the 2 OPT1 with a (crossover) netowkr cable. Give it some other subnet e.g. 192.168.3.1/24 and 192.168.3.2/24
Do not give either OPT1 interface a gateway. Do create a gateway on each pfSense pointing to the OPT1 address of the other.
Add a static route on each pfSense to route to the opposite LAN subnet via the gateway.
Add pass rule/s on each OPT1 to allow traffic coming from the opposite LAN subnet.
This way all the traffic is routed cleanly through the default gateway of the clients in each LAN. Clients do not need to know anything about the network topology.

b) Add a manual NAT rule for traffic from 192.168.1.0/24 going out pfSense1 OPT1. Then clients on 192.168.2.* will see the traffic as coming from 192.168.2.5 and can reply directly and correctly to that.
The downside of this is that clients on 192.168.2.* cannot determine the real source IP address of traffic coming from 192.168.1.*

ruben.rothermel · Feb 4, 2015, 6:10 PM

Thanks. I'll play around with it later…. but responding to your points:

1) Yes, 192.168.2.1 is the gateway for that subnet and it is not served by PfSense but other FreeBSD flavored setup which I might be able to reconfigure somewhat.
There's a IIS7 web server with 2 sites on 2 different ports on 192.168.2.10 to access by the other subnet... that's it. I'll try the manual NAT with that specific IPs since I don't have to modify hardware.

What happens with the automatic NATing? Do I leave it enabled and just add this single manual routing rule for 192.168.2.10 or do I have to go fully manual NATing?

2) Probably true and I would have to check on this because it is an administrative subnet with tighter controls, domain controller, VPN, VoIP, etc., etc.

Again, thanks for the answer. I've just updated to PfSense 2.2 with no issues.

phil.davis · Feb 4, 2015, 6:27 PM

Good to be on pfSense 2.2 - that has "hybrid outbound NAT". Switch to "hybrid outbound NAT", that retains all automatic outbound NAT rules and then you can add your own also.

If you just need to access 1 particular server, then the other possibility is to add a route on that server itself pointing 192.168.1.0/24 at 192.168.2.5 - then it will be able to reply to 192.168.1.0/24 as well as retain its default route to the rest of the internet.

ruben.rothermel · Feb 4, 2015, 10:12 PM

Now that I can sit and play with the new 2.2 release… I see the new options under Firewall, NAT, Outbound and...

1. Selected "Hybrid Outbound NAT rule generation (Automatic Outbound NAT + rules below)"
2. Added a manual mapping: on OPT1 interface, from 192.168.1.x, to 10.83.24.x, OPT1 address translation (where I see other interesting options)

From my browser (in 192.168.1.x) I can access the WHOLE 192.168.2.x subnet in OPT1! I can also ping everyting! Now I have to step back and allow just the web server on 192.168.1.10 and the specific ports for the web sites.

Nice, nice. I feel I'm getting somewhere now...

On another note and somewhat related... Would it be better to let this PfSense box have an extra OPT interface with another subnet sharing the PfSense WAN gateway or have a single higher level subnet? Something like 255.255.254.0 or even 255.255.0.0 I'm not that far from running out of available IP addresses.

I'm guessing that with gigabit ports it's a matter of CPU, memory and general throughput capabilities of my PfSense box as there will be a lot of traffic between the two school levels (sharing multimedia materials.... Moodle... etc.)

Internet–---WAN [PfSense Box] LAN (192.168.2.1)–---- (for the High School level, different classroom building but same library subnet 192.168.2.x)
OPT1 (192.168.3.1)----- (for the Junior College level stuff subnet 192.168.3.x)
OPT2 (192.168.1.5)------¬
|
|
Internet-----WAN [Other Box] LAN (192.168.1.1)–--- (Administrative subnet 192.168.1.x)

Tomorrow I'll define better manual NAT rules to open specific services and not the WHOLE subnet.

Thanks for your help

mikeisfly · Feb 4, 2015, 10:30 PM

I will give an OPT C that could work if you have access to both routers.

If you run a routing protocol like RIPv2 on both sides both the routers will exchange their routing tables and communication should happen on both sides. To run RIP in PfSense you need to install a package like routed. You will have to run it on the opposite end too. Once that is done like Phil.Davis has pointed out with the firewall rules you should be good to go. I prefer this way because as you make changes to your network you can advertise those routes by just adding them under the RIP menu under services and the other side will know about them. Static routes are nice but you can make mistakes as your network grows. Also like Phil pointed out NAT hides the true originator of a packet so it makes it harder to block specific clients.

P.S. Static routes add no over head to your router, while dynamic routing protocols add some overhead as they are constantly sending network topology updates (At least RIP does). Although I doubt you will see a CPU hit but just thought I through that out there.