Help routing between multiple LANs

70tas · Mar 2, 2016, 3:36 AM

I have pfSense running with four NIC's:

1. em19 is the WAN being NAT'ed via the firewall and Squid assigned by provider DHCP
2. em2 is my first LAN, 192.168.2.0/24, GW 192.168.2.99
3. em9 is my second LAN, 192,168.9.0/24, GW 192.168.9.99
4. em12 is my third LAN, 192.168.12.0/24, GW 192.168.12.99

ok, ok, the names don't matter, WAN/LAN/OPT1/OPT2

I want to continue going through the firewall for all traffic going to the Internet via the WAN port. Everything I've read says pfSense should handle the internal routing, after all, it is a router.

Most of my systems or on the em9 NIC/subnet, some on em12. I can access the internet fine. I can ping 192.168.2.99 and 192.168.12.99. But that is as far as pfSense will go. I cannot ping or access any servers on the routed sub-nets.

I've tried following all kinds of examples, I've setup gateways and static routes, groups, rules for any to any but nothing seems to work.

Does anyone have any ideas?

Thank you
Tas

But I can't route between em2, em9 or em12 unless I add NIC's on the systems that need to route. If, for example, I set up a system on the em9 with one NIC on that subnet, which whould

marvosa · Mar 2, 2016, 4:05 AM

PFsense adds routes for all locally connected interfaces, so the routing is already there. Remove any static routes or groups for these interfaces. What PFsense does not do is add firewall rules to the OPT interfaces by default.

After you remove the static routes, etc…. add any/any firewall rules to all your LAN interfaces and everything should start working.

But I can't route between em2, em9 or em12 unless I add NIC's on the systems that need to route. If, for example, I set up a system on the em9 with one NIC on that subnet, which whould

I'm not sure what you were trying to say here… "unless I add NIC's..." don't ALL your systems have NIC's? Please rephrase and clarify your statement.

70tas · Mar 3, 2016, 12:04 AM

To clarify, I'd like to have one NIC per system, based on it's subnet. But in order to get routing between workstations, I have to add a NIC on each system to its native subnet and any subnet I want it to route. The workstation is then handling the routing on the two NIC's, i.e. em9, and em2.

I've tried your suggestions, however, they don't appear to be making a difference.

70tas · Mar 3, 2016, 3:51 AM

Ok, I've scoured through the entire configuration and even removed routed even though it was disabled.

For each of the three LAN interfaces I created a rule as:

EM2 has the anti-lockout rule
and
EM2 IPV4/TCP,,,,,*
EM9 IPV4/TCP,,,,,*
EM12 IPV4/TCP,,,,,*

Still nothing works. I can ping each LAN's own GW, and the GW of any of the other LAN's. But I cannot route or ping to any other IP's on the non-owned LAN's.

Stumped… Anything else I can check?

70tas · Mar 3, 2016, 4:05 AM

Ok, I think I have it.

First, I was pinging a Win box whose firewall did not allow ICMP. However, the solution was to add the following rule to each of my internal NICs:

EM2,IP4*,,,This firewall,,,None
EM9,IP4*,,,This firewall,,,None
EM12,IP4*,,,This firewall,,,None

"This firewall" is assigned to the Destination field.

There may be an easier way of doing this, but I am a noobie… Thank you for the help.

Tas

marvosa · Mar 3, 2016, 3:15 PM

If the "this firewall" rule works over an any/any rule then there's something else more fundamental configured incorrectly.

What IP do you have configured for each interface? Your DHCP server should be handing out the interface IP as the default gateway for each segment… is that happening? Where is your DHCP server (PFsense, Windows, Linux, Infoblox, etc)?

After re-reading this:

1. em19 is the WAN being NAT'ed via the firewall and Squid assigned by provider DHCP
2. em2 is my first LAN, 192.168.2.0/24, GW 192.168.2.99
3. em9 is my second LAN, 192,168.9.0/24, GW 192.168.9.99
4. em12 is my third LAN, 192.168.12.0/24, GW 192.168.12.99

None of your LAN interfaces should have a gateway unless the traffic is being forwarded to another router.

Provided your clients are receiving the proper default gateway and your LAN interfaces do indeed have gateways configured:

Remove the gateways from your LAN interfaces
Configure an any/any rule on each LAN interface, i.e.:
Disable the software firewall for any device you're trying to ping until successful communication is established

Then re-test.

70tas · Mar 3, 2016, 6:20 PM

Here is how I'm setup. BTW, having issues with CIFS… ;D

DHCP server is configured for each LAN as:

em2 - pfSense IP 192.168.2.99, DNS 192.168.2.99, GW 192.168.2.99
em9 - pfSense IP 192.168.9.99, DNS 192.168.9.99, GW 192.168.9.99
em12 - pfSense IP 192.168.9.99, DNS 192.168.12.2, GW 192.168.12.99 (This is an AD segment so I use DNS within AD, but DHCP from pfSense)

NICs are configured with NONE as the upstream gateway

So I have two rules for em2/LAN, the anti-lockout, and the IPV4*, sourced from em2.
em9 and em12 only have one rule, the IPV4*, sourced from em9 and em12.

I disabled Squid, and ClamAV, and Darkstat.

I then checked it out, it now seems to be working and routing. I've removed all but one NIC from my host, the em2 is enabled, and em9/em12 disabled. I can now route. I went to a host on em9, and I can route back to 192.168.2.0. H U R R A Y ! ! !

So I re-enabled Squid and ClamAV. All seems to be in order. I'm not sure why, but it appears that Squid/ClamAV may have played a role, but for the life of me, I don't know how. I will eventually re-enable Darkstat and we'll see what happens.

But now, I'm as happy as a pig with lipstick...

Thanks a lot for your help. I think your suggestion of bringing the config back to as close as zero first, helped.