Routing Between Separate PFSense Firewall Pairs
-
Hello everyone!
I have spent the last week setting up a new installation to replace a Cisco RV325. Things have been going great accept for one snag that I have been unable to overcome and I am sure it's something super stupid on my part.
I have a two paris of PFsense units setup with CARP + VLAN + LAGG hosting multiple VLANs behind each of them. Each PFsense pair is the router for all the networks behind them. There is a switch between the pairs but its only L2.
The first pair does only routing and firewalling for the internal networks while the second is supposed handle NAT and VPN duties.
There is a /24 network between PFsense pairs that is being used to route traffic between them (10.0.70.0/24) and each firewall pair has a CARP ip on that subnet (.2 (internal) and .1 (external)).
The default gateway of the internal PFsense pair is set to the transit CARP IP of the external PFsense pair and I have static routes setup on the external PFsense pair for all the internal networks that point to the transit CARP IP of the internal PFsense pair.
I can see traffic reaching the external PFsense CARP ip but it is not being forwarded and instead its getting grabbed by the default deny rule.
I have attempted to turn on the "Ignore firewall rules for traffic on the same interface" option but that hasn't helped either.
The only way I have gotten things to work between a host on an internal LAN and a host on a DMZ net (sitting behind the external PFsense pair) is to put a rule on the transit network that allows ALL to ALL, or IP of internal node allow to IP of DMZ Node.
This makes my control point the transit network and not the interfaces of the subnets themselves, which will be problematic as the rule set grows.
My first question is:
- Does routing happen before rule matching?
- How do I get this to work with writing rules on the subnet interfaces and not the transit interface?
Simple Diagram:
NodeA (10.0.10.6, GW 10.0.10.1) -> Internal_PF -> External_PF -> 10.0.90.0_Subnet (Directly Connected) -> NodeB (10.0.90.6, GW 10.0.90.1)
-
For incoming packet on any given interface it goes like this:
1. Address rewriting, rdr or nat rules.
2. Packet filtering by the filter rules. Rules can set route-to for the packets to take different route at 3.
3. Routing if the destination of the packet (after NAT mind you) is not a local address.Outgoing is in the same order for 1. and 2. but routing has already happened obviously.