Traffic getting blocked from remote subnet via OPT interface
-
First let me say that I love PFSense. I've swapped out my old FreeBSD firewall (using ipfw) with PFSense mostly to load balance a second DSL line. Everything was working fine, but then I added another OPT interface in, which connects to a remote site via a private wireless network (canopy equipment). I can access the remote network, but any connection initiated from their side is getting dropped. I had this running on a fairly simple ipfw setup on the old box, so I suspect this problem is due to my lack of pf mojo.
Layout: (with trivial obfuscation)
My LAN: 10.10.10.0/24
Wireless: 192.168.100.0/24
Remote LAN: 10.40.0.0/16fxp0=LAN 10.10.10.1 fxp1=WAN fxp2=OPT1(WAN2) fxp3=OPT2 192.168.100.10
I have a router on the wireless network at 192.168.100.20 with another interface on the remote LAN that has a static route back to 10.10.10.0/24 via 192.168.100.10.
My pfSense has a static route to 10.40.0.0/16 via 192.168.100.20, I have tried associating the route with both the LAN and the OPT2 interfaces, with the same results. Also, the advanced, static route filtering box is checked to bypass rules for traffic on the same interface.
Seeing as how I can access the remote network, but they cannot access me, it seems to be an issue with the incoming traffic via OPT2(fxp3). If I open a shell from pfSense and try a pfctl -si, I see interface statistics for fxp3, IPv4
Packets In
Passed 29946 0
Blocked 8677 0
Packets Out
Passed 28490 2
Blocked 4 0
So, it looks like something IS getting blocked coming in.My rules are set up basically like this:
LAN:
pass any from lan net to wireless net
pass any from lan net to remote net
default lan to any (goes to load balancer on wans)OPT2
pass any from wireless net to lan net
pass any from remote net to lan netUnder NAT I have:
OPT2 no nat from lan net to wireless net
OPT2 no nat from lan net to remote net
WAN nat from lan net at any
OPT1 nat from lan net to any
OPT2 nat from lan net to anyAny help as to what I'm doing wrong would be appreciated. Oh yeah, I'm running 1.0.1
Thanks -
What do you need the no nat rules for? Those are not needed.
-
The Wireless can be used for Internet Access, the provider has a firewall at 192.168.100.1 running NAT. I have 192.168.100.1 set as the gateway for the OPT2 interface. It's handy to be able to force a user out that interface is they have an ftp site that won't work on the load balancer (WAN+OPT1).
Yes, that's a nasty double-NAT hack, but it works. The providers firewall does not have knowledge of my subnet… -
There is a workaround rule that you have to add for ftp when using loadbalancing. Search the forum. Besides the other problem I would start with a simple setup and test step by step when adding additional configurations and see where it breaks.
-
I was hoping something in the setup would send up a red flag, but I guess I will have to setup a test box and test with the wireless connected to it. Everything but the return traffic from the remote subnet works fine on the production box… Oh, I did do the ftp fix and haven't had any complaints since. I just put in the nat so I could use it if needed. I did try removing all the nat rules for OPT2 and I still couldn't ping the lan subnet from the remote subnet.
Anyway, I will get a test box running in a few days and see what I can figure out. The config was started on a 1.0 test box with different NICs (sis) which was upgraded to 1.0.1, which was restored to the production server with fxp NICs, so maybe there is something screwy in the config left over from testing... I'll start clean on the test box and see how it goes. -
Getting frustrated. So far, all I've figured out is that it works fine when I go to advanced and disable the filter altogether. That's not really a workable solution on the production box. It won't even work when I have allow any proto, any source, any dest rules on both OPT2 and LAN. Is there any way to do a quick hack to put an allow in from fxp3 that would hit before whatever is killing the traffic gets triggered?
-
Here's one more (interesting?) note- from the remote subnet I can ping the fxp3 address, but not the LAN address… Anyway, I've found that copying /tmp/rules.debug to a file, adding 'pass out quick on fxp3 all' before the final 'block all just to be sure' lines and loading from shell (pfctl -f) fixes it. Is there a less ugly way to add this rule? One that would survive reloading the ruleset/rebooting the box?
-
Finally got this working. I started clean and didn't import my nat or ruleset then set OPT2 with
pass any from 192.168.100.0/24 to 10.10.10.0/24
pass any from 10.40.0.0/16 to 10.10.10.0/24
pass any from 10.10.10.0/24 to 192.168.100.0/24
pass any from 10.10.10.0/24 to 10.40.0.0/16I didn't add the gateway to OPT2 while testing, and now I get an error when I try to add it in, I guess because of the balancer. Eventually I'll try deleting the balancer, adding the gateway, then re-creating, but for now I don't care because everything is working that I need.
