PfSense blocking traffic from secondary LAN subnet


  • Hi,

    I feel a bit ashamed asking that "simple" question, but I've got an strange issue here. I've read other related posts and compared the solutions given with my settings with no luck.

    I've got two different IP subnets on the LAN side. 10.16.0.0/16 and 192.168.1.0/24.
    Therefore, I set up pfSense with the LAN address 10.16.53.1 and added 192.168.1.111 as virtual IP on the LAN interface.
    I've added a rule to let pass all traffic from 192.168.1.0 by copying the default LAN rule and I've checked that an outbound NAT rule was added for that subnet.

    Nevertheless, when I check the log, I get blocked packets (to the WAN) from the server 192.168.1.131 (the only machine in the subnet at the moment) with the following text (after clicking on the red cross) :
    @5(1000000103) block drop in log inet all label "Default deny rule IPv4"

    I feel that I'm missing something stupid here…

    Can you help ?

    Thanks in advance,

    Thilroy


  • If the default deny rule is applied to a packet no other rule matches to it.

    So obviously the rule for the 192.168.1.0 subnet doesn't match to this packet. Check your rule. Is it an any:any to any:any? Check the mask.

    If you can't resolve, post the hole ruleset, outbound NAT an block log.

  • LAYER 8 Netgate

    I've got two different IP subnets on the LAN side. 10.16.0.0/16 and 192.168.1.0/24

    Why?


  • I checked the masks already but no, they match. The server is 192.168.1.131/24 too, same for the rule.
    As for the "why", well I've no choice : there is a subcontractor on our LAN and he asked to use a different subnet than ours. His server must be able to reach the outside world to do backups.

    Here's the LAN interface rule. Quite basic as you can see :

    IPv4 * LAN net                 * * * * none   Default allow LAN to any rule 
    IPv4 * 192.168.1.0/24 * * * * none   subnet jonestel

    Here's the FW log, filtered to source address 192.168.1.13 :

    x Feb 26 12:42:31 LAN 192.168.1.131:8888        94.23.250.17:52703 TCP:SA
    x Feb 26 12:42:16 LAN 192.168.1.131:8622 94.23.250.17:40606 TCP:SA

    I'm logging the rule 192.168.1.0 and basically, everything coming from the server is blocked. Nothing passes.

    Here's the outbound NAT setting :

    WAN  192.168.1.0/24 * * 500 WAN address * YES
    WAN  192.168.1.0/24 * * * WAN address * NO  
    WAN  127.0.0.0/8         * * 500 WAN address * YES  
    WAN  127.0.0.0/8         * * * WAN address * NO  
    WAN  10.16.0.0/16         * * 500 WAN address * YES
    WAN  10.16.0.0/16         * * * WAN address * NO

    A bit messy, but quite logical. (everything was auto-generated).

    Meanwhile, I'm looking for more info regarding what you are saying about the "no match" in the system logs….
    No luck until then, your help will be appreciated

    Thanks in advance,

    Thilroy


  • Have you set an "IPv4 Upstream Gateway" on LAN interface?

    A clean solution would be to use VLANs for the 2 subnets, however, you will need a VLAN capable switch.


  • Well, for internal reasons, I'd rather prefer not to have to implement VLANs.

    As for the LAN gateway, no I don't have one set up. Should I ? I'm failing to see where it should lead to… pFsense already knows its way apparently, because I can ping 192.168.1.131 from our LAN in 10.16.0.0 AND of course there is internet access from our LAN.

    So : route between 192.168.1.0 and 10.16.0.0 : OK
        : route from 10.16.0.0 to WAN : OK

    ...and btw, I've added a rule for allowing our subcontractor to reach his server from the internet (on ports 8888 an 8622), and this passes through the FW without any problems.

    thus : route from WAN to both 10.16.0.0 and 192.168.1.0 : OK

    Is this some kind of limitation ???

    I really am at lost here.


  • …some more info : I tried to ping the server 192.168.1.131 from different interfaces of the FW :

    from the primary LAN intf :

    PING 192.168.1.131 (192.168.1.131) from 10.16.53.1: 56 data bytes
    64 bytes from 192.168.1.131: icmp_seq=0 ttl=64 time=0.288 ms
    64 bytes from 192.168.1.131: icmp_seq=1 ttl=64 time=0.261 ms
    64 bytes from 192.168.1.131: icmp_seq=2 ttl=64 time=0.239 ms

    from the WAN intf :

    PING 192.168.1.131 (192.168.1.131) from 172.16.53.3: 56 data bytes
    36 bytes from 10.254.32.141: Communication prohibited by filter
    Vr HL TOS  Len  ID Flg  off TTL Pro  cks      Src      Dst
    4  5  00 0054 b7d3  0 0000  3e  01 2197 172.16.53.3  192.168.1.131

    Thus indeed, something is wrong there… I don't even know where this source address 10.254.32.141 comes from !!! Is it some automatic internal address of pf Sense ?

  • LAYER 8 Netgate

    @Thilroy:

    Well, for internal reasons, I'd rather prefer not to have to implement VLANs.

    As for the LAN gateway, no I don't have one set up. Should I ? I'm failing to see where it should lead to… pFsense already knows its way apparently, because I can ping 192.168.1.131 from our LAN in 10.16.0.0 AND of course there is internet access from our LAN.

    So : route between 192.168.1.0 and 10.16.0.0 : OK
        : route from 10.16.0.0 to WAN : OK

    ...and btw, I've added a rule for allowing our subcontractor to reach his server from the internet (on ports 8888 an 8622), and this passes through the FW without any problems.

    thus : route from WAN to both 10.16.0.0 and 192.168.1.0 : OK

    Is this some kind of limitation ???

    I really am at lost here.

    Yeah. The limitation is running "multiple subnets" on one segment is no solution at all.  Do it right or don't do it at all.  Any illusion of security is just that - an illusion.


  • Well, I must say that I do not agree with that. Multiple subnets on a flat LAN, even if not the best practice, is fairly common. I'm failing to see the security issue there.

    Besides, here is a quote that may be of interest too :
    "IP aliases work just like any other IP address on an interface, such as the actual interface IP address (…) They can also be used to handle multiple subnets on the same interface."
    –- pfSense : The Definitive Guide Version 2.1, page 168.

    Thus in short, what I am trying to do is planned by the pfSense team, or I am mistaken.

    Any clever help is still heartly welcome.

    Thanks in advance,

    Thilroy

  • LAYER 8 Netgate

    What do you seek to gain?  Anyone can just change their IP to the other "network" any time they want.

    Multiple subnets on a LAN can be useful in temporary cases like renumbering, or other situations where you have no choice, but to design a network around it is just, well, stupid and pointless.

    You would be better off putting them on one subnet and keeping a certain group in a range that can be covered with a CIDR, like a /28, so you can make rules accordingly.

    You cannot firewall between two subnets like you can between VLANs.


  • @Derelict:

    Multiple subnets on a LAN can be useful in temporary cases like renumbering, or other situations where you have no choice, but to design a network around it is just, well, stupid and pointless.

    Derelict, you are totally right. I wouldn't never design a network like that either. But this is indeed a situation where I have no choice.
    Besides, I DON'T CARE of any security issues on the LAN. The role of the FW on the LAN is limited to routing, I don't want more, but that's working. It's just that this subcontractor doesn't want to change his server's IP address and I don't want to judge why. Perhaps he will in the future, but not now.

    Technically speaking, what I'm trying to implement is not rocket science, is it ? And it's a case planned in pfSense documentation like I showed in previous message.

    So why is it not working ?

    Can someone answer, please ????

    Thanks in advance,

    Thilroy


  • Something else :

    Between other tests, I've tried to reorder the LAN rules and I've put the rule for the server before the default LAN rule.
    (I also restricted the rule to server only and used another gateway, we've got two WAN routers, but that is not relevant)

    IPv4 *     192.168.1.131         * * * GW_BELGACOM none   subnet jonestel 
    IPv4 *     LAN net                 * * * *                              none   Default allow LAN to any rule

    Then, all of a sudden, I've got packets from the server that are allowed to pass BUT ONLY DNS packets (port 53). The other stay blocked.
    (V: pass, X: blocked)

    V Mar 2 13:45:40    LAN 192.168.1.131:59697    195.238.2.21:53 UDP
    X Mar 2 13:45:33 LAN 192.168.1.131:8888      94.23.250.17:59729 TCP:SA
    V Mar 2 13:45:30 LAN 192.168.1.131:54599    195.238.2.21:53 UDP
    X Mar 2 13:45:19 LAN 192.168.1.131:8622      94.23.250.17:47632 TCP:SA

    Still no clue ?

    Thanks in advance,

    Thierry


  • …in fact I guess the dns packets were already passing, but that I wasn't logging them. But why ?


  • The packets which are blocked are SYN/ACK packets. That is the server response to a service request. The reason for that would be that pfSense have no correct state for this connection. But why that?
    The two routers / gateways may be a possible reason and are relevant for that anyway. If that is the cause you can try to check "Static route filtering" in System > Advanced > Firewall/NAT to prevent pfSense to check these packets.


  • Nope. I had that checked sooner after having read some posts on the same subject.
    I shouldn't apply here, since it normally applies to traffic from and to the same interface, which is not the case here.
    I nevertheless tried it, with no success.

    As for the two gateways, the traffic from the subcontractor to his server comes from our GW2, all the other traffic uses the default route through GW1.

    At first, the server's replies were going (well…should have been going...) through GW1, as it was default. But then I indeed feared SYNC problems with the server's replies going out through another GW (although normally TCP is there for something, isn't it ? I mean, that is NOT a problem theoretically) and therefore I set the rule for 192.168.1.131 to go through GW2.

    Briefly said : This hasn't changed anything, and I'm still in the blue...

    Help/suggestions... pleaaaase ?

    Thilroy


  • Update.

    @Thilroy:

    …some more info : I tried to ping the server 192.168.1.131 from different interfaces of the FW :

    from the primary LAN intf :

    PING 192.168.1.131 (192.168.1.131) from 10.16.53.1: 56 data bytes
    64 bytes from 192.168.1.131: icmp_seq=0 ttl=64 time=0.288 ms
    64 bytes from 192.168.1.131: icmp_seq=1 ttl=64 time=0.261 ms
    64 bytes from 192.168.1.131: icmp_seq=2 ttl=64 time=0.239 ms

    from the WAN intf :

    PING 192.168.1.131 (192.168.1.131) from 172.16.53.3: 56 data bytes
    36 bytes from 10.254.32.141: Communication prohibited by filter
    Vr HL TOS  Len  ID Flg  off TTL Pro  cks      Src      Dst
    4  5  00 0054 b7d3  0 0000  3e  01 2197 172.16.53.3  192.168.1.131

    Thus indeed, something is wrong there… I don't even know where this source address 10.254.32.141 comes from !!! Is it some automatic internal address of pf Sense ?

    Well, a trace route to that address revealed that it was in our internet provider's network. Thus it means that when pinging from the WAN address, the FW uses its default gateway for 192.168.1.0 which is NOT correct. I tried to add a static route but no luck.
    There definitely is a routing problem here. Probably the VIP is not added to the routing table of the FW somehow…

    ideas ?

    Thilroy


  • and now, the states table for the server :

    WAN tcp 192.168.1.131:8622 <- 94.23.250.17:35042 SYN_SENT:ESTABLISHED
    LAN         tcp 94.23.250.17:35042 -> 192.168.1.131:8622 ESTABLISHED:SYN_SENT

    I do not understand the second line. Why LAN intf and arrow indicating connection from a wan address to the server's address ?

    But indeed, arrow from server to wan is nowhere to be seen