Permitted traffic to LAN blocked silently



  • Hi all,
    this is the problem: I made some rules both in NAT and in WAN interface (automatically generated rules) to reach some of my LAN IPs but the packets can't pass pfSense.

    The strange thing is that the same rules from NAT to DMZ works fine and also from DMZ to LAN, I really can't understand what's goin' on… I also completely changed my hardware to understand if it could be an hardware related problem but it obviously wasn't.
    Moreover, if I activate logging on that rules I find that the packets passed correctly but, on the interface of the machine that should receive those packets, anything arrives (as reported by my sniffing on that interface).
    Between pfSense's LAN interface and the LAN's destination machine there's nothing else but a simple (not managed) switch and, anyway as before said, the same rules between DMZ-->LAN and between NAT-->DMZ work flawlessly.

    I'm really stuck at this point, can anyone help resolving the trouble? I've also attached some screenshots of my rules/config and network topology, and I can attach anything else needed. Please help :(












    [NET - it is a DIA file.txt](/public/imported_attachments/1/NET - it is a DIA file.txt)



  • So I have to suppose I have found a bug in pfSense?



  • @http://www.catb.org/~esr/faqs/smart-questions.html:

    Don't claim that you have found a bug

    When you are having problems with a piece of software, don't claim you have found a bug unless you are very, very sure of your ground. Hint: unless you can provide a source-code patch that fixes the problem, or a regression test against a previous version that demonstrates incorrect behavior, you are probably not sure enough. This applies to webpages and documentation, too; if you have found a documentation “bug”, you should supply replacement text and which pages it should go on.

    Remember, there are many other users that are not experiencing your problem. Otherwise you would have learned about it while reading the documentation and searching the Web (you did do that before complaining, didn't you?). This means that very probably it is you who are doing something wrong, not the software.

    The people who wrote the software work very hard to make it work as well as possible. If you claim you have found a bug, you'll be impugning their competence, which may offend some of them even if you are correct. It's especially undiplomatic to yell “bug” in the Subject line.

    When asking your question, it is best to write as though you assume you are doing something wrong, even if you are privately pretty sure you have found an actual bug. If there really is a bug, you will hear about it in the answer. Play it so the maintainers will want to apologize to you if the bug is real, rather than so that you will owe them an apology if you have messed up.

    ;)

    Could be a bit more specific what's "not working".
    Your error descriptions doesnt really say what you're trying to do or what the actual problem is.
    From what i can see in the screenshots everything is working as it is configured.



  • Respect your quote. But I'm not a network/firewalling newbie and I can assure you I've really tried everything reasonable before posting here. And also I red all the documentation I can find about pfSense and M0n0wall too. So, as a last chance, I've come here, registered and asked for help.
    This because I don't like to let people lose their time with silly questions (I mean the classic RTFM :)

    The problem is stated in the first lines of my first post:
    "I made some rules both in NAT and in WAN interface (automatically generated rules) to reach some of my LAN IPs but the packets can't pass pfSense"

    So, for example, if I open the 110/TCP port on the NAT and WAN to reach PC 192.168.1.9 (residing in my LAN network) where an OpenSSH server is listening on that port (OS: Ubuntu 8.04.2), no packet arrives to that machine (see tcpdump log) but the logs inside pfSense says "there's no problem, the packets passed" (and on my external machine trying to connect the connection hung at:

    OpenSSH_4.7p1 Debian-8ubuntu1.2, OpenSSL 0.9.8g 19 Oct 2007
    debug1: Reading configuration data /home/<myuser>/.ssh/config
    debug1: Reading configuration data /etc/ssh/ssh_config
    debug1: Applying options for *
    debug2: ssh_connect: needpriv 0
    debug1: Connecting to <external.dns.name>[<external.ip>] port 110.

    )

    At the same time, if I create a rule on the DMZ interface to let one of my servers do the same thing (log in by SSH to the same LAN PC and to the same port), everything goes flawlessly and I'm able to login.

    But, this is an example, I can do the same thing with any other port/protocol and the results are the same.
    I tried to forward the 22 port to the LAN PC too (enabling SSH to bind also on that port), to exclude an ISP related problem (port blocking or something), because I'm able to log in by SSH to my DMZ machines on that port, so there's no ISP port blocking.
    I tried to change ALL of my firewall hardware to exclude an hardware related problem.
    I also verified the routing tables on the LAN PC to see if the pfSense's LAN interface was correctly set as the default gateway, and it is.

    The strange thing is that EVERYTHING else works fine with pfSense and my networks, so I'm really stuck :(

    I'm now convinced that there should be a problem in some "internals" of the pf mechanism, maybe a missing rule or something that can cause this strange issue (maybe caused by some hard-reboot that I had with my old machine). Could a "pfctl -sa" be useful?

    Please help :(</external.ip></external.dns.name></myuser>



  • I'm sorry if I can't buy pfSense support to solve this issue 'cause all my work is done for a non-profit organization where I provide all the hardware, the software and a lot of my time to administer things, for free. I'm a little bit tired to spend also money for that (I changed all the machines running there, recently, because were too old…)



  • Ok i see what the problem is now.

    What i would do is put a hub between the pfSense and the server and listen with a 3rd computer with wireshark to see if the packet really leaves the pfSense.
    (Since they accordingly to the log do get allowed by the firewallrule)



  • So, that is the first thing I'll try to do tomorrow morning: but can I attach my own laptop to the pfSense interface and use wireshark on that without the need to buy an hub? There should be no problem at all, isn't it?



  • Sure that should work.
    If your switch is able to mirror ports you could do that as well.
    I suggested the hub just because you might have one lying around and it's easier than configuring a switch to mirror a port.



  • Understood. Unfortunately I have no hub and the switch is a stupid-cheap accesspoint from netgear that has no capability to mirror a port. So I'll try with my laptop.
    Thank you very much for your help :)



  • tcpdump from the firewall would be more useful. Start from the inside since the traffic is getting logged as passed.

    Can you telnet to port 110 on that host from the firewall?

    It certainly isn't a bug. Config looks fine at a glance.



  • So, I've made all the tests I could.
    1. Used the "Diagnostic–->Packet Capture" on pfSense pointing to the LAN interface and on the port SSH use: nothing logged
    2. Attached my own laptop directly to the pfSense's LAN interface and tried an SSH session (from outside) with Wireshark sniffing: nothing logged
    3. Telnetted/SSHed to the LAN machine from the pfSense's shell and both the tests went fine: can connect
    4. Telnetted/SSHed to the LAN machine from the DMZ and both the tests went fine: can connect

    So it's defitively something in the process of passing packets from the port-forwarding-NAT/WAN-rules to the LAN interface, 'cause logs on pfSense shows the rules (NAT+WAN) have matched properly but nothing exits from the LAN interface. I remember the same port-forward to my DMZ works flawlessly (despite the protocol/port I use).

    And now what can I try? Any suggestion/request? :(



  • What a tcpdump on my Lan nic shows when i try from a outside connection

    tcpdump -t -i vr0 port 3333

    tcpdump: WARNING: vr0: no IPv4 address assigned
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on vr0, link-type EN10MB (Ethernet), capture size 96 bytes
    IP 0x4dd40e8e.<deleted>.42524 > 192.168.1.49.dec-notes: S 760313484:760313484(0) win 5840 <mss 6="" 4440200="" 1460,sackok,timestamp="" 0,nop,wscale="">what I could find in http://192.168.1.1/status.php

    #pfctl -sn
    rdr on vlan0 inet proto tcp from any to <my wan="" ip="">port = dec-notes -> 192.168.1.49
    #pfctl -sr
    pass in quick on vlan0 reply-to (vlan0 <my wan="" gateway="">) inet proto tcp from <remote  ip="">to 192.168.1.49 port = dec-notes flags S/SA keep state label "USER_RULE: NAT "
    #pfctl -sa
    rdr on vlan0 inet proto tcp from any to <my wan="" ip="">port = dec-notes -> 192.168.1.49
    pass in quick on vlan0 reply-to (vlan0 <my wan="" gateway="">) inet proto tcp from <remote  ip="">to 192.168.1.49 port = dec-notes flags S/SA keep state label "USER_RULE: NAT "
    #pfctl -s rules -vv
    @54 pass in quick on vlan0 reply-to (vlan0 <my wan="" gateway="">) inet proto tcp from <remote  ip="">to 192.168.1.49 port = dec-notes flags S/SA keep state label #"USER_RULE: NAT "
    rdr on vlan0 inet proto tcp from any to <my wan="" ip="">port = dec-notes -> 192.168.1.49
    #cat /tmp/rules.debug
    rdr on vlan0 proto tcp from any to <my wan="" ip="">port { 3333 } -> 192.168.1.49
    pass in quick on $wan reply-to (vlan0 <my wan="" gateway="">) proto tcp from {  <remote  ip="">} to {  192.168.1.49 } port = 3333 keep state  label "USER_RULE: NAT "
    #pfctl -s nat -v
    rdr on vlan0 inet proto tcp from any to <my wan="" ip="">port = dec-notes -> 192.168.1.49
      [ Evaluations: 51        Packets: 49        Bytes: 7522        States: 0    ]
      [ Inserted: uid 0 pid 48514 ]

    If it was my box i think my next move would be to boot from a live cd and keep it as close to default as possible and then make a portforward and run the tcpdump to see what happens.</my></remote ></my></my></my></remote ></my></remote ></my></my></remote ></my></my></mss></deleted>



  • As first: sorry for the late.

    If it was my box i think my next move would be to boot from a live cd and keep it as close to default as possible and then make a portforward and run the tcpdump to see what happens.

    Agreed, but I'm pretty sure the portforwardings were working in the initial install: I was thinking about some sort of "bug" (with an extended meaning, also some strange configuration interaction) for this reason. The next step could be to reinstall from scratch pfSense and to re-import the configuration, to see if anything changes. Obviously if you can't see anything strange in the output of the next commands or if you have some other ideas to try :)
    Definitively is really a strange issue :(

    Attached there are the outputs of the commands you requested.

    pfctl_sn_and_sr.txt
    cat_tmp_rules_debug.txt
    pfctl_sa.txt
    pfctl_s_nat_v.txt



  • @fr3ddie:

    Attached there are the outputs of the commands you requested.

    And a post more, because of attachment size limit.

    pfctl_s_rules_vv.txt



  • Do you have captive portal enabled on the interface that has a problem?  Does anything change if you disable traffic shaping?



  • Do you have captive portal enabled on the interface that has a problem?

    Yes I have it enabled but the PCs where I'd like to portforward a connection are excluded by meanings of MAC address ("Pass-through MAC").

    Does anything change if you disable traffic shaping?

    Tryed to, but anything changes :(

    P.S.: When will v1.2.3 be released (more or less)? Maybe I can try to reformat the machine when it comes, just to avoid a new release upgrade if I do a reformat now.



  • I found something!! It's the CaptivePortal (bug or misconfiguration?)

    • Brand new machine;
    • complete reformat with v1.2.3;
    • complete manual reconfiguration of pfSense: only the DHCP-server & CaptivePortal configurations have been imported into pfSense;
    • there are 90 CaptivePortal users and 145 uniq MAC addresses in DHCP-server (I match an identity by means of a MAC_address+user/pass)

    Please see attached files to see my CaptivePortal and DHCP-server configuration

    Now some strange things occur…

    • if I disable the any-->192.168.0.40 rule in "Allowed IP addresses" nothing works in the LAN (but I can understand this because that machine is the DNS server and on pfSense it is set as the only DNS server, passed to DHCP clients too), no access to CaptivePortal page nor internet access not anything;
    • if I enable the previous rule everything works but I have the problem described in previous posts (no access to static clients by internet and no NAT-reflecting working for any LAN machine) ---> !!note that the 2 static machines in LAN have the appropriate entry in "Pass-through MAC"!!
    • if I add 2 rules to "Allowed IP addresses" such as: 192.168.1.9-->any (static-client-->any) everything works as expected (external access to that machine, NAT-reflecting working, etc.)
    • if I completely disable the CaptivePortal the result is the same as previous point: everything works but for all the LAN's machines
    • Note: configuration and issues are the same of all my previous posts, nothing has changed (I reconfigured manually the whole pfSense the same way it was before the reformat)

    So: IMHO definitively is a CaptivePortal issue.

    Now I don't know if it's a bug or a misconfiguration by my side but, as far as I can remember, when there were a few users registered to CaptivePortal everything was working as expected: maybe an issue related to BIG number of users in the CaptivePortal?

    I can provide complete configuration-backup file if you need it, but not on the forum (because of sensitive data in it), my e-mail is: fr3ddie at fr3ddie dot it

    Please answer, I'm sure that if this is not a misconfiguration issue you'll like to fix a bug like this before 1.2.3 release.












  • More attachments










  • Is there anybody?  :'(


Log in to reply