Permitted traffic to LAN blocked silently
-
Respect your quote. But I'm not a network/firewalling newbie and I can assure you I've really tried everything reasonable before posting here. And also I red all the documentation I can find about pfSense and M0n0wall too. So, as a last chance, I've come here, registered and asked for help.
This because I don't like to let people lose their time with silly questions (I mean the classic RTFM :)The problem is stated in the first lines of my first post:
"I made some rules both in NAT and in WAN interface (automatically generated rules) to reach some of my LAN IPs but the packets can't pass pfSense"So, for example, if I open the 110/TCP port on the NAT and WAN to reach PC 192.168.1.9 (residing in my LAN network) where an OpenSSH server is listening on that port (OS: Ubuntu 8.04.2), no packet arrives to that machine (see tcpdump log) but the logs inside pfSense says "there's no problem, the packets passed" (and on my external machine trying to connect the connection hung at:
OpenSSH_4.7p1 Debian-8ubuntu1.2, OpenSSL 0.9.8g 19 Oct 2007
debug1: Reading configuration data /home/<myuser>/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to <external.dns.name>[<external.ip>] port 110.)
At the same time, if I create a rule on the DMZ interface to let one of my servers do the same thing (log in by SSH to the same LAN PC and to the same port), everything goes flawlessly and I'm able to login.
But, this is an example, I can do the same thing with any other port/protocol and the results are the same.
I tried to forward the 22 port to the LAN PC too (enabling SSH to bind also on that port), to exclude an ISP related problem (port blocking or something), because I'm able to log in by SSH to my DMZ machines on that port, so there's no ISP port blocking.
I tried to change ALL of my firewall hardware to exclude an hardware related problem.
I also verified the routing tables on the LAN PC to see if the pfSense's LAN interface was correctly set as the default gateway, and it is.The strange thing is that EVERYTHING else works fine with pfSense and my networks, so I'm really stuck :(
I'm now convinced that there should be a problem in some "internals" of the pf mechanism, maybe a missing rule or something that can cause this strange issue (maybe caused by some hard-reboot that I had with my old machine). Could a "pfctl -sa" be useful?
Please help :(</external.ip></external.dns.name></myuser>
-
I'm sorry if I can't buy pfSense support to solve this issue 'cause all my work is done for a non-profit organization where I provide all the hardware, the software and a lot of my time to administer things, for free. I'm a little bit tired to spend also money for that (I changed all the machines running there, recently, because were too old…)
-
Ok i see what the problem is now.
What i would do is put a hub between the pfSense and the server and listen with a 3rd computer with wireshark to see if the packet really leaves the pfSense.
(Since they accordingly to the log do get allowed by the firewallrule) -
So, that is the first thing I'll try to do tomorrow morning: but can I attach my own laptop to the pfSense interface and use wireshark on that without the need to buy an hub? There should be no problem at all, isn't it?
-
Sure that should work.
If your switch is able to mirror ports you could do that as well.
I suggested the hub just because you might have one lying around and it's easier than configuring a switch to mirror a port. -
Understood. Unfortunately I have no hub and the switch is a stupid-cheap accesspoint from netgear that has no capability to mirror a port. So I'll try with my laptop.
Thank you very much for your help :) -
tcpdump from the firewall would be more useful. Start from the inside since the traffic is getting logged as passed.
Can you telnet to port 110 on that host from the firewall?
It certainly isn't a bug. Config looks fine at a glance.
-
So, I've made all the tests I could.
1. Used the "Diagnostic–->Packet Capture" on pfSense pointing to the LAN interface and on the port SSH use: nothing logged
2. Attached my own laptop directly to the pfSense's LAN interface and tried an SSH session (from outside) with Wireshark sniffing: nothing logged
3. Telnetted/SSHed to the LAN machine from the pfSense's shell and both the tests went fine: can connect
4. Telnetted/SSHed to the LAN machine from the DMZ and both the tests went fine: can connectSo it's defitively something in the process of passing packets from the port-forwarding-NAT/WAN-rules to the LAN interface, 'cause logs on pfSense shows the rules (NAT+WAN) have matched properly but nothing exits from the LAN interface. I remember the same port-forward to my DMZ works flawlessly (despite the protocol/port I use).
And now what can I try? Any suggestion/request? :(
-
What a tcpdump on my Lan nic shows when i try from a outside connection
tcpdump -t -i vr0 port 3333
tcpdump: WARNING: vr0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vr0, link-type EN10MB (Ethernet), capture size 96 bytes
IP 0x4dd40e8e.<deleted>.42524 > 192.168.1.49.dec-notes: S 760313484:760313484(0) win 5840 <mss 6="" 4440200="" 1460,sackok,timestamp="" 0,nop,wscale="">what I could find in http://192.168.1.1/status.php#pfctl -sn
rdr on vlan0 inet proto tcp from any to <my wan="" ip="">port = dec-notes -> 192.168.1.49
#pfctl -sr
pass in quick on vlan0 reply-to (vlan0 <my wan="" gateway="">) inet proto tcp from <remote ip="">to 192.168.1.49 port = dec-notes flags S/SA keep state label "USER_RULE: NAT "
#pfctl -sa
rdr on vlan0 inet proto tcp from any to <my wan="" ip="">port = dec-notes -> 192.168.1.49
pass in quick on vlan0 reply-to (vlan0 <my wan="" gateway="">) inet proto tcp from <remote ip="">to 192.168.1.49 port = dec-notes flags S/SA keep state label "USER_RULE: NAT "
#pfctl -s rules -vv
@54 pass in quick on vlan0 reply-to (vlan0 <my wan="" gateway="">) inet proto tcp from <remote ip="">to 192.168.1.49 port = dec-notes flags S/SA keep state label #"USER_RULE: NAT "
rdr on vlan0 inet proto tcp from any to <my wan="" ip="">port = dec-notes -> 192.168.1.49
#cat /tmp/rules.debug
rdr on vlan0 proto tcp from any to <my wan="" ip="">port { 3333 } -> 192.168.1.49
pass in quick on $wan reply-to (vlan0 <my wan="" gateway="">) proto tcp from { <remote ip="">} to { 192.168.1.49 } port = 3333 keep state label "USER_RULE: NAT "
#pfctl -s nat -v
rdr on vlan0 inet proto tcp from any to <my wan="" ip="">port = dec-notes -> 192.168.1.49
[ Evaluations: 51 Packets: 49 Bytes: 7522 States: 0 ]
[ Inserted: uid 0 pid 48514 ]If it was my box i think my next move would be to boot from a live cd and keep it as close to default as possible and then make a portforward and run the tcpdump to see what happens.</my></remote ></my></my></my></remote ></my></remote ></my></my></remote ></my></my></mss></deleted>
-
As first: sorry for the late.
If it was my box i think my next move would be to boot from a live cd and keep it as close to default as possible and then make a portforward and run the tcpdump to see what happens.
Agreed, but I'm pretty sure the portforwardings were working in the initial install: I was thinking about some sort of "bug" (with an extended meaning, also some strange configuration interaction) for this reason. The next step could be to reinstall from scratch pfSense and to re-import the configuration, to see if anything changes. Obviously if you can't see anything strange in the output of the next commands or if you have some other ideas to try :)
Definitively is really a strange issue :(Attached there are the outputs of the commands you requested.
pfctl_sn_and_sr.txt
cat_tmp_rules_debug.txt
pfctl_sa.txt
pfctl_s_nat_v.txt -
Attached there are the outputs of the commands you requested.
And a post more, because of attachment size limit.
-
Do you have captive portal enabled on the interface that has a problem? Does anything change if you disable traffic shaping?
-
Do you have captive portal enabled on the interface that has a problem?
Yes I have it enabled but the PCs where I'd like to portforward a connection are excluded by meanings of MAC address ("Pass-through MAC").
Does anything change if you disable traffic shaping?
Tryed to, but anything changes :(
P.S.: When will v1.2.3 be released (more or less)? Maybe I can try to reformat the machine when it comes, just to avoid a new release upgrade if I do a reformat now.
-
I found something!! It's the CaptivePortal (bug or misconfiguration?)
- Brand new machine;
- complete reformat with v1.2.3;
- complete manual reconfiguration of pfSense: only the DHCP-server & CaptivePortal configurations have been imported into pfSense;
- there are 90 CaptivePortal users and 145 uniq MAC addresses in DHCP-server (I match an identity by means of a MAC_address+user/pass)
Please see attached files to see my CaptivePortal and DHCP-server configuration
Now some strange things occur…
- if I disable the any-->192.168.0.40 rule in "Allowed IP addresses" nothing works in the LAN (but I can understand this because that machine is the DNS server and on pfSense it is set as the only DNS server, passed to DHCP clients too), no access to CaptivePortal page nor internet access not anything;
- if I enable the previous rule everything works but I have the problem described in previous posts (no access to static clients by internet and no NAT-reflecting working for any LAN machine) ---> !!note that the 2 static machines in LAN have the appropriate entry in "Pass-through MAC"!!
- if I add 2 rules to "Allowed IP addresses" such as: 192.168.1.9-->any (static-client-->any) everything works as expected (external access to that machine, NAT-reflecting working, etc.)
- if I completely disable the CaptivePortal the result is the same as previous point: everything works but for all the LAN's machines
- Note: configuration and issues are the same of all my previous posts, nothing has changed (I reconfigured manually the whole pfSense the same way it was before the reformat)
So: IMHO definitively is a CaptivePortal issue.
Now I don't know if it's a bug or a misconfiguration by my side but, as far as I can remember, when there were a few users registered to CaptivePortal everything was working as expected: maybe an issue related to BIG number of users in the CaptivePortal?
I can provide complete configuration-backup file if you need it, but not on the forum (because of sensitive data in it), my e-mail is: fr3ddie at fr3ddie dot it
Please answer, I'm sure that if this is not a misconfiguration issue you'll like to fix a bug like this before 1.2.3 release.
-
More attachments
-
Is there anybody? :'(