Pfsense, two subnets, jumboframes etc.
-
Have you checked the firewall logs (Status -> System Logs Firewall tab)?
tcpdump can be very helpful to verify packets arriving at a particular interface or going out a particular interface.
On pfSense and other unix like systems,
pfsense -i <interface>will display traffic on the specified interface. You can use this to verify that the packets are being seen at each of the hops on the route. Its best done from the console but can be be done in a ssh session though it can become confusing if your ssh traffic is travelling over the interface you are tracing. You should also check the target to make sure its responding (I have recollections of sshd configuration not being "obvious".)
Concerning jumbo frames: In your case, wouldn't you also need to have jumbo frames enabled on the opt1 network? (I'm not familiar with the interactions of MTU discovery, and fragmentation in pfSense). But first you want to be sure your ssh connection is getting to its target and the target is responding.</interface>
-
Have you checked the firewall logs (Status -> System Logs Firewall tab)?
Yes, I have also monitored what's happening via ssh from the pfsense box. The problem here is that there is an obscene amount of things in the log, due to several windows machines and servers sending broadcast and udp traffic everywhere on the network. But I can see the initial ssh connection passing thru, like I mentioned in the first post.
tcpdump can be very helpful to verify packets arriving at a particular interface or going out a particular interface.
On pfSense and other unix like systems,
pfsense -i <interface>will display traffic on the specified interface. You can use this to verify that the packets are being seen at each of the hops on the route. Its best done from the console but can be be done in a ssh session though it can become confusing if your ssh traffic is travelling over the interface you are tracing. You should also check the target to make sure its responding (I have recollections of sshd configuration not being "obvious".)</interface>
I'll check this on monday.
Concerning jumbo frames: In your case, wouldn't you also need to have jumbo frames enabled on the opt1 network? (I'm not familiar with the interactions of MTU discovery, and fragmentation in pfSense). But first you want to be sure your ssh connection is getting to its target and the target is responding.
I hope not, as that would force me to use something else than pfsense. I dont want to have several boxes doing different things, just one neat package doing the stuff I need.
/jussi
-
This is what I see in the filter logs:
pass in on fxp0: 192.168.140.77.1549 > 192.168.210.113.22: tcp 28 [bad hdr length 0 - too short, < 20]
And nothing after that, form either of those IP's.
/jussi
-
This is what I see in the filter logs:
pass in on fxp0: 192.168.140.77.1549 > 192.168.210.113.22: tcp 28 [bad hdr length 0 - too short, < 20]
And nothing after that, form either of those IP's.
/jussi
This might be the reason the box was "lying around".
I presume fxp0 is the motherboard NIC. That trace would lead me to suspect that NIC was broken or you had a memory problem. Might be time to try a known working box OR try another NIC and run some memory diagnostics. Quite a few of the LINUX Live CDs have the option of booting the memtest86 memory diagnostics or you can probably download a CD image. (Google for memtest86 or memtest86+). You might also want to hook another system up to than LAN port and ping the port while watching the interface traffic with tcpdump, just to get an idea how frequently this sort of thing happens.
That an apparently well formed TCP packet passes the received FCS check but has a bad header length suggests to me either the NIC is broken (writing bad data to memory) OR the NIC is OK but some part of the memory is broken (returning bad data on reads), OR both.
If you dust the box out, take out all the cards including memory sticks, clean out the slots with a vacuum cleaner and reseat all the cards things may come good.
-
Hi
One quick. Put ifconfig in the very last of /etc/rc.instead after each reboot I need to set it up by hand using ifconfig. Any solutions to this?
cheers,
-
pass in on fxp0: 192.168.140.77.1549 > 192.168.210.113.22: tcp 28 [bad hdr length 0 - too short, < 20]
first, disable hardware checksum offloading (in advanced)
second - try different NIC -
Hi Guys, thanks for the tips.
@pan_2:
pass in on fxp0: 192.168.140.77.1549 > 192.168.210.113.22: tcp 28 [bad hdr length 0 - too short, < 20]
first, disable hardware checksum offloading (in advanced)
second - try different NICTried this. Still not succeeding with the NAT. I also changed a new 3COM NIC in instead of the built in NIC. I'm still getting the fragmentation errors:
000000 rule 44/0(match): pass in on xl0: 192.168.140.77.1317 > 192.168.210.113.80: tcp 28 [bad hdr length 0 - too short, < 20]
I discovered something else, that might explain the issues here:
When I look up the routing tables from the shell (using SSH) I get this:
netstat -r -f inet -i
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
xl0 1500 192.168.140.0 192.168.140.250 1975 - 714 - -
re0 1500 192.168.190.0 192.168.190.254 3 - 3 - -
stge0 9000 192.168.210.0 ihantala 33097 - 34218 - -
lo0 16384 your-net localhost 339 - 339 - -netstat -ri
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
xl0 1500 <link#1>00:10:5a:39:b0:ea 53105 1 25977 0 0
xl0 1500 192.168.140.0 192.168.140.250 2022 - 736 - -
xl0 1500 fe80:1::210:5 fe80:1::210:5aff: 0 - 2 - -
fxp0* 1500 <link#2>00:08:02:1a:c4:02 0 0 0 0 0
re0 1500 <link#3>00:e0:4c:69:77:5b 193 0 145 0 0
re0 1500 192.168.190.0 192.168.190.254 3 - 3 - -
re0 1500 fe80:3::2e0:4 fe80:3::2e0:4cff: 0 - 2 - -
stge0 9000 <link#4>00:05:5d:88:8b:7e 78411 0 78213 0 0
stge0 9000 192.168.210.0 ihantala 33167 - 34288 - -
stge0 9000 fe80:4::205:5 fe80:4::205:5dff: 0 - 2 - -
plip0 1500 <link#5>0 0 0 0 0
pflog 33204 <link#6>0 0 4548 0 0
lo0 16384 <link#7>344 0 344 0 0
lo0 16384 your-net localhost 344 - 344 - -
lo0 16384 localhost ::1 0 - 0 - -
lo0 16384 fe80:7::1 fe80:7::1 0 - 0 - -
enc0* 1536 <link#8>0 0 0 0 0
pfsyn 1460 <link#9>0 0 0 0 0stge0 has the correct MTU, no problems. But when I look at the routing tables from the web-interface (Diagnostic->Routes)
IPv4
Destination Gateway Flags Refs Use Mtu Netif Expire
default 192.168.140.254 UGS 0 24739 1500 xl0
127.0.0.1 127.0.0.1 UH 0 347 16384 lo0
192.168.140.0/24 link#1 UC 0 1 1500 xl0
192.168.140.13 00:1b:78:93:1d:2a UHLW 1 605 1500 xl0 1167
192.168.140.14 00:1b:78:95:4a:30 UHLW 1 91 1500 xl0 1104
192.168.140.77 00:e0:00:9b:9e:66 UHLW 1 0 1500 xl0 934
192.168.140.254 00:90:7f:3c:89:30 UHLW 2 475 1500 xl0 1194
192.168.190.0/24 link#3 UC 0 7 1500 re0
192.168.210.0/24 link#4 UC 0 0 1500 stge0
192.168.210.38 00:e0:ed:0a:49:60 UHLW 1 1 1500 stge0 66
192.168.210.49 00:1f:5b:35:9a:61 UHLW 1 13 1500 stge0 1080
192.168.210.50 00:1b:63:b7:4c:2c UHLW 1 78261 1500 stge0 405
192.168.210.103 00:0a:5e:20:a7:ed UHLW 1 6 1500 stge0 1161
192.168.210.109 00:14:51:65:02:82 UHLW 1 2 1500 stge0 1170
192.168.210.113 00:1f:5b:31:40:20 UHLW 1 21 1500 stge0 64The systems shows all routes as MTU=1500! Is my problem here? Should I be trying to use another version or build of pfsense? Any suggestions to why I am getting this kind of problems?
The box was lying around because we replaced it with higher-end machine. There were no actual stability issues with it (not more than your average XP box, that is). I am loathe to put 1000 euros into a decent FW machine before I know I can actually achieve what I am getting at.</link#9></link#8></link#7></link#6></link#5></link#4></link#3></link#2></link#1>
-
Hi,
Just MTU thingy put a side for now and focusing on port forwarding issue, I've experienced most likely the same issue with the latest 1.3-AA build yesterday. Everything on the WAN gets port forwarded by the rule are dropped with the [too short] messages. Although yours isn't 1.3 nor the latest 1.2.1 so it might not your case but I strongly recommend that you try some OLDER builds.
BTW, do you still see the issue even you revert all the MTU setting to default?
cheers,
-
Are you allowing icmp otherwise you break path mtu discovery?!
Can you monitor if RST packets are being sent by pfsense to the .140 network?
Are you sure that ssh on pfSense is not running on that port(ssh:22)?
-
@ermal:
Are you allowing icmp otherwise you break path mtu discovery?!
Yes, I have tried to let all icmp traffic thru on the interfaces.
Can you monitor if RST packets are being sent by pfsense to the .140 network?
How can I monitor them? I know for sure that nothing is reaching the www or ftp servers.
Are you sure that ssh on pfSense is not running on that port(ssh:22)?
I have tied ftp, ssh and http, and always with the same results.
I swapped hardware as well, I am know using a HP DL145 server with a 4 port Intel Server NIC (we are going to put server to other use in a week or so, I just want to verify that pfsense will work in our environment before I splash the cash on some new hardware). Should be no issues with the hardware this time…
I also decided to go with the stable 1.2 build at this point, there shouldnt be a lot of changes is basic FW rules and NATing from 1.2 to 1.2.1, right?
/jussi