Why outgoing LAN being blocked?
-
What would cause the flow of traffic to behave in a way that was not normal? Is that traffic blocked permanently?
No, it is only the "old" traffic for the timed-out state that is blocked. After a little while the client device will time-out also and attempt to start a new connection (if it is even still on the network) and thus a new state will get established and away it goes.
Some client-server software might just stop doing anything when it is finished, without nicely closing the connection, (or a client phone/tablet/laptop gets carried out of range of a WiFi and thus has no choice about abruptly disappearing…) and thus the other end might send a few packets later on, wondering if its partner is still there - that kind of stuff is "normal" and will result in these odd-looking firewall block logs. -
What would cause the flow of traffic to behave in a way that was not normal? Is that traffic blocked permanently?
No, it is only the "old" traffic for the timed-out state that is blocked. After a little while the client device will time-out also and attempt to start a new connection (if it is even still on the network) and thus a new state will get established and away it goes.
Some client-server software might just stop doing anything when it is finished, without nicely closing the connection, (or a client phone/tablet/laptop gets carried out of range of a WiFi and thus has no choice about abruptly disappearing…) and thus the other end might send a few packets later on, wondering if its partner is still there - that kind of stuff is "normal" and will result in these odd-looking firewall block logs.Okay thanks for the answer. My other question is why is my latency so high? I am having issues pinging certain things like my server and mobile devices (mind you I am logged into my server right now via LAN and WAN connection yet no ping?). Also, when I ping amazon.com I get 100.0% packet loss yet I am on Amazon right now. Again, I apologize if these are dumb questions, trying to figure out if I even have issues at all. My latency according to the dynamic gateway is almost always near 800ms which I thought was bad.
PING 192.168.1.125 (192.168.1.125) from 192.168.1.1: 56 data bytes
–- 192.168.1.125 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss -
Not every server has software enabled, listening for and responding to ping. e.g. amazon.com ignores people trying to ping it. Some servers are nice enough to let everyone ping them - e.g. Google at 8.8.8.8 and 8.8.4.4
Your server at 192.168.1.125 might have a firewall that stops pings.
800ms is not good latency. If you have a slower internet connection and do a big download with a download manager that gets lots of download streams going, then you can saturate your link. Then the pings get delayed by the download and you see high latency. Otherwise, 800ms is for satellite links!
What are you using for the monitor IP on the WAN gateway? Just the WAN gateway IP itself? or Google 8.8.8.8? or some server in outer Mongolia ;) ? -
Not every server has software enabled, listening for and responding to ping. e.g. amazon.com ignores people trying to ping it. Some servers are nice enough to let everyone ping them - e.g. Google at 8.8.8.8 and 8.8.4.4
Your server at 192.168.1.125 might have a firewall that stops pings.
800ms is not good latency. If you have a slower internet connection and do a big download with a download manager that gets lots of download streams going, then you can saturate your link. Then the pings get delayed by the download and you see high latency. Otherwise, 800ms is for satellite links!
What are you using for the monitor IP on the WAN gateway? Just the WAN gateway IP itself? or Google 8.8.8.8? or some server in outer Mongolia ;) ?hehe no, not Mongolia. The monitor IP/gateway IP were automatically added during the initial setup? Under System/Routing/Edit Gateways my settings are
Interface: WAN
Address faminly: IPv4
Name: WAN_DHCP
Gateway: dynamic
Default Gateway Enabled.My external IP address is 6x.xxx.xx.71 and the monitor IP/gateway IP is 6x.xxx.xx.1 (which I'm guessing is my ISP?) My latency is super high most of the time, even when pinging google.
My DNS servers are:
8.8.8.8
8.8.4.4
208.67.222.222 (OpenDNS)
208.67.220.220 (OpenDNS)all are pointing to my default gateway IP 6x.xxx.xx.1
I also turned OFF "Allow DNS server list to be overridden by DHCP/PPP on WAN" and ENABLED "Do not use the DNS Forwarder as a DNS server for the firewall"
-
and will result in these odd-looking firewall block logs.
A great clear explanation once again, Phil ;D
Would there be a way to hide these messages from the log? As basically they seem to me (as a noob, disclaimer ;D) as useless information in day to day life, that probably only would be useful for debugging purposes and then should be enabled temporary.
Or am I talking rubbish now (economists often do :P).
(I'm asking, because I am in a horrible fight with my own logs for months, as I wrote in this thread: http://forum.pfsense.org/index.php/topic,69686.msg389966.html#msg389966)
-
If you pinging your isp gateway and your getting 800ms.. that is high to be sure.. So either your pipe is full and that is causing it, or their router (your gateway) is loaded and not answering pings quickly or not at all.. As mentioned not all sites on the internet will answer ping.
what does a traceroute to say googledns look like?
example
traceroute -n 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 192.168.1.253 0.387 ms 0.308 ms 0.290 ms
2 24.13.x.1 12.922 ms 12.926 ms 28.036 ms
3 68.85.131.149 11.891 ms 12.437 ms 12.440 ms
4 68.86.196.33 15.119 ms 68.86.187.213 13.798 ms 68.86.197.149 14.935 ms
5 68.86.94.45 17.214 ms * *
6 68.86.88.22 23.731 ms 22.785 ms 23.554 ms
7 68.86.87.126 19.738 ms 13.399 ms 13.459 ms
8 66.208.233.142 12.663 ms 11.966 ms 15.963 ms
9 * * *
10 72.14.237.133 12.689 ms 209.85.254.240 16.457 ms 18.607 ms
11 209.85.241.22 37.496 ms 72.14.238.104 37.271 ms 36.170 ms
12 216.239.43.217 28.410 ms 28.363 ms 27.931 ms
13 * * *
14 8.8.8.8 29.482 ms 25.311 ms 27.303 msSo this is from a linux box behind pfsense.. so first hop is pfsense - notice that is very fast because its local lan. Then next hop is my ISP gateway.. not bad 10 to 12 ms.. And then notice the rest.. I snipped out part of the isp IP since it would of told you what network specific comcast network I am on, its a large /21 but no reason that is needed in this example, etc.
So curious if your seeing really slow times for the whole path, or only to specific hops in the path? The -n tells it not to do PTR lookups on the IPs makes for quicker finish to the trace. that is linux, windows it would be tracert -d 8.8.8.8 for example
-
If you pinging your isp gateway and your getting 800ms.. that is high to be sure.. So either your pipe is full and that is causing it, or their router (your gateway) is loaded and not answering pings quickly or not at all.. As mentioned not all sites on the internet will answer ping.
what does a traceroute to say googledns look like?
example
traceroute -n 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 192.168.1.253 0.387 ms 0.308 ms 0.290 ms
2 24.13.x.1 12.922 ms 12.926 ms 28.036 ms
3 68.85.131.149 11.891 ms 12.437 ms 12.440 ms
4 68.86.196.33 15.119 ms 68.86.187.213 13.798 ms 68.86.197.149 14.935 ms
5 68.86.94.45 17.214 ms * *
6 68.86.88.22 23.731 ms 22.785 ms 23.554 ms
7 68.86.87.126 19.738 ms 13.399 ms 13.459 ms
8 66.208.233.142 12.663 ms 11.966 ms 15.963 ms
9 * * *
10 72.14.237.133 12.689 ms 209.85.254.240 16.457 ms 18.607 ms
11 209.85.241.22 37.496 ms 72.14.238.104 37.271 ms 36.170 ms
12 216.239.43.217 28.410 ms 28.363 ms 27.931 ms
13 * * *
14 8.8.8.8 29.482 ms 25.311 ms 27.303 msSo this is from a linux box behind pfsense.. so first hop is pfsense - notice that is very fast because its local lan. Then next hop is my ISP gateway.. not bad 10 to 12 ms.. And then notice the rest.. I snipped out part of the isp IP since it would of told you what network specific comcast network I am on, its a large /21 but no reason that is needed in this example, etc.
So curious if your seeing really slow times for the whole path, or only to specific hops in the path? The -n tells it not to do PTR lookups on the IPs makes for quicker finish to the trace. that is linux, windows it would be tracert -d 8.8.8.8 for example
Hi John, thanks for the response. Here is the results of doing the -d 8.8.8.8 with a windows machine connected to a Cisco SG-200-08 which is connected to my pfSense box:
1 <1 ms <1ms <1ms 192.168.1.1
2 821ms 998ms 966ms 10.xxx.x.1
3 73ms 926ms 1001ms 68.6.12.38
4 * * * Request timed out.
5 211ms 676ms 324ms 68.6.8.100
6 328ms 782ms 217ms 68.1.0.136
7 217ms 33ms 966ms 68.105.30.181
8 220ms 745ms 999ms 64.233.174.238
9 1000ms 1025ms 973ms 64.233.174.192
10 218ms 52ms 73ms 72.14.239.153
11 411ms 315ms 918ms 216.239.48.167
12 * * * Request timed out.
13 228ms 585ms 432ms 8.8.8.8What. The. Hell.
Here are the results using traceroute on pfSense (directly connected to modem):
Traceroute output:
1 10.xxx.x.1 (10.xxx.x.1) 49.116 ms 988.726 ms 975.036 ms
2 (68.6.12.38) 1003.208 ms 369.721 ms 177.508 ms
3 * * *
4 (68.6.8.100) 909.782 ms 80.088 ms 64.463 ms
5 (68.1.0.136) 21.123 ms 47.357 ms 460.282 ms
6 (68.105.30.181) 155.108 ms 58.613 ms 27.881 ms
7 216.239.46.40 (216.239.46.40) 33.634 ms
64.233.174.238 (64.233.174.238) 167.878 ms 75.303 ms
8 64.233.174.188 (64.233.174.188) 211.115 ms 937.086 ms
72.14.238.0 (72.14.238.0) 195.566 ms
9 72.14.239.155 (72.14.239.155) 998.353 ms
72.14.239.162 (72.14.239.162) 347.896 ms
72.14.239.159 (72.14.239.159) 423.843 ms
10 64.233.174.131 (64.233.174.131) 999.183 ms
216.239.48.165 (216.239.48.165) 119.867 ms
216.239.48.167 (216.239.48.167) 68.999 ms
11 * * *
12 google-public-dns-a.google.com (8.8.8.8) 860.106 ms 981.388 ms 1000.152 ms -
821ms 998ms 966ms 10.xxx.x.1
10 address is not public, so your behind a double nat. is that your ISP doing gobal nat or is that the device your pfsense is directly connected too.. You mention "modem" what model number - since its seems to be doing NAT.. and then your ping times to isp would be this hop
3 73ms 926ms 1001ms 68.6.12.38
So to me it looks like you have a problem between pfsense and whatever that 10.x devices is – your "modem" Which would be local on your network.. and should be more like the speeds your seeing to pfsense of <1ms
So what need to figure out what this 10.x.x is - is that your local device or something outside your location at the ISP.. I am thinking its your modem which would be local... BTW anything that starts with 10.x.x.x is a rfc1918 address and not routeable on the internet - so no reason to hide that, just like the 192.168.x.x addresses.
edit: So your 3rd hop which I would to me be first hop to your ISP with that 10.x address as second.. I am seeing
PING 68.6.12.38 (68.6.12.38): 56 data bytes
64 bytes from 68.6.12.38: icmp_seq=0 ttl=244 time=81.579 ms
64 bytes from 68.6.12.38: icmp_seq=1 ttl=244 time=81.943 ms
64 bytes from 68.6.12.38: icmp_seq=2 ttl=244 time=80.031 ms80ms -- I am in Chicago, where are you and your see 800ms to the first hop after pfsense which I have to think is your local modem. And would cause you to see delays talking to anything past that.
-
821ms 998ms 966ms 10.xxx.x.1
10 address is not public, so your behind a double nat. is that your ISP doing gobal nat or is that the device your pfsense is directly connected too.. You mention "modem" what model number - since its seems to be doing NAT.. and then your ping times to isp would be this hop
3 73ms 926ms 1001ms 68.6.12.38
So to me it looks like you have a problem between pfsense and whatever that 10.x devices is – your "modem" Which would be local on your network.. and should be more like the speeds your seeing to pfsense of <1ms
So what need to figure out what this 10.x.x is - is that your local device or something outside your location at the ISP.. I am thinking its your modem which would be local... BTW anything that starts with 10.x.x.x is a rfc1918 address and not routeable on the internet - so no reason to hide that, just like the 192.168.x.x addresses.
edit: So your 3rd hop which I would to me be first hop to your ISP with that 10.x address as second.. I am seeing
PING 68.6.12.38 (68.6.12.38): 56 data bytes
64 bytes from 68.6.12.38: icmp_seq=0 ttl=244 time=81.579 ms
64 bytes from 68.6.12.38: icmp_seq=1 ttl=244 time=81.943 ms
64 bytes from 68.6.12.38: icmp_seq=2 ttl=244 time=80.031 ms80ms -- I am in Chicago, where are you and your see 800ms to the first hop after pfsense which I have to think is your local modem. And would cause you to see delays talking to anything past that.
I am in Southern California and my modem is a Cisco-model DPQ3212 DOCSIS 3.0.
I don't know if my ISP is doing global NAT, first time hearing about such a thing.
When I first installed pfSense my firewall kept blocking those 10.x addresses every minute so I turned off logging for that traffic because it looked like DHCP broadcast traffic.
UPDATE: So I called my ISP and told them that I was getting very high latency on the gateway IP and just before he was going to transfer me to tech level 2 he reset the modem and now I am getting 7-9ms on that gateway IP. He didn't know why I was getting that 10.x address BTW.
However, the trace route to google dns still shows that 10.x address in the hop. Is that something I need to be worried about?
Here is the new trace route to google dns:
1 10.x.x.x 7.748 ms 6.194 ms 5.948 ms
2 68.6.12.38 8.211 ms 8.286 ms 7.702 ms
3 * * *
4 68.6.8.100 9.710 ms 9.896 ms 10.090 ms
5 68.1.5.137 75.889 ms 15.220 ms 55.754 ms
6 68.105.30.181 14.028 ms 14.192 ms 13.443 ms
7 64.233.174.238 22.924 ms 14.571 ms
216.239.46.40 17.534 ms
8 72.14.238.0 39.652 ms
64.233.174.188 16.144 ms
72.14.238.0 55.597 ms
9 72.14.239.160 40.011 ms
72.14.239.162 40.368 ms
72.14.239.155 39.777 ms
10 216.239.48.165 40.960 ms
216.239.48.167 40.724 ms
216.239.48.165 48.806 ms
11 * * *
12 8.8.8.8 42.643 ms 41.818 ms 40.886 ms -
Good that the latency is better now. Next you probably want to understand what the 10.x.x.x address is about. As JohnPoz said, there is no need to hide those as it is private address space and no-one can find you using "10" addresses.
What is your WAN IP and WAN gateway addresses?
(Status->Interfaces should tell you what addresses the WAN was given)
Most likely they are 10.x.x.x and that just means your cable modem is in router mode rather than bridge mode. -
Well if that is your model number, it is just a cable modem I don't see anywhere in its docs talking about NAT.. So if your seeing a 10.x.x.x as you next hop.. Your ISP is doing it..
Again 10.x.x.x is PRIVATE its NOT routeable on the internet..
http://en.wikipedia.org/wiki/Private_network
Normally in a cable connection, I have one I have a SB6120 cable modem - my pfsense gets a public IP address 24.13.x.x – this is own by comcast.
whois 24.13.0.0
NetRange: 24.0.0.0 - 24.15.255.255
CIDR: 24.0.0.0/12
NetName: EASTERNSHORE-1
NetHandle: NET-24-0-0-0-1
Comment: ADDRESSES WITHIN THIS BLOCK ARE NON-PORTABLE
RegDate: 2003-10-06
Updated: 2012-03-02
Ref: http://whois.arin.net/rest/net/NET-24-0-0-0-1
OrgName: Comcast Cable Communications, Inc.Look up 10.x.x.x
whois 10.0.0.0
NetRange: 10.0.0.0 - 10.255.255.255
CIDR: 10.0.0.0/8
OriginAS:
NetType: IANA Special Use
NetName: PRIVATE-ADDRESS-ABLK-RFC1918-IANA-RESERVEDComment: These addresses are in use by many millions of independently operated networks, which might be as small as a single computer connected to a home gateway, and are automatically configured in hundreds of millions of devices. They are only intended for use within a private context and traffic that needs to cross the Internet will need to use a different, unique address.
Comment: These addresses can be used by anyone without any need to coordinate with IANA or an Internet registry. The traffic from these addresses does not come from ICANN or IANA. We are not the source of activity you may see on logs or in e-mail records. Please refer to http://www.iana.org/abuse/answers
So just like pfsense NATS changes your private range on your private side to normally what is a public address, pfsense is natting yours to your 10.x.x.x address, then your ISP HAS to change it again to some routeable address on the internet or sites you try to go to would not be able to talk back to you - since they can not talk to a 10.x.x.x address
If you ISP has no idea why you have a 10.x.x.x address you should really call them back and ask to talk to someone that does know ;) unless they are doing a 1:1 nat to what your public address is - its not possible for you to allow for unsolicited traffic behind a nat.. Port Forwards, maybe thats something your ok with? Maybe they do 1:1 but that seems utterly pointless for them to do.
But your connections should be much better now ;) with nice low ping time to your gateway.. Internet must be much better!
-
Good that the latency is better now. Next you probably want to understand what the 10.x.x.x address is about. As JohnPoz said, there is no need to hide those as it is private address space and no-one can find you using "10" addresses.
What is your WAN IP and WAN gateway addresses?
(Status->Interfaces should tell you what addresses the WAN was given)
Most likely they are 10.x.x.x and that just means your cable modem is in router mode rather than bridge mode.My external IP and Gateway IP match except for the last octet, they aren't 10.x.x.x but start with 68.x.x.x
-
My external IP and Gateway IP match except for the last octet, they aren't 10.x.x.x but start with 68.x.x.x
And how is that since your first hop is 10.x.x.x
So on pfsense what does it show for your wan interface?
Sorry your hop shows you talking to a 10 address.. its not possible for a 68.x.x.x address to talk to a 10 address directly.. If you have a 68 address on pfsense, I am at a complete loss to how a 10 address would show up in your trace.
-
My external IP and Gateway IP match except for the last octet, they aren't 10.x.x.x but start with 68.x.x.x
And how is that since your first hop is 10.x.x.x
So on pfsense what does it show for your wan interface?
Sorry your hop shows you talking to a 10 address.. its not possible for a 68.x.x.x address to talk to a 10 address directly.. If you have a 68 address on pfsense, I am at a complete loss to how a 10 address would show up in your trace.
This is what mine is showing.
-
Well, that is completely wacky. If you are still getting 10.x.x.x appearing early in your traceroute (from pfSense and/or a LAN client) then look in config.xml:
Diagnostics->Edit
/cf/conf/config.xml
Search for "10."
and Diagnostics->Routes - what is the default route?
Is there some VPN server and client that connects to itself and routes around in a loop to make that bonus hop, or what??? -
Well, that is completely wacky. If you are still getting 10.x.x.x appearing early in your traceroute (from pfSense and/or a LAN client) then look in config.xml:
Diagnostics->Edit
/cf/conf/config.xml
Search for "10."
and Diagnostics->Routes - what is the default route?
Is there some VPN server and client that connects to itself and routes around in a loop to make that bonus hop, or what???Hey Phil,
I did as you asked and looked in the config.xml file, I pasted it into Word and ran a search for anything matching "10".. It didn't come back with any 10.x.x.x. I also looked through the file without the search function and didn't notice anything.
The IPv4 routing tables don't have any 10.x.x.x addresses listed. The default Gateway is 68.105.x.1, as it is for 8.8.4.4 and 8.8.8.8
Ran traceroute again, its still showing the 10.x.x.x as the first hop.
I have Private Internet Access configured on my PC but that is the only VPN I use and it is almost always disconnected. Running traceroute on my PC the first hop is 192.168.1.1 and the 2nd is 10.x.x.x
I do appreciate the help from both you and John, if nothing else I am learning a lot from this!
-
I don't recall ever seeing anything like this before.
On pfsense check the mac of that 10 address if you can – we should then be able to figure out what hardware it is, maybe its your "modem" device.. Very strange!!
So in pfsense ping that hop directly 10.175.0.1 and then look in your arp table on pfsense with arp -a, do you see it listed.. What are the first 3 numbers at least and we can look them up via websites like this
http://www.coffer.com/mac_find/
-
The 10.x.x.x IP is his cable company's CMTS.
-
I don't recall ever seeing anything like this before.
On pfsense check the mac of that 10 address if you can – we should then be able to figure out what hardware it is, maybe its your "modem" device.. Very strange!!
So in pfsense ping that hop directly 10.175.0.1 and then look in your arp table on pfsense with arp -a, do you see it listed.. What are the first 3 numbers at least and we can look them up via websites like this
http://www.coffer.com/mac_find/
I pinged 10.175.0.1 and got a response but under Diagnostics -> ARP Table, or when using arp -a, I don't see any 10.x.x.x
$ arp -a
pfsense.localdomain (192.168.1.1) at 54:be:f7:X:X:72 on em1 permanent [ethernet]
? (192.168.1.152) at 6c:f0:49:ce:8a:8d on em1 expires in 1195 seconds [ethernet]
? (192.168.1.120) at 54:26:96:35:d8:ef on em1 expires in 1158 seconds [ethernet]
? (192.168.1.125) at 00:11:32:1a:a0:6e on em1 expires in 1039 seconds [ethernet]
? (192.168.1.188) at d4:3d:7e:18:94:ad on em1 expires in 1038 seconds [ethernet]
ip68-105-X-X.cox.net (68.105.X.X) at 54:be:f7:X:X:71 on em0 permanent [ethernet]
ip68-105-X-1.cox.net (68.105.X.1) at 00:26:99:X:X:X on em0 expires in 1199 seconds [ethernet]I did search the MAC address belonging to the Gateway IP with the site you linked and it returned 2 results:
Cisco Systems
Prefix: 00:26:99 -
Can you run a under diag, on pfsense a capture on your wan interface and then ping it and capture the traffic. Then we can see its mac in the wirecapture.. Then compare its mac to mac of your isp router at the 68.
Once you have the capture you can download into wireshark and see the mac.. Maybe its the same as your isp router? Very odd how you get a hop between pfsense and its gateway that reports a 10.x.x.x address.
I can honestly say I don't believe I have ever seen such a thing.
