Why outgoing LAN being blocked?
-
If you pinging your isp gateway and your getting 800ms.. that is high to be sure.. So either your pipe is full and that is causing it, or their router (your gateway) is loaded and not answering pings quickly or not at all.. As mentioned not all sites on the internet will answer ping.
what does a traceroute to say googledns look like?
example
traceroute -n 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 192.168.1.253 0.387 ms 0.308 ms 0.290 ms
2 24.13.x.1 12.922 ms 12.926 ms 28.036 ms
3 68.85.131.149 11.891 ms 12.437 ms 12.440 ms
4 68.86.196.33 15.119 ms 68.86.187.213 13.798 ms 68.86.197.149 14.935 ms
5 68.86.94.45 17.214 ms * *
6 68.86.88.22 23.731 ms 22.785 ms 23.554 ms
7 68.86.87.126 19.738 ms 13.399 ms 13.459 ms
8 66.208.233.142 12.663 ms 11.966 ms 15.963 ms
9 * * *
10 72.14.237.133 12.689 ms 209.85.254.240 16.457 ms 18.607 ms
11 209.85.241.22 37.496 ms 72.14.238.104 37.271 ms 36.170 ms
12 216.239.43.217 28.410 ms 28.363 ms 27.931 ms
13 * * *
14 8.8.8.8 29.482 ms 25.311 ms 27.303 msSo this is from a linux box behind pfsense.. so first hop is pfsense - notice that is very fast because its local lan. Then next hop is my ISP gateway.. not bad 10 to 12 ms.. And then notice the rest.. I snipped out part of the isp IP since it would of told you what network specific comcast network I am on, its a large /21 but no reason that is needed in this example, etc.
So curious if your seeing really slow times for the whole path, or only to specific hops in the path? The -n tells it not to do PTR lookups on the IPs makes for quicker finish to the trace. that is linux, windows it would be tracert -d 8.8.8.8 for example
Hi John, thanks for the response. Here is the results of doing the -d 8.8.8.8 with a windows machine connected to a Cisco SG-200-08 which is connected to my pfSense box:
1 <1 ms <1ms <1ms 192.168.1.1
2 821ms 998ms 966ms 10.xxx.x.1
3 73ms 926ms 1001ms 68.6.12.38
4 * * * Request timed out.
5 211ms 676ms 324ms 68.6.8.100
6 328ms 782ms 217ms 68.1.0.136
7 217ms 33ms 966ms 68.105.30.181
8 220ms 745ms 999ms 64.233.174.238
9 1000ms 1025ms 973ms 64.233.174.192
10 218ms 52ms 73ms 72.14.239.153
11 411ms 315ms 918ms 216.239.48.167
12 * * * Request timed out.
13 228ms 585ms 432ms 8.8.8.8What. The. Hell.
Here are the results using traceroute on pfSense (directly connected to modem):
Traceroute output:
1 10.xxx.x.1 (10.xxx.x.1) 49.116 ms 988.726 ms 975.036 ms
2 (68.6.12.38) 1003.208 ms 369.721 ms 177.508 ms
3 * * *
4 (68.6.8.100) 909.782 ms 80.088 ms 64.463 ms
5 (68.1.0.136) 21.123 ms 47.357 ms 460.282 ms
6 (68.105.30.181) 155.108 ms 58.613 ms 27.881 ms
7 216.239.46.40 (216.239.46.40) 33.634 ms
64.233.174.238 (64.233.174.238) 167.878 ms 75.303 ms
8 64.233.174.188 (64.233.174.188) 211.115 ms 937.086 ms
72.14.238.0 (72.14.238.0) 195.566 ms
9 72.14.239.155 (72.14.239.155) 998.353 ms
72.14.239.162 (72.14.239.162) 347.896 ms
72.14.239.159 (72.14.239.159) 423.843 ms
10 64.233.174.131 (64.233.174.131) 999.183 ms
216.239.48.165 (216.239.48.165) 119.867 ms
216.239.48.167 (216.239.48.167) 68.999 ms
11 * * *
12 google-public-dns-a.google.com (8.8.8.8) 860.106 ms 981.388 ms 1000.152 ms -
821ms 998ms 966ms 10.xxx.x.1
10 address is not public, so your behind a double nat. is that your ISP doing gobal nat or is that the device your pfsense is directly connected too.. You mention "modem" what model number - since its seems to be doing NAT.. and then your ping times to isp would be this hop
3 73ms 926ms 1001ms 68.6.12.38
So to me it looks like you have a problem between pfsense and whatever that 10.x devices is – your "modem" Which would be local on your network.. and should be more like the speeds your seeing to pfsense of <1ms
So what need to figure out what this 10.x.x is - is that your local device or something outside your location at the ISP.. I am thinking its your modem which would be local... BTW anything that starts with 10.x.x.x is a rfc1918 address and not routeable on the internet - so no reason to hide that, just like the 192.168.x.x addresses.
edit: So your 3rd hop which I would to me be first hop to your ISP with that 10.x address as second.. I am seeing
PING 68.6.12.38 (68.6.12.38): 56 data bytes
64 bytes from 68.6.12.38: icmp_seq=0 ttl=244 time=81.579 ms
64 bytes from 68.6.12.38: icmp_seq=1 ttl=244 time=81.943 ms
64 bytes from 68.6.12.38: icmp_seq=2 ttl=244 time=80.031 ms80ms -- I am in Chicago, where are you and your see 800ms to the first hop after pfsense which I have to think is your local modem. And would cause you to see delays talking to anything past that.
-
821ms 998ms 966ms 10.xxx.x.1
10 address is not public, so your behind a double nat. is that your ISP doing gobal nat or is that the device your pfsense is directly connected too.. You mention "modem" what model number - since its seems to be doing NAT.. and then your ping times to isp would be this hop
3 73ms 926ms 1001ms 68.6.12.38
So to me it looks like you have a problem between pfsense and whatever that 10.x devices is – your "modem" Which would be local on your network.. and should be more like the speeds your seeing to pfsense of <1ms
So what need to figure out what this 10.x.x is - is that your local device or something outside your location at the ISP.. I am thinking its your modem which would be local... BTW anything that starts with 10.x.x.x is a rfc1918 address and not routeable on the internet - so no reason to hide that, just like the 192.168.x.x addresses.
edit: So your 3rd hop which I would to me be first hop to your ISP with that 10.x address as second.. I am seeing
PING 68.6.12.38 (68.6.12.38): 56 data bytes
64 bytes from 68.6.12.38: icmp_seq=0 ttl=244 time=81.579 ms
64 bytes from 68.6.12.38: icmp_seq=1 ttl=244 time=81.943 ms
64 bytes from 68.6.12.38: icmp_seq=2 ttl=244 time=80.031 ms80ms -- I am in Chicago, where are you and your see 800ms to the first hop after pfsense which I have to think is your local modem. And would cause you to see delays talking to anything past that.
I am in Southern California and my modem is a Cisco-model DPQ3212 DOCSIS 3.0.
I don't know if my ISP is doing global NAT, first time hearing about such a thing.
When I first installed pfSense my firewall kept blocking those 10.x addresses every minute so I turned off logging for that traffic because it looked like DHCP broadcast traffic.
UPDATE: So I called my ISP and told them that I was getting very high latency on the gateway IP and just before he was going to transfer me to tech level 2 he reset the modem and now I am getting 7-9ms on that gateway IP. He didn't know why I was getting that 10.x address BTW.
However, the trace route to google dns still shows that 10.x address in the hop. Is that something I need to be worried about?
Here is the new trace route to google dns:
1 10.x.x.x 7.748 ms 6.194 ms 5.948 ms
2 68.6.12.38 8.211 ms 8.286 ms 7.702 ms
3 * * *
4 68.6.8.100 9.710 ms 9.896 ms 10.090 ms
5 68.1.5.137 75.889 ms 15.220 ms 55.754 ms
6 68.105.30.181 14.028 ms 14.192 ms 13.443 ms
7 64.233.174.238 22.924 ms 14.571 ms
216.239.46.40 17.534 ms
8 72.14.238.0 39.652 ms
64.233.174.188 16.144 ms
72.14.238.0 55.597 ms
9 72.14.239.160 40.011 ms
72.14.239.162 40.368 ms
72.14.239.155 39.777 ms
10 216.239.48.165 40.960 ms
216.239.48.167 40.724 ms
216.239.48.165 48.806 ms
11 * * *
12 8.8.8.8 42.643 ms 41.818 ms 40.886 ms -
Good that the latency is better now. Next you probably want to understand what the 10.x.x.x address is about. As JohnPoz said, there is no need to hide those as it is private address space and no-one can find you using "10" addresses.
What is your WAN IP and WAN gateway addresses?
(Status->Interfaces should tell you what addresses the WAN was given)
Most likely they are 10.x.x.x and that just means your cable modem is in router mode rather than bridge mode. -
Well if that is your model number, it is just a cable modem I don't see anywhere in its docs talking about NAT.. So if your seeing a 10.x.x.x as you next hop.. Your ISP is doing it..
Again 10.x.x.x is PRIVATE its NOT routeable on the internet..
http://en.wikipedia.org/wiki/Private_network
Normally in a cable connection, I have one I have a SB6120 cable modem - my pfsense gets a public IP address 24.13.x.x – this is own by comcast.
whois 24.13.0.0
NetRange: 24.0.0.0 - 24.15.255.255
CIDR: 24.0.0.0/12
NetName: EASTERNSHORE-1
NetHandle: NET-24-0-0-0-1
Comment: ADDRESSES WITHIN THIS BLOCK ARE NON-PORTABLE
RegDate: 2003-10-06
Updated: 2012-03-02
Ref: http://whois.arin.net/rest/net/NET-24-0-0-0-1
OrgName: Comcast Cable Communications, Inc.Look up 10.x.x.x
whois 10.0.0.0
NetRange: 10.0.0.0 - 10.255.255.255
CIDR: 10.0.0.0/8
OriginAS:
NetType: IANA Special Use
NetName: PRIVATE-ADDRESS-ABLK-RFC1918-IANA-RESERVEDComment: These addresses are in use by many millions of independently operated networks, which might be as small as a single computer connected to a home gateway, and are automatically configured in hundreds of millions of devices. They are only intended for use within a private context and traffic that needs to cross the Internet will need to use a different, unique address.
Comment: These addresses can be used by anyone without any need to coordinate with IANA or an Internet registry. The traffic from these addresses does not come from ICANN or IANA. We are not the source of activity you may see on logs or in e-mail records. Please refer to http://www.iana.org/abuse/answers
So just like pfsense NATS changes your private range on your private side to normally what is a public address, pfsense is natting yours to your 10.x.x.x address, then your ISP HAS to change it again to some routeable address on the internet or sites you try to go to would not be able to talk back to you - since they can not talk to a 10.x.x.x address
If you ISP has no idea why you have a 10.x.x.x address you should really call them back and ask to talk to someone that does know ;) unless they are doing a 1:1 nat to what your public address is - its not possible for you to allow for unsolicited traffic behind a nat.. Port Forwards, maybe thats something your ok with? Maybe they do 1:1 but that seems utterly pointless for them to do.
But your connections should be much better now ;) with nice low ping time to your gateway.. Internet must be much better!
-
Good that the latency is better now. Next you probably want to understand what the 10.x.x.x address is about. As JohnPoz said, there is no need to hide those as it is private address space and no-one can find you using "10" addresses.
What is your WAN IP and WAN gateway addresses?
(Status->Interfaces should tell you what addresses the WAN was given)
Most likely they are 10.x.x.x and that just means your cable modem is in router mode rather than bridge mode.My external IP and Gateway IP match except for the last octet, they aren't 10.x.x.x but start with 68.x.x.x
-
My external IP and Gateway IP match except for the last octet, they aren't 10.x.x.x but start with 68.x.x.x
And how is that since your first hop is 10.x.x.x
So on pfsense what does it show for your wan interface?
Sorry your hop shows you talking to a 10 address.. its not possible for a 68.x.x.x address to talk to a 10 address directly.. If you have a 68 address on pfsense, I am at a complete loss to how a 10 address would show up in your trace.
-
My external IP and Gateway IP match except for the last octet, they aren't 10.x.x.x but start with 68.x.x.x
And how is that since your first hop is 10.x.x.x
So on pfsense what does it show for your wan interface?
Sorry your hop shows you talking to a 10 address.. its not possible for a 68.x.x.x address to talk to a 10 address directly.. If you have a 68 address on pfsense, I am at a complete loss to how a 10 address would show up in your trace.
This is what mine is showing.
-
Well, that is completely wacky. If you are still getting 10.x.x.x appearing early in your traceroute (from pfSense and/or a LAN client) then look in config.xml:
Diagnostics->Edit
/cf/conf/config.xml
Search for "10."
and Diagnostics->Routes - what is the default route?
Is there some VPN server and client that connects to itself and routes around in a loop to make that bonus hop, or what??? -
Well, that is completely wacky. If you are still getting 10.x.x.x appearing early in your traceroute (from pfSense and/or a LAN client) then look in config.xml:
Diagnostics->Edit
/cf/conf/config.xml
Search for "10."
and Diagnostics->Routes - what is the default route?
Is there some VPN server and client that connects to itself and routes around in a loop to make that bonus hop, or what???Hey Phil,
I did as you asked and looked in the config.xml file, I pasted it into Word and ran a search for anything matching "10".. It didn't come back with any 10.x.x.x. I also looked through the file without the search function and didn't notice anything.
The IPv4 routing tables don't have any 10.x.x.x addresses listed. The default Gateway is 68.105.x.1, as it is for 8.8.4.4 and 8.8.8.8
Ran traceroute again, its still showing the 10.x.x.x as the first hop.
I have Private Internet Access configured on my PC but that is the only VPN I use and it is almost always disconnected. Running traceroute on my PC the first hop is 192.168.1.1 and the 2nd is 10.x.x.x
I do appreciate the help from both you and John, if nothing else I am learning a lot from this!
-
I don't recall ever seeing anything like this before.
On pfsense check the mac of that 10 address if you can – we should then be able to figure out what hardware it is, maybe its your "modem" device.. Very strange!!
So in pfsense ping that hop directly 10.175.0.1 and then look in your arp table on pfsense with arp -a, do you see it listed.. What are the first 3 numbers at least and we can look them up via websites like this
http://www.coffer.com/mac_find/
-
The 10.x.x.x IP is his cable company's CMTS.
-
I don't recall ever seeing anything like this before.
On pfsense check the mac of that 10 address if you can – we should then be able to figure out what hardware it is, maybe its your "modem" device.. Very strange!!
So in pfsense ping that hop directly 10.175.0.1 and then look in your arp table on pfsense with arp -a, do you see it listed.. What are the first 3 numbers at least and we can look them up via websites like this
http://www.coffer.com/mac_find/
I pinged 10.175.0.1 and got a response but under Diagnostics -> ARP Table, or when using arp -a, I don't see any 10.x.x.x
$ arp -a
pfsense.localdomain (192.168.1.1) at 54:be:f7:X:X:72 on em1 permanent [ethernet]
? (192.168.1.152) at 6c:f0:49:ce:8a:8d on em1 expires in 1195 seconds [ethernet]
? (192.168.1.120) at 54:26:96:35:d8:ef on em1 expires in 1158 seconds [ethernet]
? (192.168.1.125) at 00:11:32:1a:a0:6e on em1 expires in 1039 seconds [ethernet]
? (192.168.1.188) at d4:3d:7e:18:94:ad on em1 expires in 1038 seconds [ethernet]
ip68-105-X-X.cox.net (68.105.X.X) at 54:be:f7:X:X:71 on em0 permanent [ethernet]
ip68-105-X-1.cox.net (68.105.X.1) at 00:26:99:X:X:X on em0 expires in 1199 seconds [ethernet]I did search the MAC address belonging to the Gateway IP with the site you linked and it returned 2 results:
Cisco Systems
Prefix: 00:26:99 -
Can you run a under diag, on pfsense a capture on your wan interface and then ping it and capture the traffic. Then we can see its mac in the wirecapture.. Then compare its mac to mac of your isp router at the 68.
Once you have the capture you can download into wireshark and see the mac.. Maybe its the same as your isp router? Very odd how you get a hop between pfsense and its gateway that reports a 10.x.x.x address.
I can honestly say I don't believe I have ever seen such a thing.
-
Again, this is the cable company CMTS: http://en.wikipedia.org/wiki/Cable_modem_termination_system
It's not that rare. Doing a traceroute over a Charter or Comcast connection will show a 10.x.x.x IP as well.
-
"Doing a traceroute over a Charter or Comcast connection will show a 10.x.x.x IP as well."
No not really - I am on comcast, and as you see there is no 10.x in my trace.
See hop 2, next hop after my pfsense box
;; ANSWER SECTION:
xx.xx.13.24.in-addr.arpa. 7194 IN PTR c-24-13-xx-xx.hsd1.il.comcast.net.NetRange: 24.0.0.0 - 24.15.255.255
CIDR: 24.0.0.0/12
OrgName: Comcast Cable Communications, Inc.
-
I've seen it on some Comcast connections in the past. Here is mine (Charter):
Tracing route to 8.8.8.8 over a maximum of 30 hops 1 <1 ms <1 ms <1 ms 10.1.16.1 2 8 ms 7 ms 8 ms 10.216.96.1 3 11 ms 10 ms 9 ms 96.34.70.34 4 13 ms 10 ms 9 ms 96.34.70.116 ... -
Your first hop is 10, which is local with that <1ms response time, and then your second hop is also 10..
So your saying your router (pfsense/other) shows a public IP on it like his and mine, 68.x and my 24.x or does yours have a 10.x.x.x something on where the mask puts in in the same network as your hop 3 10.216.96.1
What your showing makes sense where nat to public happening between hop 2 and 3.
What doesn't make sense in his setup is he has a public showing a public gateway – but a 10.x in the middle. Your trace looks like a typical double nat setup to me..
-
My router (pfSense) is 10.1.16.1. My first hop outside of my network is 10.216.96.1 which is the CMTS interface (Charter).
-
No 10.1.16 is your LAN of pfsense - what is the WAN of your pfsense. Is it 10.216 or say something public like my 24.x or his 68.x
Your routers WAN ip would never been shown in a hop. Unless tracing inbound to your IP.
-
My WAN IP is 68.186.x.x which of course isn't shown on an outbound tracert.
-
Can you run a under diag, on pfsense a capture on your wan interface and then ping it and capture the traffic. Then we can see its mac in the wirecapture.. Then compare its mac to mac of your isp router at the 68.
Hi John, can you please clarify the process of running an under diag on pfSense? On pfSense I would go to "Diagnostics -> Packet Capture"? I apologize if that is incorrect, this is all still somewhat new to me.
I get this when I ran a packet capture on the WAN interface and used that 10.x (found in my tracert) as the Host Address->
"IP 10.175.0.1.67 > 255.255.255.255.68: UDP, length 300".
When I opened that packet capture in WireShark and looked for the MAC address I found–->
"Ethernet II, Src: Cisco_X:X:X (00:26:99:X:X:X), Dst: Broadcast (ff:ff:ff:ff:ff:ff)".
The arp -a showed that "ip68-105-X-1.cox.net (68.105.X.1) at 00:26:99:X:X:X on em0 expires in 1199 seconds [ethernet]"
Under the Bootstrap Protocol section for the DHCP ACK its showing the Client MAC Address as "Motorola" prefix 00:0b:06.
Under the same section, but for the DHCP Offer, its showing the Client MAC Address as "Cisco" prefix 00:22:6b.
-
"My WAN IP is 68.186.x.x which of course isn't shown on an outbound tracert."
And how exactly does a 68.186 address talk to a 10.x address? And what exactly does pfsense say is your gateway address is?
Where is anything close to 68.186?
1 <1 ms <1 ms <1 ms 10.1.16.1
2 8 ms 7 ms 8 ms 10.216.96.1
3 11 ms 10 ms 9 ms 96.34.70.34Your trace makes NO sense if your saying pfsense shows your public IP as 68.186.x.x
Notice in my trace..
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 192.168.1.253 1.726 ms 1.603 ms 1.557 ms
2 24.13.xx.1 19.559 ms 20.384 ms 38.945 ms
3 68.85.131.149 19.922 ms 19.911 ms 19.906 msWhere my wan IP is 24.13.x.x with a /21 mask - and when I trace I show that hop my router talked to next – in the same network as actually IN.. ie 24.13.x.x/21
You are looking at dhcp packets - no you want icmp in the dropdown of the packet capture.. And ping the 10.175.0.1 address from a client.. And only capture stuff to 10.175.0.1
See where I use 8.8.8.8 use that 10.175.0.1 address you see in your trace
-
The cable modem requests an IP address, it is given a 10.x address. It communicates with the CMTS which also has a 10.x address. The CMTS is also configured with a routable address which is the gateway IP.
-
Okay, I did as you said and pinged the 10.x address from a client while I was capturing the ICMP packets from that IP and looked at it in WireShark.
The 10.175.0.1 address has the 00:26:99 MAC prefix, which is Cisco. My WAN Interface (68.x) has a MAC prefix of 54:be:f7. Searching it gives me no results.
I looked at my modem's MTA MAC and it is "e4:48:c7", which is "Cisco SPVTG".
-
I show that as
http://www.wireshark.org/tools/oui-lookup.html
54:BE:F7 PEGATRON CORPORATIONhttp://en.wikipedia.org/wiki/Pegatron
Pegatron Corporation (Chinese: 和碩聯合科技股份有限公司; pinyin: Hé shuò liánhé kējì gǔfèn yǒuxiàn gōngsī, lit. Grand Mastery United Technology Corporation) is a Taiwanese electronics manufacturing company that develops mainly computing, communications and consumer electronics to branded vendors, but also engages in the development, design and manufacturing of computer peripherals and components. Pegatron's primary products include notebooks, netbook computers, desktop computers, game consoles, handheld devices, motherboards, video cards, LCD TVs, as well as broadband communication products such as smartphones, set-top boxes and cable modems.[6][7]
Your only going to be able to see macs of of devices directly connected to you, or over a bridge. So is the mac of 10.175 the same as mac of your 68.x gateway?
-
Your only going to be able to see macs of of devices directly connected to you, or over a bridge. So is the mac of 10.175 the same as mac of your 68.x gateway?
Yes, I ran a packet capture on the 10.175.0.1 address and the MAC is the same as the 68.x Gateway.. both are 00:26:99:XX:XX:XX
-
New knowledge is very attractive.
