PfSense behind another router, partial connectivity
-
I have internet connectivity via an LTE Modem+router (ZTE MF28B). I have it sitting on 192.168.0.1.
pfSense its behind it with a WAN ip of 192.168.0.150 which is in the DMZ. Then I have LAN on 192.168.1.xxxI have DHCP disabled on the LTE gateway and pfSense is doing it, however Im unable to disable routing functions on the modem, as the "bridged mode" doesn't work on it (the modem is a known POS).
The behavior Im seeing is quite strange. Windows machine on the LAN shows network connectivity, Im able to ping external ips, Skype logs in, and Im able to make calls, however Im not able to load any web pages. Additionally, when I just connect to the network, within a few seconds I can get it to load a page or 2, after which point it starts stalling again. Sites that I initially loaded would continue to function (sometimes).
I have nothing on the network.
What could be causing this?
-
Did you uncheck 'block private networks' in the WAN setup?
Possibly a dns issue. What DNS settings are you using?
Anything in the logs to indicate a problem?
Mobile networks often use private subnets and large scale NAT. Perhaps you have a subnet conflict with something upstream. If you traceroute to somewhere does it show that?
Steve
-
No I believe I didn't, I'll check that out when I have access tonight.
DNS is set to 8.8.8.8 in pfsense, as well as on my windows machine. Im able to resolve domains just fine. Logs show nothing out of the ordinary.
I'll try tracerouting when I get home tonight, but I doubt it has much to do with it. I tried adjusting the MTU, which seemed to have improved my ability to access websites, as in they keep working after I make the initial few requests, but any requests to NEW domains that I havent accessed before just hang there.
-
Okay, I take it back about the traceroute, this is the oddest thing I've ever seen.
With pfSense:
Tracing route to yahoo.com [98.138.253.109] over a maximum of 30 hops: 1 4 ms 1 ms 1 ms 192.168.1.1 2 3 ms 3 ms 1 ms MF28D [192.168.0.1] 3 * * * Request timed out. 4 114 ms 208 ms 201 ms 192.168.102.2 5 44 ms 29 ms 36 ms 10.128.87.1 6 51 ms 24 ms 33 ms 192.168.3.75 7 68 ms 30 ms 34 ms 192.168.3.98 8 49 ms 28 ms 51 ms 10.118.23.37 9 42 ms 42 ms 27 ms 10.118.20.129 10 40 ms 32 ms 40 ms 10.118.20.2 11 30 ms 27 ms 32 ms 24.156.157.137 12 23 ms 30 ms 54 ms 24.156.146.46 13 24 ms 54 ms 51 ms 24.156.157.113 14 40 ms 39 ms 35 ms 69.63.248.233 15 * 99 ms 140 ms 24.156.144.178 16 * * * Request timed out. 17 48 ms 61 ms 56 ms ae-7.pat2.nez.yahoo.com [216.115.104.126] 18 66 ms 60 ms 60 ms xe-7-0-0.msr1.ne1.yahoo.com [216.115.100.5] 19 122 ms 67 ms 105 ms xe-5-0-0.clr2-a-gdc.ne1.yahoo.com [98.138.0.19] 20 56 ms 62 ms 58 ms et-18-25.fab8-1-gdc.ne1.yahoo.com [98.138.93.15] 21 63 ms 62 ms 66 ms po-15.bas1-7-prd.ne1.yahoo.com [98.138.240.16] 22 60 ms 66 ms 54 ms ir1.fp.vip.ne1.yahoo.com [98.138.253.109] Trace complete.Tracing route to forum.pfsense.org [66.219.34.171] over a maximum of 30 hops: 1 1 ms <1 ms 1 ms 192.168.1.1 2 2 ms 1 ms 1 ms MF28D [192.168.0.1] 3 * * * Request timed out. 4 154 ms 192 ms 208 ms 192.168.102.2 5 38 ms 37 ms 31 ms 10.128.87.9 6 38 ms 33 ms 33 ms 192.168.3.75 7 54 ms 46 ms 23 ms 192.168.3.98 8 39 ms 46 ms 34 ms 10.118.23.37 9 24 ms 40 ms 33 ms 10.118.20.129 10 55 ms 30 ms 39 ms 10.118.20.2 11 45 ms 55 ms 44 ms 24.156.157.137 12 51 ms 26 ms 25 ms 24.156.146.46 13 47 ms 42 ms 52 ms 24.156.157.113 14 37 ms 38 ms 31 ms 69.63.248.233 15 * 38 ms 40 ms 24.156.144.178 16 59 ms 38 ms 49 ms 0.xe-5-2-1.pr1.chi10.tbone.rr.com [66.109.9.85] 17 72 ms 74 ms 67 ms 66.109.1.67 18 105 ms 203 ms 75 ms ae-0-0.cr0.chi30.tbone.rr.com [66.109.6.21] 19 61 ms 82 ms 119 ms ae-2-0.cr0.dfw10.tbone.rr.com [66.109.6.22] 20 70 ms 68 ms 142 ms agg3.dllatxl301r.texas.rr.com [107.14.17.137] 21 93 ms 172 ms 89 ms agg1.ausutxla01r.texas.rr.com [24.175.41.47] 22 89 ms 124 ms 86 ms tge9-5.rdrktxaz01h.texas.rr.com [66.68.0.11] 23 129 ms 118 ms 87 ms xe-0-2-0-0.RDRKTXAZ1CW.tx.twcbiz.com [97.77.0.53 ] 24 80 ms 67 ms 87 ms xe-1-3-0.ausxtxti1zw.tx.twcbiz.com [24.73.240.16 3] 25 153 ms 96 ms 83 ms rrcs-24-227-210-130.sw.biz.rr.com [24.227.210.13 0] 26 101 ms 90 ms 83 ms 66.219.34.171 27 94 ms 97 ms 98 ms 66.219.34.171 Trace complete.No pfsense (switch plugged directly into LTE modem)
Tracing route to yahoo.com [206.190.36.45] over a maximum of 30 hops: 1 2 ms 1 ms 1 ms MF28D [192.168.0.1] 2 * * * Request timed out. 3 61 ms 207 ms 236 ms 192.168.102.2 4 32 ms 25 ms 29 ms 10.128.87.5 5 43 ms 26 ms 31 ms 192.168.3.75 6 45 ms 43 ms 41 ms 192.168.3.98 7 32 ms 33 ms 91 ms 10.118.23.37 8 22 ms 36 ms 47 ms 10.118.20.129 9 40 ms 36 ms 39 ms 10.118.20.2 10 32 ms 39 ms 36 ms 24.156.157.137 11 30 ms 37 ms 40 ms 24.156.146.46 12 38 ms 42 ms 39 ms 24.156.157.113 13 40 ms 39 ms 24 ms 69.63.248.233 14 * * * Request timed out. 15 * * * Request timed out. 16 97 ms 76 ms 74 ms ae-5.pat1.dnx.yahoo.com [216.115.96.34] 17 189 ms 95 ms 145 ms ae-6.pat1.gqb.yahoo.com [216.115.101.195] 18 114 ms 98 ms 98 ms ae-1.msr1.gq1.yahoo.com [66.196.67.5] 19 102 ms 92 ms 95 ms xe-5-0-0.clr1-a-gdc.gq1.yahoo.com [67.195.0.21] 20 93 ms 95 ms 104 ms et-17-1.fab1-1-gdc.gq1.yahoo.com [98.137.31.164] 21 96 ms 99 ms 98 ms po-16.bas1-7-prd.gq1.yahoo.com [206.190.32.27] 22 84 ms 108 ms 97 ms ir1.fp.vip.gq1.yahoo.com [206.190.36.45] Trace complete.Tracing route to forum.pfsense.org [66.219.34.171] over a maximum of 30 hops: 1 1 ms 1 ms 1 ms MF28D [192.168.0.1] 2 * * * Request timed out. 3 226 ms 201 ms 236 ms 192.168.102.2 4 31 ms 24 ms 37 ms 10.128.87.9 5 22 ms 23 ms 24 ms 192.168.3.75 6 47 ms 38 ms 39 ms 192.168.3.98 7 28 ms 37 ms 39 ms 10.118.23.37 8 41 ms 34 ms 38 ms 10.118.20.129 9 32 ms 24 ms 45 ms 10.118.20.2 10 37 ms 33 ms 37 ms 24.156.157.137 11 41 ms 29 ms 37 ms 24.156.146.46 12 47 ms 39 ms 38 ms 24.156.157.113 13 43 ms 34 ms 35 ms 69.63.248.233 14 34 ms 50 ms 44 ms 24.156.144.178 15 43 ms 48 ms 52 ms 0.xe-5-2-1.pr1.chi10.tbone.rr.com [66.109.9.85] 16 80 ms 58 ms 59 ms 66.109.1.67 17 75 ms 59 ms 75 ms ae-0-0.cr0.chi30.tbone.rr.com [66.109.6.21] 18 71 ms 78 ms 60 ms ae-2-0.cr0.dfw10.tbone.rr.com [66.109.6.22] 19 79 ms 67 ms 75 ms agg3.dllatxl301r.texas.rr.com [107.14.17.137] 20 82 ms 71 ms 86 ms agg1.ausutxla01r.texas.rr.com [24.175.41.47] 21 91 ms 77 ms 64 ms tge9-5.rdrktxaz01h.texas.rr.com [66.68.0.11] 22 73 ms 76 ms 78 ms xe-0-2-0-0.RDRKTXAZ1CW.tx.twcbiz.com [97.77.0.53 ] 23 84 ms 101 ms 78 ms xe-1-3-0.ausxtxti1zw.tx.twcbiz.com [24.73.240.16 3] 24 93 ms 98 ms 84 ms rrcs-24-227-210-130.sw.biz.rr.com [24.227.210.13 0] 25 101 ms 98 ms 89 ms 66.219.34.171 26 100 ms 90 ms 88 ms 66.219.34.171 Trace complete.Whats interesting, the 2nd tracert to yahoo.com shows more timeouts, yet the site loads perfectly. In the first one it doesn't load at all.
-
Im currently on this network, and forum.pfsense.org was the first domain I loaded after connection. I am now unable to load ANY site except forum.pfsense.org, which works perfectly fine. If I reconnect my connection, and load a different domain, it will be the only one I will be able to access for that session. What is going on here?
-
Hmm, weird indeed.
MTU seems like a likely suspect, that can result in some websites being inaccessible, though it doesn't explain why you can get through once and then to nothing else.
I would definitely try changing your LAN subnet to something definitely not in use somewhere in your ISPs network. Perhaps: 172.16.1.0/24
When you are stuck accessing only one website can your still ping other sites? You seem to be able to traceroute to other places. :-\Steve
-
Can you try pinging with packet size 1500 or above something in Intenret?
-
Hmm, weird indeed.
MTU seems like a likely suspect, that can result in some websites being inaccessible, though it doesn't explain why you can get through once and then to nothing else.
I would definitely try changing your LAN subnet to something definitely not in use somewhere in your ISPs network. Perhaps: 172.16.1.0/24
When you are stuck accessing only one website can your still ping other sites? You seem to be able to traceroute to other places. :-\Steve
I will try that, however 192.168.1.xx is the default LAN subnet that comes the modem. I just moved it over to pfSense. I tried using 192.168.0.xx which yielded no results.
Yes, Im still able to ping and resolve all other sites, just not load them. I tried doing packet capture, and then accessing a site that doesn't work. I do get a response packet from the remote server as I see the response headers in the log, however it only sends one packet. Not sure where the rest of them are.
Can you try pinging with packet size 1500 or above something in Intenret?
Says packet needs to be fragmented, thats about it.
-
I take it back about what works and doesn't. I reset MTU settings back to defaults, and I started getting more predictive behavior. Some sites just work, while others do not. Here are some that do:
forum.pfsense.com, doc.pfsense.oeg (but not www.pfsense.org)
google.com
di.fm
rona.ca
highscalability.com
arstecica.com (but not their static content server at cdn.arstechnica.net)Sites that don't work:
arduino.cc
stackoverflow.com
en.wikipedia.org
serverfault.com
bbc.co.uk
facebook.comWhen I traceroute them, the trace looks the same, which seems like it doesn't reach my LTE gateway at all, and gets "trapped" in pfsense.
Pinging facebook.com [173.252.110.27] with 32 bytes of data: Request timed out. Request timed out. Reply from 192.168.99.1: Destination host unreachable. Reply from 192.168.99.1: Destination host unreachable. Ping statistics for 173.252.110.27: Packets: Sent = 4, Received = 2, Lost = 2 (50% loss), C:\Users\Gerty>tracert facebook.com Tracing route to facebook.com [173.252.110.27] over a maximum of 30 hops: 1 1 ms 1 ms 1 ms 192.168.99.1 2 192.168.99.1 reports: Destination host unreachable. Trace complete. Tracing route to bbc.co.uk [212.58.253.67] over a maximum of 30 hops: 1 1 ms 1 ms 1 ms 192.168.99.1 2 * * * Request timed out. 3 * * 192.168.99.1 reports: Destination host unreachable. Trace complete. Tracing route to arduino.cc [174.129.243.245] over a maximum of 30 hops: 1 1 ms 1 ms 1 ms 192.168.99.1 2 * * * Request timed out. 3 * * 192.168.99.1 reports: Destination host unreachable. Trace complete. Tracing route to stackoverflow.com [198.252.206.16] over a maximum of 30 hops: 1 1 ms 2 ms 1 ms 192.168.99.1 2 * * * Request timed out. 3 * * 192.168.99.1 reports: Destination host unreachable. Trace complete.I moved my LAN to 192.168.99.0/24 subnet.
-
Ok, found the issue. I had the WAN interface set to 192.168.0.150/1 instead of /24. As soon as I changed that, everything started working.
