SG-3100 restore initially worked, now cannot get it working
-
Hello:
I recently restored an SG-3100 with files from support and according to their instructions. Initially reinstall worked. Then as changing settings, the device had some issues. At first was pretty sure it was something I had done so restored from USB again.
However, as I continued to reconfigure everything, this happened multiple times with settings that did not initially cause problems. The second to last time I restored from USB cannot get DNS working no matter what I tried. I restored again one more time, left mostly default except to connect to the device from a laptop. The diagnostic tests fail including ping and DNS lookups from the SG-3100.
I found your troubleshooting steps and have listed below what I did and my settings which, as mentioned, are mostly default.
WAN
Check that the IP address is correct:
- DHCP from ISP
- Gets an IPv6 but not an IPv4 IP from ISP.
- When I connect a laptop to the cable modem it gets an IPv4 address so not sure why SG-3100 doesn't get one.
Check WAN IP has correct subset mask:
- Set to 255.0.0.0 by ISP for IPv4.
- No mask for IPv6
Check WAN has a gateway
- Didn't see any gateways (see next steps)
Check status of Gateways
- IPv6 which has an IP shows offline
- IPv4 which does not have an IP shows pending
Check WAN gateway is set as default
- IPv6 and IPv4 Default gateways were set to automatic. I hard coded them respectively and that filled in the default column in the gateways list that was previously blank. Then I went back to interfaces and the gateway shows up for IPv6 but not IPv4.
- (Why weren’t these set by default via Automatic?)
- DNS Lookup still doesn’t work.
LAN
Check that LAN IP is correct- Did not change - is default (192.168.1.1)
Check that subnet mask is correct
- Did not change and do not see anything that says subnet mask but I’m guess is after the IP? (/24)
- Status > Interfaces shows subnet mask 255.255.255.0 for LAN.
Check that the LAN does NOT have a gateway.
- Correct
Does not block Private
- Correct
Does not block Bogon
- Correct
Firewall
Firewall Logs- Nothing blocked in logs
Lan (IPv4 only which worked before this last refresh)
- 53 TCP/UDP anywhere
- 80/443 TCP anywhere
- ICMP anywhere
- Block everything else so I can see this tagged in logs.
- All have logs enabled
Lan Firewall Rules - destination ANY
- Yes, for now. Will be more specific for DNS later and that worked in the past.
Lan Gateway
- Gateway column is *
Outbound NAT
- all default, no problems.
Diagnostic Tests
- All fail - that’s why I’m here…
Client
- Not even there yet. Should work once the diagnostic tests on the SG-3100 work. Client works on other networks.
Captive Portal
- Not enabled
Other
- DHCP was setup and working but now fails again (which it randomly seems to do). Sets client to a 169.x.x.x address instead of the default 192.x.x.x address. I can manually set the IP and it works.
- Want to use DNS forwarding. Used that in the past and worked fine. Turned off DNS resolver and set to that (didn’t work with resolver either)
- System > General - tried setting Google DNS Server IPs for IPv4 and IPv6
- Unchecked and checked Allow DNS Server list to be overridden - neither worked
- Disable DNS Forwarder for the firewall checked. Do not want that want the queries to go out to the actual DNS servers.
Thanks in advance.
-
@haxx said in SG-3100 restore initially worked, now cannot get it working:
DHCP from ISP
ok, but ....
@haxx said in SG-3100 restore initially worked, now cannot get it working:
Gets an IPv6 but not an IPv4 IP from ISP.
You can stop right there.
"DHCP client running" on WAN is the defalt mode. Nothing to be done by you. It should work out of the box.
If not : read the doc provided by the ISP again.Ok that you obtained a IPv6 but still, today, augustus 2020, it's a bit to earlt to build an IPv6 network only. It can be done, though.
@haxx said in SG-3100 restore initially worked, now cannot get it working:
Sets client to a 169.x.x.x address instead of the default 192.x.x.x address.
Noop.
A device auto assigns itself a 169.a.b.c IP if it can't obtain IP/mask/gateway/DNS/etc from a DHCP server.
Normally, your devices are connected to pfSense using a switch. When a device throws (broadcasts) out a DHCPDISCOVER, this request should be seen by the DHCP server running on LAN, and it should answer.If this :
@haxx said in SG-3100 restore initially worked, now cannot get it working:
Google DNS Server IPs for IPv4 and IPv6
When every things work great, if possible for days or even weeks then you can try to
mess up....start forwarding to Google or who ever (but why should you ? you have to ? have some kind of a contract with them ? Google is a resolver : you already have one locally - why use the remote one ? )@haxx said in SG-3100 restore initially worked, now cannot get it working:
Lan Firewall Rules - destination ANY
Yes, for now. Will be more specific for DNS later and that worked in the past.
is what you have as a firewall rule, then what is this :
@haxx said in SG-3100 restore initially worked, now cannot get it working:
Lan (IPv4 only which worked before this last refresh)
53 TCP/UDP anywhere
80/443 TCP anywhere
ICMP anywhere
Block everything else so I can see this tagged in logs.
All have logs enabled?
The forum accepts cutted / pasted images :
Like :
The firewall rule I've shown is tested for a decade.
It works just great, never had an issue.My WAN rules : it's empty. No rules. Works great also.
Btw : nice write up. That's a good thing
What's missing : the list with the answers : your logs ....
Logs are very important . Way more important as what the Dashboard can show you.
The dashboard is for the day when everything works. The logs are there for the other days ;) -
Thanks for your quick response.
After thinking about this further, I hooked up a laptop to the cable modem and remembered that in order for it to work and allow me to connect to the Internet via my laptop, I have to set a static IP, subnet mask, and router IP. I used the .100 address for the netgear IP as the router. That allowed my laptop to connect to the Internet.
So, I presume the problem has either to do with my cable provider (Xfiinity/Comcast) or a netgear modem that is not correctly getting or passing the DHCP info for IPv4 to the SG-3100. I saw various posts about this but the issue is still not exactly clear to me. Some posts say to disallow a lease from the .100 address. I'm still thinking this through and looking for answers, if anyone has any.
Responses to recent below:
"DHCP client running" on WAN is the defalt mode. Nothing to be done by you. It should work out of the box.
If not : read the doc provided by the ISP again.I understand, I am just answering each of the questions in the troubleshooting document.
Ok that you obtained a IPv6 but still, today, augustus 2020, it's a bit to earlt to build an IPv6 network only. It can be done, though.
I don't actually want IPV6. Just leaving the defaults and reporting what happens.
Sets client to a 169.x.x.x address instead of the default 192.x.x.x address.
Noop.
A device auto assigns itself a 169.a.b.c IP if it can't obtain IP/mask/gateway/DNS/etc from a DHCP server.
Normally, your devices are connected to pfSense using a switch. When a device throws (broadcasts) out a DHCPDISCOVER, this request should be seen by the DHCP server running on LAN, and it should answer.I understand this. I am just reporting what I see.
If this :
Google DNS Server IPs for IPv4 and IPv6
When every things work great, if possible for days or even weeks then you can try to mess up.... start forwarding to Google or who ever (but why should you ? you have to ? have some kind of a contract with them ? Google is a resolver : you already have one locally - why use the remote one ? )
Why: To prevent new types of Emotet attacks that send C2 traffic over DNS instead of HTTPS in phishing attacks, for one thing. This is one of the most prevalent types of malware getting on people's systems via phishng attacks right now. Locking down firewall rules to a specific DNS server prevents this (but not HTTPS C2 traffic). That's just one example. I work in cybersecurity.
I'm also interested in testing the PFSense for various DNS rebinding attacks (yes, I see the setting there) in relation to a research paper I'm working on (not specific to PFSense).
Lan Firewall Rules - destination ANY
Yes, for now. Will be more specific for DNS later and that worked in the past.
is what you have as a firewall rule, then what is this :
Lan (IPv4 only which worked before this last refresh)
53 TCP/UDP anywhere
80/443 TCP anywhere
ICMP anywhere
Block everything else so I can see this tagged in logs.
All have logs enabled?
Outbound to ANY is setup in the rules I have. I also ran with any/any to all locations initially and did not resolve the problem. I've run it this way before and it worked fine. Nothing is blocked in firewall logs. This is not the issue.
Leaving your LAN open to ANY/ANY is a security risk. Zero-trust networking is a best practice. I leave the details as an exercise for the reader.
My WAN rules : it's empty. No rules. Works great also.
Btw : nice write up. That's a good thing
Thanks
What's missing : the list with the answers : your logs ....
*Logs are very important . Way more important as what the *
Dashboard can show you.
The dashboard is for the day when everything works. The logs are there for the other days ;)Not sending logs for security reasons at this point. If I do will be redacted. All logging is on for firewall rules and nothing is blocked. Also checked via Wireshark, TCPDump, Packet capture in PFSense. Would some other logs be helpful? There are many other logs in the pfsense such as DHCP, system logs, etc.
-
@haxx Usually, when you have WAN trouble with getting an IP address from your modem, especially cable modems, it is because the cable modem has "locked down" to the last known MAC address that successfully connected to it.
Have you tried powering down the cable modem at any time in your troubleshooting steps? I don't see it mentioned anywhere. This process usually allows the cable modem to "lock down" onto a new MAC address after powering back up, in this case, the WAN port of your SG-3100 box.
Jeff
-
No that is a very good point and is my next step.
Also came back to mention when I try to release/renew the IP in PFSense/SG-3100 I see IPv6 traffic in packet capture but not IPv4. So no IPv4 traffic between cable modem and PFSense according to PFSense packet capture.
I'll try this first a restart leaving it off the appropriate amount of time, then a reset if that doesn't work, and report back. Thanks.
Side note/question:
I presume only the WAN interface needs to interact with the ISP and/or cable modem, not the LAN interface so locking out the ISP addresses on the LAN interface should have no effect. Was thinking about this as I recently looked into local DHCP traffic and noticed that Wireshark does not report a failure when a firewall rule allows access to the DHCP server IP/Router specifically but not a broadcast address. One request in the DHCP protocol goes to a broadcast address and if that is blocked, Wireshark and I presume firewall logs don't show it. I have all traffic allowed on the WAN currently to allow getting the IPv4 address from the ISP but disallow access traffic from ISP to LAN. I believe this should be correct and not affect obtaining an IPV4 address from the ISP, but since no IPV4 traffic in packet captures between the two, this is a moot point at the moment.
-
FYI, you don’t have to create or modify any firewall rules on the WAN interface for the pfsense box to get an IP address from your ISP modem.
-
Hooray! Restating the cable modem fixed the problem. I left it unplugged for a couple minutes to clea out the ISP settings.
Before connecting the SG-3100 to the modem I restored it once more and turned off all IPV6 on the WAN interface, unchecking the box to allow IPv6 and DHCP6. I also set up the firewall rules mentioned earlier and set up DNS forwarding instead of resolver locked to Google 8.8.8.8 and Cloudfoare 1.1.1.1. I set up NTP to use the NIST NTP service instead of a random pool of servers. If you wonder why there’s a great talk from DefCon on NTP hacking.
Then I plugged the SG-3100 into the modem. The ISP correctly assigned an IPv4 address to the box. I do not want to use IPv6 because I don’t need it and one less thing to monitor. It also routes MAC addresses which I don’t like, even though no one should be using those for security. Some vendors still do.
I understand that leaving the defaults may make something work, like leaving default passwords so you don’t forget them - one of the most common sources of compromise of IOT devices. However the goal is not simply to make it work, it is to come up with a secure solution.
I used to work for a Firewall vendor. I am trying to understand exactly what is required to be open to the ISP so I don’t break anything, but apply Zero Trust networking rules (security best practice.)
I’ll probably post a separate question about all of that later as doing some research in that area. The issue in this ticket is resolved.
Thanks everyone.
-
@haxx said in SG-3100 restore initially worked, now cannot get it working:
I am trying to understand exactly what is required to be open to the ISP so I don’t break anything, but apply Zero Trust networking rules (security best practice.)
You don't have to open anything to the ISP to get your pfsense box working on the internet. The only reason to open any ports, and you need to be REALLY CAREFUL with that, is to get into your LAN stuff from outside on the web. It's always best to use a VPN setup to do this.
Jeff
-
@akuma1x I understand this, and you are pretty much reiterating what I wrote. Thanks.
