sg-1100 breakage: exposing internal devices to the WAN
-
Hi again,
I know no one is paying attention right now, but for reference (in case anyone stumbles across this thread in the future) here is my (probably final) update.
I attempted the built-in factory reset, but it was going into a boot loop rather than restoring. So I got a full firmware image from Netgate, and installed that. Everything seemed to be working normally. I then loaded in the config backup I had from about a week before the failure occurred. When it came back up, it was still working normally. I've left it running for several days, and still no issues.
I must therefore surmise that it was a software fault, and that the hardware is fine. So, my skepticism stands, and I will need to seriously consider whether to risk continued usage of pfSense. I am currently running a different firewall platform at the edge, and although I don't like the customization options of that as much as pfSense, it will probably be awhile before I will dare trying pfSense as the primary defense of an attack surface. I'm sure that what I'm seeing here is an an edge case, but it should be simply impossible for this sort of failure to occur (in my opinion). Unfortunately, the "bad guys" probably read forum posts like this too, so I hope that by discussing this, I have not put more users in danger.
Hopefully the platform will improve further over time, and such possible faults will be weeded out.
-
@cyberminion said in sg-1100 breakage: exposing internal devices to the WAN:
I know no one is paying attention right now
I presume many saw you messages.
I could rewrite it in a basic : "my SG-1100 became a simple 5$ switch".
Now that would be a major security issue.I don't have an SG-1100, and I can only use the manual as a reference.
I guess it's possible to 'add' the NIC's together so they behave as a switch using software (GUI) settings, as I saw many forum posts from people trying to do so : adding the LAN and OPT1 port together. to gain an extra switched port.That such a setup, WAN, LAN and OPT are all switched, could happen by 'accident' ?
You're the first .... -
@gertjan I guess I'll play along. Disclaimer I do not have an SG1100, but do have an SG2100 which I believe has a similar hardware configuration.
All LAN and OPT ports are on a switch, by default they are all on the same network segment, just like a $20 network switch you get at your favorite store.
If you want them to provide different network segments (which by the first picture seems to be the desired configuration) you have to put a bit of effort in creating distinct vlans and such (I followed a very nice straightforward guide from Netgate website).
Have no idea how that could happen magically, but who knows? -
By any chance did this checkbox on "System/Advanced/Firewall & NAT" get checked?
Ignore as needed! :) -
@gertjan said in sg-1100 breakage: exposing internal devices to the WAN:
I could rewrite it in a basic : "my SG-1100 became a simple 5$ switch".
Now that would be a major security issue.Well, that would be an accurate title, wouldn't it? Ugh.
I guess it's possible to 'add' the NIC's together so they behave as a switch using software (GUI) settings, as I saw many forum posts from people trying to do so : adding the LAN and OPT1 port together. to gain an extra switched port.
By default "out of the box", this device was using isolation to prevent direct access between the two. That is what I wanted, so I left it that way, and set different subnets for each one. Then I created firewall rules to block access between then, just to be doubly-sure.
This is probably why people were asking about it, because linking the two ports is not a default configuration.
That such a setup, WAN, LAN and OPT are all switched, could happen by 'accident' ?
I don't think so, since it was working with isolation intact, then apparently on its own, decided to self-destruct. The fact that the entire OS seemed to vanish would imply a greater problem. It was acting as if the OS was no longer booting, (thus the rapid start-up time), but instead of failing secure, it just opened up all the ports and made it a free-for-all.
Anyway, thanks for the input!
-
@mer said in sg-1100 breakage: exposing internal devices to the WAN:
@gertjan I guess I'll play along. Disclaimer I do not have an SG1100, but do have an SG2100 which I believe has a similar hardware configuration.
Yes, the two models are very similar. The maid difference is that the 2100 has more ports, which are switched together. On the SG-1100, the Opt1 port was disabled be default, and I needed to create a vlan for it in order to bring that port online. (I also added firewall rules to block access between subnets)
If you want them to provide different network segments (which by the first picture seems to be the desired configuration) you have to put a bit of effort in creating distinct vlans and such (I followed a very nice straightforward guide from Netgate website).
I did put in a sizable about of effort on this device to do exactly that. After a lot of tinkering, I got it just right. Since then, I'm used the config file from this box as a template to set up others, and those have worked fine as well. It has worked very well for a couple years, since then.
Have no idea how that could happen magically, but who knows?
Same. Perhaps it was memory corruption for some reason? I would still expect a security product to fail secure, though. Or maybe a nation state decided that they needed to get in, and this little potato was collateral damage...haha!
-
@provels said in sg-1100 breakage: exposing internal devices to the WAN:
By any chance did this checkbox on "System/Advanced/Firewall & NAT" get checked?
Ignore as needed! :)Good point, that could certainly cause an issue. In this case, no it did not. There was no checkbox, web UI, or even OS for me to access in order to check such a box.
Thank you for the suggestion, though!
-
@cyberminion I still have nothing more than speculation, but from the description on netgate website all the ports on SG1100 are on a switch. The sg2100 looks like all the "LAN" ports are on a switch, with the WAN on a distinct device.
I've worked at a low level with Marvell switch stuff in the past, default mode (power cycle, don't configure anything) is the device acts like an unmanaged switch.
Why am I mentioning this? There were reports on the forum (don't recall exactly where) about a bad/empty config.xml. Don't know the root cause, but folks wound up with a config.xml of 0 byte length. If that happened, I would expect that "no config to apply, just stay with the power cycle defaults".
So taking your symptom of "the sg1100 acting like a $5 unmanaged switch", it's possible that somehow the config.xml was corrupted/0 byte length as the root cause.
That's all speculation on my part, based solely on your description and what other posts I've run across. I don't know if there was ever a root causing of "why" the 0 byte config.xml, but perhaps it gives you something to monitor.
I'm simply a user of pfSense and have a couple of Netgate products that I like and work fine for me, so all this is just my opinions and speculation.
-
@mer said in sg-1100 breakage: exposing internal devices to the WAN:
So taking your symptom of "the sg1100 acting like a $5 unmanaged switch", it's possible that somehow the config.xml was corrupted/0 byte length as the root cause.
Interesting...that may be. It's too late to check now, since it has been wiped, but if I continue working with pfSense and see this again, I will keep that in mind.
I'm simply a user of pfSense and have a couple of Netgate products that I like and work fine for me, so all this is just my opinions and speculation.
Any thoughts and speculations are welcome, thanks!
So if I put pfSesne on a unit which runs separate NICs, like a device from Protectli, do you think such a config file corruption would cause it to fail in a secure way, since they are not connected to an internal switch in any way?
EDIT:
The sg2100 looks like all the "LAN" ports are on a switch, with the WAN on a distinct device.
One counter-point to that theory, is that all three ports, LAN, OPT1, and WAN were being switched together. So that means that both independent NICs were still linked somehow.
-
@cyberminion
The SG2100: the default configuration all the ports labelled LAN are on the switch. If you don't do anything all the ports are on the LAN segment. The WAN is a distinct device; default mode is like a good old WRT54G: WAN goes to the Internet, all the LAN ports in the back are switched together. If you want to create a LAN and OPT1 (your original picture) you have to do explicit configuration to create VLANs and Tagging for the different ports on the Switch itself.Unit with separate NICs. Hard to say, it may depend on how the separate NIC devices are connected. Easy to see them connected to an unmanaged switch, if there is no explicit configuration, I think again you wind up with the $5 unmanaged switch from the store.
I'm currently behind a SG2440 that has distinct NICs for WAN, LAN/OPT1/OPT2 and I am not going to break my configuration to test the theory :) ( wife would get annoyed at me )
Sometimes the switch devices let you have pullup/pulldown resistors on pins to force a configuration after power on. I don't have the Netgate schematics or the datasheets so can't say if anything like this is being done, but most switch devices I've used default to unmanaged mode after a power cycle.
If you have the serial console cable if it breaks you should be able to get to a shell and poke around. If I'm recalling correctly, basically look for a 0 byte config.xml and then look for a backup of config.xml that is non-zero length and simply copy that over to fix it.
