Changing Default Boot Device + Multi LAN with DMZ setup
-
I preemptively apologize if any of this sounds odd to the network gods on here. I currently have two issues. The first is setting the default boot device. Upon booting pfsense everything works fine, but during boot the default boot device is incorrect and from the console it prompts me to enter the correct boot device. Along the lines of:
? List valid disk boot devices <empty line=""> Abort manual input</empty>
Entering ? provides a list of devices and I having determined the correct one I enter:
ufs:/dev/ad4s1a
Which will then continue with the boot process and start pfsense. I've done a lot of searching on here and from the sound of things it sounds like I need to edit the etc/fstab to set the default boot device.
http://forum.pfsense.org/index.php?topic=28751.0
https://forum.pfsense.org/index.php/topic,41201.msg212866.html#msg212866
https://doc.pfsense.org/index.php/Boot_TroubleshootingThe issue is that I am not aware of how to do this and continually turning things off and on is not currently an option. I know this one sounds simple so any help would be greatly appreciated.
–-----------------------
My second question is in regards to setting up a multi or bridged LAN setup throughout my house with a separate DMZ. My current setup as I hope to have it:
modem --> pfsense --> LAN = 24 port managed switch (garage) / includes Dlink WLAN
OPT1 = 8 port unmanaged switch (upstairs)
OPT2 = 16 port unmanaged switch (downstairs)
OPT3 = E3000 second wireless router (upstairs / different room DMZ)The pfsense box is in a central location so having all LAN traffic flow through the 24 port switch / default lan is not an option as it is not physically possible without rewiring the whole house. I hope to have all of the LAN and DLink WLAN traffic to be able to communicate to each other while the second WLAN e3000 is setup as a DMZ for guests.
I previously had the setup run as a LAN bridge by following the information posted here:
http://forum.pfsense.org/index.php/topic,20917.0.html1. On Interfaces Assign page create another OPT interface called OPT2 (create more than one if you want to bridge physical ports too - we'll call any additional ports OPTx.)
2. Assign a physical port or a vlan port, or a PPP port to OPT2 just as a place holder.
2a. Assign physical ports to the other OPTx interfaces you've created.
3. Go to the Interfaces -> OPT2 page and click the "Enable" checkbox. (Repeat for all OPTx interfaces.)
4. Go to the Interfaces -> (assign) page and click on the "Bridge" tab.
5. Select OPT1, OPT2, OPTx (if you created additional ports for the bridge) to be members of the bridge interface and Save.
6. Go to Interfaces -> (assign) page again and select "BRIDGE0" for the LAN interface, and select the fx1 port (formerly assigned to LAN) for the OPT2 interface and Save.The only thing I had to do differently from the above was step 6. Instead I changed the LAN to BRIDGE0 and left the OPT1 interface as the port it was actually connected to otherwise nothing could connect on that interface. This worked fine for me aside from the fact that the devices on separate line were not able to ping each other.
Two days ago I had a power outage that somehow broke my LAN bridge, basically not passing internet through any interface. I power cycled all devices, but was ultimately forced to reset everything to default through the console. I thought I would just reset it the way I had it and I'd be good to go, wrong... I attempted to duplicate my setup as I had it before, but it would not work. As soon as I changed the LAN interface to BRIDGE0 it would lock me out of the web GUI, I would be forced to reset the interfaces in the console, and power cycle the machine. If you combine that with the original issue of the incorrect default boot device this became annoying very quickly.
I ended up only being able to get traffic through the LAN interface and had to use an extra 20 foot CAT5e cable and a 5 port 10/100 switch to make the line from 8 port switch long enough to plug into the 24 port switch to allow upstairs to function. While this does currently function it is an unacceptable long term solution so I'd like to get this figured out.
At this point I've been doing a lot of research and I'm a little unsure of how to attack this. I've read a lot of bridge and DMZ guides, but am still confused. I'm not sure if the bridge is really necessary or if I could set firewall rules to allow the multiple LAN interfaces to communicate, but then combining that with the DMZ makes it more confusing.
TLDR
Problem 1:
Change default boot device from unknown setting to ufs:/dev/ad4s1a so pfsense will boot automatically without prompting.Problem 2:
Setup multi lan interface that can communicate to each other with a separate DMZ. VLANs and passing all traffic through LAN is not an option.
Any help would be greatly appreciated.
-
Hey.
You can edit the fstab any number of ways. From the webgui Diagnostics: Edit file: click browse and navigate to /etc/fstab.
From the console using the easy editor (or vi if you're a masochist!):ee /etc/fstab
In the fstab you will see a list of mounted file systems. You will want to change the root and swap references appropriately. You haven't listed what it was trying to mount before but you know where root is. The swap slice will also be on ad4.
Have read through this post bout creating a bridge:
http://forum.pfsense.org/index.php/topic,48947.msg269592.html#msg269592
It seems like you didn't move the bridge filtering.
So your want what is your current LAN interface bridged with OPT1 and OPT2 but keep OPT3 separate yes? I would reocmmend you add firewall rules, if you haven't alreday, to OPT3 to allow you to access the webgui. Then setup the bridge from the OPT3 network. That way you are far less likely to be locked out as you change the interface configs.Steve
-
Hey.
You can edit the fstab any number of ways. From the webgui Diagnostics: Edit file: click browse and navigate to /etc/fstab.
From the console using the easy editor (or vi if you're a masochist!):ee /etc/fstab
In the fstab you will see a list of mounted file systems. You will want to change the root and swap references appropriately. You haven't listed what it was trying to mount before but you know where root is. The swap slice will also be on ad4.
That did indeed work and it does automatically boot, thank you. Changes:
Old # Device Mountpoint FStype Options Dump Pass# /dev/ad6s1a / ufs rw 1 1 /dev/ad6s1b none swap sw 0 0 New # Device Mountpoint FStype Options Dump Pass# /dev/ad4s1a / ufs rw 1 1 /dev/ad4s1a none swap sw 0 0
Have read through this post bout creating a bridge:
http://forum.pfsense.org/index.php/topic,48947.msg269592.html#msg269592
It seems like you didn't move the bridge filtering.
So your want what is your current LAN interface bridged with OPT1 and OPT2 but keep OPT3 separate yes? I would reocmmend you add firewall rules, if you haven't alreday, to OPT3 to allow you to access the webgui. Then setup the bridge from the OPT3 network. That way you are far less likely to be locked out as you change the interface configs.Steve
Yes, I read that before I before I setup the first bridge and couldn't get it to work. As odd as it sounds the bridge I had setup previously worked fine without bridge filtering, but it cut my download speed in half. As soon as I enabled bridge filtering, speed returned to normal: http://i.imgur.com/PcC9FBn.png
Currently it is set as follows:
System Tunables
Interfaces:
Bridge:
All interfaces are assigned, enabled, and block bogon networks. The LAN interface has the default LAN settings of Static IPV4 and Track Interface with the default IP. All other interfaces are simply enabled without having an IPV4, IPV6, or IP.
All interfaces have firewall rules passing IPV4 and IPV6 traffic as the basic * * * * * none .
As soon as I change the LAN interface to BRIDGE0 it locks me out of the web GUI and cuts out WLAN traffic. LAN traffic stays running, but the OPT interfaces do not work. Adding the LAN to the bridge does not work as it returns:
You cannot set port bridge0 to interface LAN because this interface is a member of bridge0.
If you need any additional screen shots or info let me know.
-
I assume you have renamed LAN as TPLINK24? But is still has the usual default LAN rules?
Where/how are you connected to the pfSense box when you set LAN to bridge0? I strongly advise you to be connected via an interface that isn't (and isn't going to be) part of the bridge when you set this up. I've set up bridging a number of times and I'm familiar with what pfSense will do and I still manage to lock myself out if I'm not really paying attention. ::)
Your device mount point for the swap partition should probably be:
/dev/ad4s1b
The layout of the disk itself has not changed only where it is attached to the computer.
Steve
-
I assume you have renamed LAN as TPLINK24? But is still has the usual default LAN rules?
In LAN bridge:
TPLINK24 = LAN
BEDROOM = OPT1
NETGEAR16 = OPT2Not in LAN bridge:
GUESTDMZ = OPT 3TPLINK24 / LAN Still retains all default LAN rules and IP. I did change the IP of the switch to .1.2 so I could access it as its default IP is the same as pfsense's.
Where/how are you connected to the pfSense box when you set LAN to bridge0? I strongly advise you to be connected via an interface that isn't (and isn't going to be) part of the bridge when you set this up. I've set up bridging a number of times and I'm familiar with what pfSense will do and I still manage to lock myself out if I'm not really paying attention. ::)
I was using one of two wireless connections on separate interfaces, but they would both lock out when I was working on setting up the bridge. I've enabled another interface and used a cabled interface through that to get it semi working which, is where I'm at now.
Currently, after a few reboots I have the bridge semi functional. Currently, OPT1 / BEDROOM and OPT2 / NETGEAR16 are working fine with the bridge, but the LAN / TPLINK24 is not. At this point I've tested adding and removing the LAN interface from the bridge and it doesn't seem to make a difference in regards to functionality. LAN is setup as static IPV4, track interface IPV6, has the default IP, and is acting as the default DHCP server. When I attempt to set LAN to none / none it returns an error that LAN is acting as the default DHCP server and that requires a static IP.
At this point I rigged it to work by again running extra CAT5e cables between switches to connect them, and now it goes pfsense –> 16 port switch --> 5 port switch --> LAN ... = ridiculous. I'm guessing I should just create an additional interface with what was the default LAN network adapter, RE0 and add it to the bridge, but I didn't have to do that before to get it to work. As a side note I have 17 computers on the LAN interface so keeping that running is the number one priority right now.
Your device mount point for the swap partition should probably be:
/dev/ad4s1b
The layout of the disk itself has not changed only where it is attached to the computer.
Steve
Fixed, thank you.
-
There are a whole load of ways you could conceivably setup the bridge and have it work. The scheme I outlined in the post I linked to above is what makes the most sense logically and conceptually to me. The behavior of bridge interfaces changed between 1.2.3 and 2.X so that threw additional confusion into the mix. Since in 2.X the bridge interface itself is assignable I prefer to have that assigned to LAN. Hence the default dhcp server etc operate on the bridge interface itself and all the bridge member interfaces are configured identically.
Here's the order I would do things in, whilst connected via OPT3.Change the sysctls to move bridge filtering. Do this first because when you create the bridge it will inherit the properties at that point. If you didn't then just reboot, the bridge is setup at boot time with the new settings.
Create the bridge and add OPT1 and OPT2 to it.
Now add the new interface. It will appear as OPT4 with bridge0 assigned to it.
Now change assignments of LAN to bridge0 and OPT4 to re0.
Now add OPT4 to the bridge.Steve
-
I setup OPT 4 as TPLINK24 and changed the LAN to LANBRIDGE. Then, I added OPT 4 / TPLINK24 to the bridge interface, rebooted, and all three interfaces are on now through a wired connection. Thank you.
Now, as it is set I can ping as follows:
pfsense console: WAN / pfsense
OPT 1 / Bedroom: WAN / pfsense
OPT 2 / Netgear16: WAN / pfsense
OPT 3 / DMZ = not setup
OPT 4 / TPLINK24: WAN / pfsenseNotes on OPT4 / TPLINK24:
Pinging .1.2 which is the 24 port switch = Destination host unreachable
Pinging .0.1 which is a wireless router on the switch = request timeout
Pinging .1.118 which is a computer on the same switch = request timeoutIn short regardless of the interface and connection I am only able to ping the WAN and pfsense. All interfaces have rules that pass all IPV4 and IPV6 traffic. Outbound NAT is set to automatic, but it had not created any rules.
-
Ok, I'm slightly confused. I'm not quite sure where you were pinging from. :-\
If the three interfaces (Bedroom, Netgear16 and TPLINK24) are now all bridged they should all be in the same subnet with devices attaches to any of them receiving an IP from the pfSense DHCP server. Any device on any of them should be able to ping any other device attached to them.
If that isn't the case then check the firewall logs. With the sysctls in place to move filtering from the bridge memebers to the bridge interface there shouldn't be any firewalling between the bridge member interfaces and hence nothing in the logs.Your TP-LINK switch may have settings in place since it's a managed switch (yes?). They could be doing something.
The wireless router at 192.168.0.1 is attached directly to the TP-LINK switch? Why is it not in the same subnet? If it is still routing (rather than just acting as an AP) then you will not be able to ping it's LAN IP. If that's the case you should reconfigure it as an AP only unless you have a good reason not to.
https://doc.pfsense.org/index.php/Use_an_existing_wireless_router_with_pfSenseSteve
-
Ok, I'm slightly confused. I'm not quite sure where you were pinging from. :-\
If the three interfaces (Bedroom, Netgear16 and TPLINK24) are now all bridged they should all be in the same subnet with devices attaches to any of them receiving an IP from the pfSense DHCP server. Any device on any of them should be able to ping any other device attached to them.
If that isn't the case then check the firewall logs. With the sysctls in place to move filtering from the bridge memebers to the bridge interface there shouldn't be any firewalling between the bridge member interfaces and hence nothing in the logs.If I am on any interface I can only ping the local IP, pfsense, and the WAN. All of them react the same. When I look at the firewall logs it only shows the last 50 entries, but… Those entries start at Jan 20 22:32:28 and end at Jan 20 22:31:13 . It looks like hieroglyphics to me, but if a screenshot of the log would help I'll take one.
Your TP-LINK switch may have settings in place since it's a managed switch (yes?). They could be doing something.
Yes, it is a managed switch. I have a feeling that the default subnet on the switch is different and may be causing issues.
The wireless router at 192.168.0.1 is attached directly to the TP-LINK switch? Why is it not in the same subnet? If it is still routing (rather than just acting as an AP) then you will not be able to ping it's LAN IP. If that's the case you should reconfigure it as an AP only unless you have a good reason not to.
https://doc.pfsense.org/index.php/Use_an_existing_wireless_router_with_pfSenseSteve
I didn't do this purposely, but it may end up working out better. The plan with the second router was to use it for my parent's house as they live directly across the street from me. I figured if I could setup the second router as a network separate from the rest of my traffic I could setup an access point at their house to amplify the signal. They only use wireless devices so they could cancel their internet service and save some money every month. They've done enough for me so I try to help out in little ways when I can. I was going to setup that interface as a DMZ, but I may just add the interface to the bridge, and have the router set a separate subnet. It probably wouldn't hurt to filter their traffic as well. I will use that to setup the wireless router that I have setup for me though. Thanks.
-
Yes please post screen shots of your firewall rules and firewall logs.
If you can't access machines on other interfaces, even other bridged interfaces, then it sounds a lot like the sysctls haven't worked correctly. Once the filtering has been moved to the bridge interface the switches should appear exactly as though they were physically daisy chained.Steve
-
I realized that I reset the TPLINK switch in an attempt to see if that was the issue. The default IP for the is the same as pfsense so I can't currently log in to check it without moving some lines around.
Firewall rules:
Logs:
-
Hmm, interesting.
So all of that blocked traffic is IPv6. It is DHCPv6 and Link-local Multicast Name Resolution.
There are several interesting things about that. If the filtering has been moved from the bridge memebers then why is anything being blocked on TPLINK24, which is a bridge memeber? Why is that traffic being blocked at all, it's legitimate traffic?
The answer to the second part of that is probably that you have 'block bogons' set on all your internal interfaces. The special destination IPs used for those protocols are probably in the reserved list. You should uncheck 'block bogons' except on WAN. That leads to another question. Usually they wouldn't be checked by default, did you check that yourself?None of that should block traffic between clients though. I take it either you weren't attempting to connect between switches or the logs filled too fast to see it.
Steve
-
Hmm, interesting.
So all of that blocked traffic is IPv6. It is DHCPv6 and Link-local Multicast Name Resolution.
There are several interesting things about that. If the filtering has been moved from the bridge memebers then why is anything being blocked on TPLINK24, which is a bridge member? Why is that traffic being blocked at all, it's legitimate traffic?
The answer to the second part of that is probably that you have 'block bogons' set on all your internal interfaces. The special destination IPs used for those protocols are probably in the reserved list. You should uncheck 'block bogons' except on WAN. That leads to another question. Usually they wouldn't be checked by default, did you check that yourself?None of that should block traffic between clients though. I take it either you weren't attempting to connect between switches or the logs filled too fast to see it.
Steve
The bridge contains; OPT1 / BEDROOM, OPT2 / NETGEAR16, and OPT4 / TPLINK24 . Most of the traffic comes through OPT4 / TPLINK24 that has 17 Windows computers connected. I originally checked block bogon networks on all of the interfaces. I just removed it, but I am still unable to ping outside of the current interface.
I noticed originally looking at the logs that it was all IPV6 traffic that was getting blocked, but the DUID (thanks for the links) were what looked confusing to me. The logs fill up almost instantly although I have watched them while trying to ping an outside interface and it looks exactly the same. If you'd like a screenshot of that as well I can take one if it would help.
In testing all possibilities I thought it might be something with the switch (This is the exact one I own: http://www.newegg.com/Product/Product.aspx?Item=N82E16833704152 ) , but looking into it has the same default IP and subnet as pfsense / what I used so there shouldn't be any issues there. Could it be something with spanning tree settings? I clicked, " Enable spanning tree options for this bridge" when I had the ethernet cables crossed through multiple interfaces and pfsense trying to troubleshoot issues. Perhaps it isn't playing nicely with the bridge?
-
@Beaflag:
I originally checked block bogon networks on all of the interfaces. I just removed it, but I am still unable to ping outside of the current interface.
Is your firewall log still filling with dhcpv6 traffic?
Spanning tree shouldn't cause problems but it shouldn't be necessary either so I'd disable it.
Steve
-
@Beaflag:
I originally checked block bogon networks on all of the interfaces. I just removed it, but I am still unable to ping outside of the current interface.
Is your firewall log still filling with dhcpv6 traffic?
Spanning tree shouldn't cause problems but it shouldn't be necessary either so I'd disable it.
Steve
No, the firewall logs aren't filling up anymore. The only traffic that appears to be getting blocked is from the WAN. Here is a new screenshot of the firewall logs.
I've also disabled spanning tree on the bridge.
-
Ok, that's good. So still not able to ping (or otherwise contact) devices attached to another switch that is part of the bridge? And nothing appearing the logs when you do attempt to?
Hmmm. ???
Steve
-
Ok, that's good. So still not able to ping (or otherwise contact) devices attached to another switch that is part of the bridge? And nothing appearing the logs when you do attempt to?
Hmmm. ???
Steve
That's exactly what has happened. I just tried to ping from one interface to another, I get, "Request timed out," and nothing shows up in the firewall logs. The only interface that is having traffic blocked now is the WAN which has had a few more items have pop into the firewall log since I took that last screenshot.
-
Hmm, hard to say why that isn't working. ???
Lets do a test to make sure that firewall rules are being applied in the correct place.
Add a firewall rule on the LANBRIDGE interface that blocks IGMP (ping) traffic with destination 8.8.8.8. Then verify that you can't ping 8.8.8.8 from any client on any of the bridge member interfaces. Check that the blocked traffic appears in the firewall log.It's just possible that the TP-Link switch is causing an issue here. Have you tried pinging between devices on the BEDROOM and NETGEAR16 interfaces?
After that I think it will be time to run some packet captures to see where the traffic is going.
Steve
-
Hmm, hard to say why that isn't working. ???
Lets do a test to make sure that firewall rules are being applied in the correct place.
Add a firewall rule on the LANBRIDGE interface that blocks IGMP (ping) traffic with destination 8.8.8.8. Then verify that you can't ping 8.8.8.8 from any client on any of the bridge member interfaces. Check that the blocked traffic appears in the firewall log.It's just possible that the TP-Link switch is causing an issue here. Have you tried pinging between devices on the BEDROOM and NETGEAR16 interfaces?
After that I think it will be time to run some packet captures to see where the traffic is going.
Steve
Agreed, I'm confused as to why it isn't working either. I just set a firewall rule on LANBRIDGE that blocks IPV4 traffic / IGMP from any interface to any interface and set it as the highest priority.
Note: Power just went out and it actually saved my post while I was typing it. PFSense actually restarted properly as well after changing the boot device. Thanks.
With the rule in place I am still able to ping pfsense .1.1 from my desktop .1.119 on the BEDROOM interface. It doesn't seem to be blocking anything, but I may have it setup wrong:
I think that somehow the items that are supposed to be bridged are ending up on / with separate DHCP ranges (not sure on exact term). Despite any issues that may be originating from the TPLINK switch / interface as you mentioned I should be able to ping from the BEDROOM interface to the NETGEAR16 . With my laptop plugged in to the NETGEAR16 interface it sets the IP differently. On my laptop ifconfig returns:
inetaddress: .1.28
Bcast: .1.255
Mask: 255.255.255.0In pfsense the DHCP server range is the default LAN of .1.10 to .1.245 . How my laptop ended up outside of that range is a mystery to me? Both of those interfaces are basic unmanaged switches, neither should be able to change what pfsense is sending them. After the power went out I dug deeper into things as everything was down, and it takes a few minutes for everything to restart. I went through all 16 machines on my TPLINK switch and found that all of the IPs lay within .1.1xx with a subnet of 255.255.255.0 . I wanted to get into the switch so I moved some things around. After getting really annoyed with it not connecting I finally pulled the manual to check that the default IP is .1.1 as it says online. Guess what?… it isn't, it's .0.1 with a default subnet of 255.255.255.0 . As of now, the subnet reports 255.255.255.0, on every machines regardless of interface, yet none of them can ping each other.
After getting everything reconnected and running I tried pinging from my desktop to multiple interfaces. It reacted exactly the same as it did before where regardless of the interface I was only able to ping pfsense and the local IP. At the same time I was watching the firewall logs and noticed that it started having IPV4 / LANBRIDGE traffic again. The weird thing I noticed was this:
After looking through every machine I didn't remember seeing a .1.118 so I went through them again to make sure. I don't currently have a .1.118 so this is rather confusing as well as the fact that it keeps sending traffic to google servers. The only thing I can think is that it was my phone or my brother's tablet as he came over to use the internet. The thing is the wireless router has its own DCHP server as I wanted to keep my family off of my network.
Noticing odd traffic I started watching the logs. I'm getting a lot of blocked WAN traffic with a couple IPs coming from China, one from Russia, a server I'm trying to connect to, and a couple other random items. At this point combining network issues with the fact my power keeps randomly shutting off I'm a little frustrated and getting tired of dealing with it. Any help you could provide would be greatly appreciated.
-
.1.28 is within the range .1.10-.1.245 I see no problem there.
I typo'd that firewall rule sorry. Ping is ICMP (not igmp). :-[
So you appear to have several devices on your network that are not being handled by the pfSense DHCP server. Your TP-Link switch is presumably on a static IP at 192.168.0.1? Good job it wasn't at .1.1 because that would have intercepted traffic intended for the LANBRIDGE interface. You say you also have a wireless router handing out IPs? The one connected to the DMZ? That is potentially bad. What range is it using? You can use an additional DHCP instance on the pfSense DMZ interface handing out IPs in a completely different range and then have everything logged and controllable from pfSense. Is the wifi router still routing/NATing?
Those firewall logs are interesting. Why was it blocking traffic from a LAN side client to an external address? You don't seem to have any rules that might do that. Do you have any floating rules? Are you running Snort?
Check the dhcp leases table to see what .1.118 was.
Just to confirm your sysctl tunables look like the attached picture? Though since you have allow all rules on every interface it shouldn't matter.
Steve
![bridge tunables.jpg](/public/imported_attachments/1/bridge tunables.jpg)
![bridge tunables.jpg_thumb](/public/imported_attachments/1/bridge tunables.jpg_thumb)