Two pfSense boxes to make a lab network accessible through the same network using vlans
-
@djay Thanks for the response, this is part of the hangup of my configuration.
The production pfSense is currently only running on the LAN interface [mxlen1] and on the second pfSense device, the network is running on the LAN interface [em1]. These are technically both tagged with VLAN0 [or 1], so I would definitely want to have one of those networks on a different VLAN so that the packets sent though the network won't be misidentified [at least I think that's how it would work].
My current hangup is that I'm trying to do what I think needs to be done as a test on the secondary pfSense to reconfiguring the network/vlans so that I could apply those same settings to the production pfSense system.
I'm trying to get away from using the main LAN interface on these systems, and migrate the networks to VLANs, but as soon as I do that on the second pfSense, everything breaks.
Here is what networks I had configured on the secondary system and what I think I need to do:
My thought is that I need to get away from using the LAN interface since that would conflict vlan-wise when with the primary pfSense box.
I have configured the Unifi devices to tag these vlans one the ports that I want those networks to be on, and before the change, the server that was being forwarded the DHCP requests and serving out DNS, worked properly, however, as soon as I put them on a VLAN with the same previously configured network [10.32.1.X], things break. The Unifi switch port is set to ALL, which I would assume doesn't have any VLAN tagging. I want to say that it the port was reconfigured to be vlan 3201 since that was the vlan that system that network was configured to? If that's the case, then none of the other vlans are going over that same port...? The server on the 10.32.1.X network serves out IP addresses, it used to serve out each respective vlan's IP range.
I'm just a baffled as to why this is all breaking, unless I need to figure out how to get all of these vlans to go through one port so that all of the systems on these vlans can communicate?
Again, just trying to avoid any potential vlan conflicts based on the right side of the above diagram.
Any and all comments and suggestions would be greatly appreciated!
-
@uquevedo
Not exactly sure what you're trying to do or what you're saying in this thread, in your after image you have the pfSense LAB router with nothing connected to it.... why have it then??
But you're mistaken in thinking you would have vlan 1 overlapping. You can use the same vlan id in different networks. If you couldn't we would've run out of vlan id's long ago, considering with that thinking, you wouldn't be able to use any vlan id I used. -
@jarhead Thanks for the response, in the left, the lab is physically a separate network with the lab pfSense separating it from the production network. I'm now wanting to logically isolate things through vlans and managed switches by moving the lab server physically somewhere else and reconnecting back to the lab unifi switch and the devices connected to it.
Below is hopefully a better representation of the before and after lab networks:
My questions about vlan 0 come from that the LAN interface on the production pfSense would potentially conflict with the vlan 0 coming from the lab pfSense box, which is why I'm trying to more granularly define each network with a specific vlan so there isn't the possibility that the traffic would co-mingle if the same vlan is defined on a port but it could mean different networks depending on which network connect [production vlan 0 or lab vlan 0].
If I could physically pull a cable from the location where I want to move the lab pfSense box to, I wouldn't have this issue at all, but I can't and I was already going to get managed switches, so I thought that I could leverage vlans to logically isolate the lab network from the production network.
-
@uquevedo Ok, I see what you're doing now.
pfSense LAN port will use whatever pvid you connect it to on your switch. Just so happens just about every switch made uses vlan1 for it's pvid.
All you have to do is set the LAB pvid to whatever you want, then plug one of those ports into the LAB LAN. -
@jarhead Thanks for the confirmation. I too thought it might be as easy as moving the LAN network to a VLAN network, but I’ve been having difficulty getting this to work. I’ll also think I need to somehow get the multiple vlans I have setup on the lab pfSense through one port onto the other switches so that the traffic is properly recognized at the destination switch where the lab systems are at.
I’m assuming that I would also need to create the equivalent vlans [but not use them] on the production pfSense so that I would be able to apply firewall rules to prevent the lab traffic from commingling with the production vlans?
Replicating what is being done with one physical cable to separate things logically is turning out to be way more complex than I anticipated.
-
@uquevedo said in Two pfSense boxes to make a lab network accessible through the same network using vlans:
If I could physically pull a cable from the location where I want to move the lab pfSense box to, I wouldn't have this issue at all, but I can't and I was already going to get managed switches, so I thought that I could leverage vlans to logically isolate the lab network from the production network.
@uquevedo If I may, if you cannot pull a cable to where the physical pfsense is, how are you connecting it to the network. You got to bear with me as its always difficult to understand other people's setup.
-
This post is deleted! -
@uquevedo said in Two pfSense boxes to make a lab network accessible through the same network using vlans:
@jarhead Thanks for the confirmation. I too thought it might be as easy as moving the LAN network to a VLAN network, but I’ve been having difficulty getting this to work. I’ll also think I need to somehow get the multiple vlans I have setup on the lab pfSense through one port onto the other switches so that the traffic is properly recognized at the destination switch where the lab systems are at.
I’m assuming that I would also need to create the equivalent vlans [but not use them] on the production pfSense so that I would be able to apply firewall rules to prevent the lab traffic from commingling with the production vlans?
Replicating what is being done with one physical cable to separate things logically is turning out to be way more complex than I anticipated.
No no, you're not moving it to a vlan. Just change the pvid on the switch alone. pfSense LAN port will use whatever pvid is et on the switch. Like I said, pretty much any switch made uses vlan1 but you can just change that to whatever you want and pfSense will still just use it.
I really think you're making this way harder than it actually is. Just reading your thoughts here has my head spinning so I can only imagine what you're going through.
-
@djay said in Two pfSense boxes to make a lab network accessible through the same network using vlans:
If I may, if you cannot pull a cable to where the physical pfsense is, how are you connecting it to the network. You got to bear with me as its always difficult to understand other people's setup.
In my before image, the network connection is going directly into the lab pfSense device and then out of the other interface directly into a unifi Switch Flex Mini. It is physically separate from my main network. If I could pull a cable to the location where I plan on moving the lab pfSense system to, I could continue this separation of networks physically. My original plan was to leverage vlans to get this lab information back to where I physically have it setup.
This lab was setup to test vlans [among other things], and I was able to basically get the different ports to be on different vlans, but now I'm second guessing myself on how that was configured and working based on other comments in this thread [pfSense vlan defining port vlan]? I had everything going through the lan interface which is where I also had the vlan interfaces configured [10, 20, 30, and 40]. I labelled the ports through the Unifi interface with what each vlan they needed to be on. The DHCP and DNS server were on the main lan interface [vlan 1?] and I set that port [the uplink from the pfSense system] to allow all vlans through, which is why I think this was working properly. Now that I'm trying to redefine that same network to be on a vlan other than the lan vlan [again, I think vlan 1], and things just don't seem to be working unless I specify the vlan on the port, which makes some things work, but breaks the access to the other vlans since it limits the definition to the one vlan.
@jarhead said in Two pfSense boxes to make a lab network accessible through the same network using vlans:
No no, you're not moving it to a vlan. Just change the pvid on the switch alone. pfSense LAN port will use whatever pvid is et on the switch. Like I said, pretty much any switch made uses vlan1 but you can just change that to whatever you want and pfSense will still just use it.
If I don't want to confuse things with the switches thinking that two systems are using vlan 1, but with different network for each, don't I want to move/re-assign the previous network from vlan 1 on the pfSense lan interface to a vlan interface?
@jarhead said in Two pfSense boxes to make a lab network accessible through the same network using vlans:
I really think you're making this way harder than it actually is. Just reading your thoughts here has my head spinning so I can only imagine what you're going through.
I agree and I wish I could pull a cable to continue the setup I currently have [separate physical network], I just figured with vlans that there would be a way to logically separate things to accomplish the same physical setup.
I'd also like to make all of the vlans that I have defined on the pfSense lab system broadcast through the one port that I plan to use when I move this pfSense lab system so that the systems I had configured on the other vlans will work on those separate vlans.
I'm sorry these questions are all so obtuse, I thought I had a handle on this with my initial testing. There might be documentation out there individually for all of these features, but I think I'm trying to leverage features that aren't documented in a way that lends itself to an example that I can relate my setup to, which is why I'm hear asking all of the questions that I'm asking [and thankful for the responses!].
-
@uquevedo said in Two pfSense boxes to make a lab network accessible through the same network using vlans:
If I don't want to confuse things with the switches thinking that two systems are using vlan 1, but with different network for each, don't I want to move/re-assign the previous network from vlan 1 on the pfSense lan interface to a vlan interface?
Let's say you have 1 pfSense with 4 interfaces, and 1 - 24 port switch.
Set the switchports 1-12 to pvid 2, ports 13-18 to pvid 3 and ports 19-24 to pvid 4.
pfSense port 1 is WAN, port 2 is LAN, port 3 is LAN2 and port 4 is LAN3.
Plugging LAN into switchport 1 makes that network pvid 2.
Plugging LAN2 into switchport 13 makes that network pvid 3.
Plugging LAN3 into switchport 19 makes that network pvid 4.I think that's the part you aren't getting. There won't be overlapping of vlan1.
Set the switchports to the pvid you want and the network you plug into that port will use that pvid. -
@jarhead as all other festive seasons and us all been "good" in our field, we hook up and help crazy uquevedo solve his issue. I have been in infrastructure, networking and cyber sec for only about 25 year so I dont know much but I like to play. I think uquevedo is running my same kinda environment. Its christmas lets get him sorted
-
@jarhead said in Two pfSense boxes to make a lab network accessible through the same network using vlans:
I think that's the part you aren't getting. There won't be overlapping of vlan1.
Set the switchports to the pvid you want and the network you plug into that port will use that pvid.Thanks, that does help clear some things up. I had read somewhere that the vlan# needed to match on the pfSense to what the port vlan# was designated to be and that they needed to match. I suppose that isn't entirely the case? The vlan# on the pfSense is the gateway for that vlan, as long as the switch vlan port matches the pfSense vlan designation, then traffic will go to that network and out that pfSense vlan interface to get to the wan interface and/or where ever the firewall rules would allow the traffic to go.
If I wanted all of the vlan interfaces going through one port to go out to the network, I'd need to set that port on the switch to allow all [at least that's the setting in Unfi] all the way to the destination switch where the lab network is and then assign vlans to the ports so those systems will be on the appropriate vlan?
If I'm right about the above, I think I'd still want to have my other systems on a different vlan with firewall rules so that the lab traffic isn't broadcast to the lab and vice-a-versa?
@djay said in Two pfSense boxes to make a lab network accessible through the same network using vlans:
help crazy uquevedo solve his issue
I appreciate it, I really do. I've worked in an IT department before where other staff handled this kind of stuff and I was just on the outside looking in and catching bits and pieces. A lot of their head aches make sense now...especially people coming in and plugging a unmanaged switch into a managed one!
Any other advice or even skolding are welcome, this has all been very helpful!
-
@uquevedo said in Two pfSense boxes to make a lab network accessible through the same network using vlans:
Thanks, that does help clear some things up. I had read somewhere that the vlan# needed to match on the pfSense to what the port vlan# was designated to be and that they needed to match. I suppose that isn't entirely the case?
Now you're talking about different things again. You were very specific about vlan 1 only, so I answered you in regards to vlan 1 only.
Again, almost every switch ever made uses vlan 1 as it's default vlan. So if you bought a switch, did zero config and plugged it into a routers LAN, that LAN would be using vlan 1.
If you were to trunk a port between the router and switch, vlan 1 would be untagged and any other vlans would be tagged. Those would have to match the vlan id of the switch and router.If I wanted all of the vlan interfaces going through one port to go out to the network, I'd need to set that port on the switch to allow all [at least that's the setting in Unfi] all the way to the destination switch where the lab network is and then assign vlans to the ports so those systems will be on the appropriate vlan?
Correct. Trunk ports carry multiple vlans through the network, then access ports allow you to use those vlans where you need them.
If I'm right about the above, I think I'd still want to have my other systems on a different vlan with firewall rules so that the lab traffic isn't broadcast to the lab and vice-a-versa?
A vlan is a separate broadcast domain.
-
Hi Everyone that has been putting up with me for starting this thread.
I finally moved the lab server to the "after" I had above, and yes, everything just worked like everyone said it would.
Thanks for all of your input and of course patience!
-
@uquevedo Groovy. Glad it worked