Design help for better control
-
Hi,
I need some help to better separate out the traffic.Current setup has pfsense vm in an AIO machine. Trying to setup vlans to separate out CCTV, IOT and guest traffic. Looking for an easier setup to maintain. currently it is flat.
Will separating out to its own physical machine better. Also connect unifi 16 switch to pfsense directly a better option?
In the past when I tried to turn on the guest vlan, i was unable to get an IP address when on guest network. The trace capture the packet at the VM host nic when using port mirroring.
Portgroup vlan is set to 4095.Tried researching on different forums but info are mostly specific to either Unifi setup or incomplete.
-
The order you connect the switches in should be largely irrelevant. That's probably more of a physical location decision.
Steve
-
Yeah really the only time it could be relevant is uplink speed and where you uplink too, or other feature sets like where 1 switch doesn't support what your wanting to do.
Say you had a few switches downstream of one switch, you might want to put switch that connects to where routing is happening with a 10ge or multigig connection if your router also supports such speed.
If you had just a dumb switch that didn't understand vlans you could put that downstream of one that does, as long as you just going to put everything connected to that switch in the one vlan.
But normally if your switches can all do the same thing, and it just comes down to brand/model etc.. Shouldn't really matter which switch is upstream or downstream..
Port density could come into play for location of one switch over the other for sure as well. POE might come into play as well, if say your AP and camera's are poe, and due to physical location of these external devices yeah you would want to use your poe switch where it makes most sense to connect these devices to your network..
Another aspect that could come into play is devices and bandwidth consideration. If you had say heavy bandwidth using devices that talk to something specific, you wouldn't want to connect your devices to different switches because they could put a strain on your uplink between the switches. Normally you would want your heavy talkers on the same switch if possible (if they are in the same vlan).. If the traffic has to be routed and that is all upstream, might not matter as much.. But when when you have multiple switches and different devices the what devices are connected to which switch could come into play, and depending on other factors that could factor into which switch should be upstream or downstream of other switches, etc..
-
@johnpoz
I read that if access to esxi mgmt will be lost if it depends pfsense routing. i plan keeping within vlan20 and not have a separate mgmt vlan.In keeping most stuff in vlan 20. The cross vlan traffic will be from vlan20 to vlan60 to access NVR. The rest should just need access to the Internet.
The idea of not linking the 2 switches is for the ease of troubleshooting. I had issue trying to figure out why devices on guest vlan is not getting IP.
-
You don't have to link them if you didn't want too.. If your just going to use vlan xyz on 1 switch and vlans abc on the other switch.
You could still have a common vlan for managment, that is only uplinked by 1 switch and they could still be interconnected, they just wouldn't carry vlans over their uplinks that are not used on the actual switch, etc..
As to not linking them for ease of troubleshooting? In what sense - at a loss to how that would help in troubleshooting anything to be honest.
You understand you could do all kinds of different configurations you don't even really have to have 1 downstream from the other with shared uplink to the router..
You could have multiple uplinks from each switch, that carry specific vlans, or only one network. to the router - depending on how many interfaces you have on the router. And the only reason you need to link the switches at all is for any vlans you want on the other switch, this could be just management vlan.. Or could be say you want something on switch 2, in vlan X and the uplink to the router for X is on switch 1..
There nothing saying that that management of IP of the switch has to be in any specific vlan or can not be 2 different vlans if you want to isolate that from other devices for your 2 switches.
The only thing I could see being problematic is unifi devices like to all be in the same L2 for discovery. But you can also do L3 adoption.. Once things are adopted they don't have to be in the same L2.. I completely manage my sons devices in his home on my controller, just over the internet, etc.
-
@johnpoz said in Design help for better control:
You could have multiple uplinks from each switch, that carry specific vlans, or only one network. to the router - depending on how many interfaces you have on the router. And the only reason you need to link the switches at all is for any vlans you want on the other switch, this could be just management vlan.. Or could be say you want something on switch 2, in vlan X and the uplink to the router for X is on switch 1..
The only thing I could see being problematic is unifi devices like to all be in the same L2 for discovery. But you can also do L3 adoption.. Once things are adopted they don't have to be in the same L2.. I completely manage my sons devices in his home on my controller, just over the internet, etc.
Good to know you can have multiple uplinks from the 1 switch to the router.
For the unifi APs, will it be easier to put in all in a mgmt vlan together with the cloud key?
-
Yeah its much easier to just run your controller and all your unifi devices in the same L2.. But it can be done via L3 if you don't want to do that..
This doesn't really need to be any sort of special "management" vlan - could just be your normal lan where only your trusted devices sit.. And you put all your non trusted sort of devices in other vlans.
-
For the last few days, I made some changes to configure the vlans as needed.
NVR vm and Cams are in its own vlan 40. Viewing of video is accessible from normal lan.
Guest vlan 50 is setup on Unifi APs using the controller.
But devices is not getting IPs when connecting to guest network.
Trace using wireshark to capture vlan traffic is seeing dhcpdiscover.
-
You are capturing that on a client in the VLAN 50 subnet?
What do you see if you capture on VLAN 50 in pfSense?
Is the DHCP enabled on the VLAN 50 interface?
Steve
-
@stephenw10
I setup port mirroring on the switch with wireshark on the PC.
DHCP is enabled on the interface. I have not tried any trace on pfsense.. However, i do not see the dhcpdiscover in the dhcp logs.
-
@moosport said in Design help for better control:
DHCP is enabled on the interface.
Doesn't matter if dhcp was enabled or not - if the interface is connected to the L2 network that discover goes out - it will be seen. A discover is seen on all devices in a network - its a broadcast. The dhcp server is the only one this going to answer is all.
If pfsense is not seeing the discover - then no its not going to hand out an offer. You have something wrong in your network if your vlan is not seeing the discover.
-
The next trace to run is on the vswitch which both the interface and pfsense is connected to.
-
vswitch - so are you not passing the tag? did you set an id of 4095 in the portgroup on the vswitch..
-
It is set to trunk. Ran trace at the vmnic but did not see dhcpdiscover traffic. At this point I'm not sure where the tags are being dropped.
Trunk is set from Unifi port (APs), both Unifi port and Aruba port, Aruba SFP+ port, portgroup vlan 4095.
-
If you pcap on the parent NIC in pfSense you can see what VLANs traffic is tagged with when it arrives.
-
I see it in the SFP+ going into Esx host.
Following these instruction to trace traffic for vmnic. No vlan traffic was captured.
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.networking.doc/GUID-F1AC7100-FFBB-4414-9B70-E5537C15E192.html
Next step is to capture dropped packets. This will probably shed some light.
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.networking.doc/GUID-84627D49-F449-4F77-B931-3C55E4A8ECA1.html -
Found out the setup requires vdswitch. Moving to physical.
-
@moosport said in Design help for better control:
vdswitch
So you have multiple esxi hosts? No mention of that
-
@johnpoz running the free version which only support vdswitch for first 60 days.
-
@moosport you sure do not need vdswitches to do vlans on esxi.. My point is you make no mention of having multiple esxi hosts in a cluster - nor do you show such a thing on your drawing.
