VLAN behind VLAN?

taenzerme

Good morning,

sorry if this might be a basic question, but after reading a lot of basic stuff about VLANs, tagged and trunked ports etc. I played with the VLAN functions in pfSense and got stuff working so far as intended:

1. Managed switch with 2 VLANS (VLAN IDs 1, 2) for WAN and LAN.
2. 3 Proxmox hosts
3. Virtualized pfSense in the Proxmox cluster
4. 2 VLANs (50,60) on the pfSense appliance.
5. Assigned 2 VMs in Proxmox to these VLANs.
5. VM A gets an IP from DHCP server for VLAN 50, VM B for VLAN 60.
6. Clients without assigned VLAN can access both VLANS if not forbidden by fw rules.

So far, so good. Is this common and good practice to seperate clients in different subnets on one NIC? What role do the 2 VLAN on the switch play? Are the VLAN id's per se "global", so creating another VLAN 2 on the pfSense will mess with the VLAN on the switch?

In the end, it comes down to one question: Can and should I have VLANs "inside" vlans - which I understand I have here?

If this is just "bullshit" I'd be happy to learn how to set up seperate subnets with the same firewall in a virtualized enviroment with pfSense using the same WAN with public IP subnet.

Best
Sebastian

taenzerme

After more reading I narrowed down my questions a bit more.

My setup currently is this:

HP 1810-24G Managed switch:
Port 1 EXCLUDE (Management if.)
Port 2-12 VLAN2 => LAN, Ports 1,13-24 set to EXCLUDE
Port 13-24 VLAN3 => WAN, Ports 1-12 set to EXCLUDE

Idea behind this directly have some virtual machines with public interfaces bypassing the firewall.

Proxmox host(s):
vmbr0 => eth0 => LAN, VLAN2 on switch
vmbr1 => eth1 => WAN, VLAN3 on switch

pfSense VM:
vtnet0 => vmbr0 => LAN
vtnet1 => vmbr1 => WAN

What I want:

VMs in different subnets that should be able to access other subnets only if allowed by rule.
All VMs should have an outgoing internet connection if allowed by rule through pfSense
All VMS should be able to have an public IP bypassing the pfSense if interface with public bridge (vmbr1) exists.

What I don't understand:

do I have to switch ports to trunking on my switch if I want to use the VLAN functionality on the pfSense, the ports that go to vmbr0 on the Proxmox hosts and so to vtnet0 (LAN) on the pfSense need to be set to trunking instead of VLAN?
can I have VLANs and "non-VLANs" on the same (virtual/physical) interface?

It would be really great if someone would help me to understand how the basic setup should look like.

mikeisfly

Disclaimer, I have never used proxmox but it seems like it is some type of hypervisor like VMware ESXi or Microsofts Hyper-V or Citrix Zen Server. With that being said here is how it works. You would create vlans on your switch because you want to segment traffic for security and to break up your broadcast domains. There are two types of ports untagged which Cisco calls access ports. The second is called tagged ports which Cisco calls trunk ports. You mentioned that you were using a HP switch, for HP a trunk port is a Ether Channel in the Cisco world so you shouldn't need this unless you want to setup a Lagg between your switch and Pfsense.

When you create vlans on your switch you should create the same VLANs on your Pfsense this keeps everything sane, plus when PfSense is tagging the traffic to send to the switch the switch will know what to do with it. Since you are running PfSense in a Virtual Machine I would suspect that there is some type of Virtual Switch in Proxmox that your VM clients are connecting to that is why your clients are pulling an IP and everything seems to work. If you want those clients to get online there is no need to give them a Virtual IP, NAT should take care of that. If you need for unsolicited traffic to get from the Outside -> In then just use a port forward. If one of your devices needs a public IP then I would use the VIP feature.

Connecting your proxmox server (virtual switch) to your HP switch, they both should have the same VLANs that way if you connect a physical machine to your switch it will work through PfSense. It is very common to have multiple VLANs on one physical port and I would go a far as to say that it is uncommon to have different VLANs on Different Ports. You do want to stay away from putting untagged traffic and tagged traffic on the same port on PfSense just as good practice. On the switch this is how the voice vlan works if you are in a Voice over IP situation where the computer in going to connect through the phone. On the port where there is untagged traffic and tagged traffic you would tell the switch which tagged vlan is for voice and then the untagged traffic would be passed to the computer through the phone.

For security reasons you should only put tagged traffic on a port if the device connecting to the port needs it.

Short Answer:

Don't mix vlans!

taenzerme

Mike, thanks a lot for the detailed answer. This really helps to put things in perspective.

So I'd:

on the switch create a vlan 2 for clients, one for admins (vlan 3) and one for WAN (vlan 4)
create the same vlans in pfSense with the same ids
assign the virtual nics in proxmox vms to these vlan ids

My question would be: Do I need a seperate nic and switch port for each vlan (vlan 3 and 4 already are as these are the current lan and wan) or can vlan 2+3 use the same switch port and be tagged with the same vlan id?

jahonix

In pfSense you stack multiple VLAN interfaces on one physical IF, making it a trunk.
Configure one switchport as trunk and add the same VLAN IDs tagged.
Connect with one cable (multiple don't fit a single socket anyways! ;-)))

And make sure to NOT use VLAN1 anywhere.
That's a default in lots of gear and could create a mess.
Start with VLAN ID2 and count upwards. Gaps don't matter. 4k is max.