Routing Traffic Between Two vLANS

eliteassassin07

Hi,

As the subject suggests I am trying to figure out how I can route traffic between two vLANS on my local network…

Current setup/configuration is as follows:
I am using a TP-Link TL-SG1016DE 16 Port Switch that is connected to my pfSense Firewall via a single LAN connection on Port 1 of the switch. I have then configured three vLANS on the switch using the 802.1Q protocol as follows...

VLAN 1 >> This is the default VLAN on the switch >> Member Ports 1 - 16 >> Tagged Ports NONE >> Untagged Ports 1-16
VLAN 2 >> This is for iSCSI traffic only >> Member Ports 8 and 12 >> Tagged Ports NONE >> Untagged Ports 9 and 12
VLAN 3 >> This is for an isolated network that I want to have >> Member Ports 1 and 9 >> Tagged Ports NONE >> Untagged Ports 1 and 9

8021Q PVID Settings are as follows on the switch..
PVID 1 >> Port 1, 2, 3, 4, 5, 6, 7, 10, 11, 13, 14, 15, and 16
PVID 2 >> Port 8 and 12
PVID 3 >> Port 9

Testing has indicated that the VLANs are working as they should... VLANs 1, 2, and 3 are unable to communicate with each other (which is what I want)... VLAN 1 and 3 are able to connect to the internet via the up-link port on the switch (Port 1) which is connected to the LAN port on the pfSense Firewall. In addition both VLAN 1 and 3 are receiving DHCP from the LAN port on the pfSense Firewall.

pfSense Setup/Configuration:

I have not configured any VLANs on the pfsSense Firewall
Pure NAT is ENABLED (this is something that I want to leave turned on)
I currently have my WAN port set to DHCP
LAN port is set to Static 192.168.1.1
DHCP is enabled on the LAN interface

What I want to do is allow specific network traffic for a given port in VLAN 3 to be able to communicate with a corresponding device in VLAN 1... Now since I have Pure NAT enabled this is already occurring for devices that are configured for external access. For example I have a web server that is sitting on VLAN 3 and port forwarding is configured to allow external access to the site which also seems to be allowing devices on VLAN 1 to also reach the site. However I have some services that I do NOT want to open to external access (IE to the internet), but I need the devices on VLAN 3 to be able to communicate with those devices on VLAN 1... I feel like this should be pretty easy to setup, but I am just missing it for some reason...

Any help would be greatly appreciated!!!

jahonix

Since your TL-SG1016DE switch is L2 "only" you need to route via pfSense.
To do so you have to create additional interfaces for each VLAN and put those three interface on the physical NIC which connects to your switch.
The switch port connected to your pfSense needs to be a trunk with all VLANs tagged.

When working with VLANs always remember to NOT use VLAN ID1 for anything else than nothing.
VLAN 1 is used as default in most gear and oftentimes cannot be changed or deleted.
Use VLANs 10, 20, 30 or whatever idea you have instead.

eliteassassin07

@jahonix:

Since your TL-SG1016DE switch is L2 "only" you need to route via pfSense.
To do so you have to create additional interfaces for each VLAN and put those three interface on the physical NIC which connects to your switch.
The switch port connected to your pfSense needs to be a trunk with all VLANs tagged.

When working with VLANs always remember to NOT use VLAN ID1 for anything else than nothing.
VLAN 1 is used as default in most gear and oftentimes cannot be changed or deleted.
Use VLANs 10, 20, 30 or whatever idea you have instead.

I am not sure that I understand what you mean by additional interfaces for each VLAN and put them on the physical NIC, do you mean create VLANS on the pfSense box (under interfaces)?

johnpoz

"VLAN 1 >> This is the default VLAN on the switch >> Member Ports 1 - 16 >> Tagged Ports NONE >> Untagged Ports 1-16"

So all your ports are in vlan one it looks like..
And then you have your vlan 2 also untagged on 9 and 12? or 8 and 12..
And then your vlan 3 is also untagged on 1 and 9?

There seems to be a basic misunderstanding on how vlans work I think.. As jahonix mentions this is a layer2 only switch.. It does no layer 3 routing between vlans(networks)

You don't put ports as member of multiple vlans which seems you have everything in vlan 1 as untagged native, and then say 8 and 12 also in vlan 2 untagged? And vlan 3 on 1 and 9?

So this is how this normally works… See my attached drawing.

So pfsense has its physical lan interface.. If you connect this into a dumb switch this is the default or managment or vlan 1, etc.. No tagging all ports talk to all other ports, etc.. You can change this number it really has no meaning other than hey all ports are in this vlan.. You can leave it as vlan 1 on your switch or you could change it to 10, or 100 whatever you want doesn't really matter its just the native vlan for every port on that switch and since everything is untagged all ports talk to each other.

So there are couple of different ways you can go about this you can use the native vlan on the physical interface along with actual tagged vlans, or you can just use tagged vlans on the physical interface and not use any untagged, etc.. Doesn't really matter.

So as we connect the physical interface of pfsense to port 1 on the switch.. This is the uplink to pfsense from the switch... This is how all traffic from any of the devices connected get to pfsense to get off there local network. Then you see the other switch ports in green yellow and blue.. Those are all in their own vlan. Lets call them vlan 10, 20 and 30.

The networks on these vlans lets use
10 = 192.168.10.0/24
20 = 192.168.20.0/24
30 = 192.168.30.0/24

Keep it simple to see which is which.. But the actual address space you assign to a vlan doesn't really matter as long as you don't put the same space on different vlans.

So on your lan interface in pfsense you need to create vlans, and then you assign them to the physical interface.. See for example pic 2 mine are all in interface em2 You would then in pfsense assign the IP address in the networks you want to use on those vlans, enable dhcp on that interface you want.. And then setup your firewall rules for those vlans.

Normally when a port switch is connected to a single device that port would have only 1 vlan on it and it would be untagged.. When you connect say a switch or a router to a port and you want multiple vlans to be sent to that device these vlans would be tagged on that port going to that device.. In cisco speak this is called a trunk port. But other switch makers might call it something else.. But vlans are either tagged or untagged.

When talking to the device that you put in a specific vlan, the device does not freaking care.. He just knows his network 192.168.20.0/24 So that traffic is untagged.

Now when we send it to another switch or a router like pfsense, need to be able to tell what traffic is what, so this traffic is tagged based upon what port it came from and what vlan that port is in. So the switch says oh that traffic came from port in vlan 20, that traffic can talk on any port that is untagged member of vlan 20.. Say the green ports.. And if sent out the uplink port it will tag that traffic as 20..

So if the computers in the pic want to talk to each other, the switch allows it since they are all in the same vlan and those ports are allowed to talk to each other. But if the computer wants to talk to the server that is yellow (vlan 10) in this example.. And on a different network 192.168.10.0/24 The computer needs to talk to his gateway (pfsense) to get off its local network of 192.168.20.. So he says hey gateway I want to talk to 192.168.10.x.. Pfsense says oh I have that on my vlan 10.. Let me send that traffic there for you (if the firewall rules allow that from vlan 20 firewall ruels).. So pfsense puts that traffic out the same physical interface but its tagged vlan 10.. So the switch sees this traffic and says oh this is tagged vlan 10 traffic... I can only send it to ports that are in vlan 10.. And will send it out the port in that vlan the device is on..

Does that make it any clearer? It sure looks to me like you have ports with multiple untagged vlans on them... It doesn't work that way.. And yes you need something to route the traffic at layer 3 between your vlan networks.

vlaninterface.png_thumb

vlans.jpg_thumb

whosmatt

All the advice in this thread is spot on. I just got the 8 port version of the same switch for home use and haven't really investigated the features of it yet as far as vlans are concerned, but if I have some time in the next couple of days I can dig in a little bit and give you some specific guidance for this particular switch. In my job I have lots of experience with L2 managed switches and pfsense with trunked VLAN interfaces so I'll try to translate that to this particular switch if I can.

In pfsense, however, the config is pretty simple. Say your interface connected to your switch is em0. You set up a vlan (say 10) on that interface. Then, you have a new interface named em0_vlan10. Then, you can assign that interface as anything you want… LAN, OPT1, whatever. What makes this work is having it connected to a switch port that is in trunk (tagged) mode (not sure what it's called on the TP-LINK switch, but I'll try and investigate) and then having any device that needs to be in that same network connected to a switchport that is configured untagged for vlan 10. The device connected to that port knows nothing about vlans, but the switch knows that all the traffic on that port is in vlan10 and will send it to the port connected to your pfsense box with a tag that says it's in vlan 10, and pfsense, knowing about the vlan, will say "hey! this traffic is tagged with vlan10 and needs to go to em0_vlan10." It doesn't actually say that, of course, because that would be annoying. But that's what happens, in essence.

whosmatt

Ok, I had a quick look at the TP-LINK config and it looks pretty simple. But you need to start with pfsense. On the physical interface connected to the switch, define your vlans with that as the parent interface. Then, set up new interfaces with those vlans (see my example above with vlan 10). Give each of those interfaces an IP address in a different subnet. It would make sense to have the vlan ID match the subnet in some way. For vlan 2 and 3, for example, maybe 192.168.2.0/24 and 192.168.3.0/24 respectively, but the actual mechanism is arbitrary; the vlan ID is just an integer between 1 and 4094.

Define the same vlans in the switch, under the 802.1Q config. Then, set the port that pfsense is connected to to carry all (both) of those vlans tagged. Other ports connected to vlan unaware devices will be untagged, and you'll generally have just one vlan assigned to those. So, a device that needs to be on 192.168.3.0/24 will be connected to a port that is in vlan 3 untagged, etc. It's pretty much as simple as that.

In pfsense, of course, you'll need to set up firewall rules on each of your vlan interfaces to allow traffic. Could be as simple as allowing any to any, or you can get really granular if you wish. Just remember that without any rules at all, no traffic will flow between subnets or to the internet.

This will help with things on the switch side: http://www.tp-link.com/resources/document/TL-SG1024DE_V1_User_Guide_Easy_Smart_Configuration_Utility.pdf

eliteassassin07

Here is what I did to correct everything…

So I configured the VLANs on the switch as follows:

Using 802.1Q I configured a total of 4 VLANs…
VLAN 1 >> This is the default VLAN on the switch and is not being used at this time
VLAN 100 >> Used for all local devices that are not included in the following two VLANs
VLAN 200 >> Used for iSCSI traffic
VLAN 300 >> Used as part of an “isolated” network

VLAN 300,
For this VLAN I “tagged” ports 1 and 9…
Member Ports >> 1 and 9
Tagged Ports >> 1 and 9
Untagged Ports >> NONE

VLAN 200,
For this VLAN I “untagged” ports 8 and 12…
Member Ports >> 8 and 12
Tagged Ports >> NONE
Untagged Ports >> 8 and 12
Note, I could not use “tagged” ports here as it appears that one of the NICs in my server is not VLAN aware. However since this VLAN is only used internally and does NOT need to communicate with the firewall this is really not an issue. I also changed the PVID settings for ports 8 and 12 to 200.

VLAN 100,
For this VLAN I “untagged” ports 1-7, 9-11, and 13-16
Member Ports >> 1-7, 9-11, and 13-16
Tagged Ports >> NONE
Untagged Ports >> 1-7, 9-11, and 13-16
Note, I changed the PVID settings for ports 1-7, 9-11, and 13-16 to 100.

Note port 1 is the uplink/trunk port on the switch that goes back to pfSense

pfSense configuration:

Since VLAN 100 is now where all of my local traffic (for desktops , laptops, and mobile devices) is located and the traffic is all untagged and using PVID 100 I did not need to change anything on pfSense as it just all routed through the default LAN interface. Only thing that I did do here is on the LAN interface under rules I created a rule that blocked all traffic from LAN to OPT1(VLAN 300).

Since VLAN 200 does not need to communicate with the firewall or internet there was nothing that needed to be configured on pfSense.

Since VLAN 300 needed to be able to communicate with the firewall and the internet I went ahead and created a VLAN on pfSense so that this could take place. I then created a rule for the OPT1 (VLAN 300) interface that blacked all traffic from OPT1 to LAN.

The above configuration appears to have effectively separated all of the network traffic like I wanted and allows me to create rules to only allow certain traffic to pass between VLAN 100 and VLAN 300.

Thanks everyone for the information!

johnpoz

your not getting it are you??

you only need tagged on the port going to pfsense.. All of the vlans would be tagged here other than the native vlan, ie your physical interface. be this 1 or 100, or 200 whatever.

All your other ports would be in the 1 vlan you want on that port as untagged.

"VLAN 300,
For this VLAN I “tagged” ports 1 and 9…"

if port 1 is connected then that is fine.. But why do you have it tagged on port 9?? Is that an uplink to another switch?

How is vlan 200 getting to pfsense if its not tagged on port 1??

Derelict

This would be:

em1_vlan100 LAN
em1_vlan200 WAN
em1_vlan300 GUEST, DMZ, etc.

VLAN-pfSense.png_thumb

eliteassassin07

@johnpoz:

your not getting it are you??

you only need tagged on the port going to pfsense.. All of the vlans would be tagged here other than the native vlan, ie your physical interface. be this 1 or 100, or 200 whatever.

All your other ports would be in the 1 vlan you want on that port as untagged.

"VLAN 300,
For this VLAN I “tagged” ports 1 and 9…"

if port 1 is connected then that is fine.. But why do you have it tagged on port 9?? Is that an uplink to another switch?

How is vlan 200 getting to pfsense if its not tagged on port 1??

I feel like something is getting lost in translation here…

For VLAN 300 I tagged port 1 which is the uplink port on the switch and tagged port 9 which is connected to my ESXi host. Port 1 has to be tagged so that pfSenese VLAN Interface can see the traffic correctly. Then on the ESXi host I have configured the virtual switch to VLAN 300 as well... So yes in a since port 9 is connected to another switch.. Probably should not have left that information out.. Sorry

As far s VLAN 200 goes... That VLAN does not need to communicate with pfSense (really I dont want it to communicate with pfSense). The only thing on that VLAN is iSCSI traffic from a FreeNAS Server to an ESXi Host, both of which are connected to the same switch and use statically assigned IPs.

johnpoz

"which is connected to my ESXi host"

Where did this esxi host come from?? Is pfsense running on your esxi host? There was no mention of any esxi host in your first post or 2nd or 3rd..

tagged ports are only used for uplink ports.. Like another switch, a router wth vlans on that interface, an AP that will have vlans on different SSIDs, sure an esxi host that will have vms that will use different tags connect to that same vswitch that physical nic is on..

You only need to tag on ports that are going to carry more than 1 vlan. Because those tags will be used on the device that is uplink to determine where that traffic should flow.

Derelict

Tag ports to devices that need to see the tag. It's that simple. That would be pfSense and ESXi.

If you put an untagged port on VLAN 100 (LAN) and connect a device to it and it gets DHCP, can query DNS, etc your VLAN and switch and pfSense are configured fine and you can concentrate on your ESXi setup.

eliteassassin07

pfSense is running on its own hardware…

"Port 9" is connected to my ESXi Host...

So I think that what you are saying is finally settling in... been ridding the struggle bus here...

I now have it configured as follows...

VLAN 100,
Untagged Ports >> 1-7, 10-11, 13-16 (PVID for these ports set to 100)
Tagged Ports >> NONE

VLAN 200,
Untagged Ports >> 8 and 12 (PVID for these ports set to 200)
Tagged Ports >> NONE, Remember though I dont want this VLAN to connect to pfSense

VLAN 300,
Untagged Port >> 9 (PVID for this port set to 300)
Tagged Port >> 1

So with this set up the only thing that is tagged is Port 1 which is the uplink port.

I believe that this is now correct...

eliteassassin07

@Derelict:

Tag ports to devices that need to see the tag. It's that simple. That would be pfSense and ESXi.

If you put an untagged port on VLAN 100 (LAN) and connect a device to it and it gets DHCP, can query DNS, etc your VLAN and switch and pfSense are configured fine and you can concentrate on your ESXi setup.

Thanks…

Yes... currently anything that I connect to ports 1-7, 10-11, and 13-16 on the switch is receiving DHCP as it should from pfSense.. iSCSI traffic on ports 8 and 12 is working as it should... And the servers connected to port 9 on ESXi are also working.

I made some changes to the way the virtual switch was configured in ESXi so that port tagging was no longer needed except for on the up link port on the switch.

johnpoz

esxi only needs tag if your going to use vms on different vlans, they your going to need multiple tags on that port.. If all your vms are going to be on the same vlan then you don't need tags.. Where is your vmkern is that he same physical nic.

Keep in mind if you want to use vlans on your vms you have to set the vswitch to 4095 so it will pass the tags on to your vms on that vswitch.

eliteassassin07

All of the VMs will be on the same VLAN. :)

VMKern/Management is on its own NIC

Thank you for all of your help…. I am sure this has been as frustrating for you as it has been for me... lol

whosmatt

Hey, off topic i know, but does that 16 port TP-LINK switch have fans?

eliteassassin07

@whosmatt:

Hey, off topic i know, but does that 16 port TP-LINK switch have fans?

No, it is fan-less.

Although it really does not appear to get all of that hot and I am using almost all of the 16 ports on there.

jahonix

@eliteassassin07:

I am sure this has been as frustrating for you as it has been for me…

Is it working now?

eliteassassin07

Yes, it is now working.

Thanks everyone!