VLAN Help



  • Hello I am configuring a VLAN and have been following tutorials found in order to configure it.  I will try and be as clear as possible to what my setup is:

    PFsense 2.2 (virtualized on ESXI 5.5)

    Dlink Smart Switch DGS-1100-24P

    VLAN ID 10

    Here is I have done:

    I created a VLAN Interface and assigned it to the same adaptor as the LAN.  I enabled the VLAN Interface and assigned it a 10.10.10.1/24 IP.

    I enabled DHCP for VLAN 10 with a range of 10.10.10.100-10.10.10.200.

    I created VLAN 10 in ESXI and it is on the same adaptor as the LAN in ESXI.

    I created VLAN 10 on the Dlink DGS-1100-24p.

    PFsense is plugged into port 1 on the dlink.  I tagged port 1 in VLAN 10.

    My Laptop is plugged into Port 24 of the Dlink.  I untagged port 24 in VLAN 10.

    I created a Firewall rule.

    I am providing screen shots of my setup.

    I am able to get a DHCP address from PFsense for VLAN 10.  I get 10.10.10.100 with a default gateway 10.10.10.1

    Problem:  I do not have Internet.

    I cannot ping the VLAN default gateway of 10.10.10.1

    From PFsense I cannot ping 10.10.10.100

    I appreciate any help you can provide.
    Bill

    ![VLAN Interface First.png](/public/imported_attachments/1/VLAN Interface First.png)
    ![VLAN Interface First.png_thumb](/public/imported_attachments/1/VLAN Interface First.png_thumb)
    ![VLAN Interface.png](/public/imported_attachments/1/VLAN Interface.png)
    ![VLAN Interface.png_thumb](/public/imported_attachments/1/VLAN Interface.png_thumb)
    ![VLAN DHCP Setup.png](/public/imported_attachments/1/VLAN DHCP Setup.png)
    ![VLAN DHCP Setup.png_thumb](/public/imported_attachments/1/VLAN DHCP Setup.png_thumb)
    ![VLAN Firewall Rule.png](/public/imported_attachments/1/VLAN Firewall Rule.png)
    ![VLAN Firewall Rule.png_thumb](/public/imported_attachments/1/VLAN Firewall Rule.png_thumb)
    ![DLINK Port 1.png](/public/imported_attachments/1/DLINK Port 1.png)
    ![DLINK Port 1.png_thumb](/public/imported_attachments/1/DLINK Port 1.png_thumb)
    ![Dlink Port 24.png](/public/imported_attachments/1/Dlink Port 24.png)
    ![Dlink Port 24.png_thumb](/public/imported_attachments/1/Dlink Port 24.png_thumb)


  • Netgate

    PFsense is plugged into port 1 on the dlink.

    How is pfSense plugged into anything if it's a virtual?



  • My apologies for the miscommunication.

    EXSI is plugged into Port 1of the Dlink.

    PFsense is attached to the network adaptor that is plugged into port 1 of the dlink.

    Bill


  • Netgate

    You have not given any description about how you configured your WAN.  Only your LAN.

    You should be able to ping 10.10.10.1.  What firewall rule did you create on LAN.

    You do not need to tag the interface in pfSense if you are just adding an interface to the vSwitch that is tagged on VLAN 10 to the switch.

    The only time you would need to tag the port in pfSense is if you were sending VLAN 4095 (All VLANs tagged in ESXi) to pfSense.

    What I don't understand is how you are getting DHCP.  That indicates Layer 2 is OK.  Sure there's not another DHCP Server available to VLAN10?

    I also don't know WTF all those VLAN options are in your switch.  What is a "Hybrid untagged VLAN" etc.

    You want to tag vlan 10 to ESXii, nothing more, nothing less.



  • So currently I have the WAN configured and working without a VLAN involved.

    I am in the beginning stages of creating VLAN's.

    I have not created any additional rules under the LAN for VLAN 10.

    That is pretty much all I did was tag the port going to PFsense/ESXI.  I had to untag the port that my laptop is plugged into to get the DHCP.

    I don't think I conveyed that I tagged a port in PFsense, if I did I am sorry for the miscommunication.


  • Rebel Alliance Global Moderator

    "I created VLAN 10 in ESXI and it is on the same adaptor as the LAN in ESXI."

    Huh??  So create your vlan on pfsense interface that vlan is going to be on.  What did you do with your vswitch in esxi?  Did you change it to trunk mode?  You have to allow for the vlan or vlans you want in 2 switches since you really have this with esxi

    pfsense - vnic – vswitch -- esxihostnic -- realswitch - realdevice

    so both your vswitch and your realswitch have to allow for whatever vlans your going to be using.

    Curious were you created vlan 10 in esxi??



  • Netgate

    @bhunter:

    I don't think I conveyed that I tagged a port in PFsense, if I did I am sorry for the miscommunication.

    Right here you did. Interface Perk is tagged to the vSwitch:



  • You do not need to tag the interface in pfSense if you are just adding an interface to the vSwitch that is tagged on VLAN 10 to the switch.

    I am a little confuse by this statement.  I had to create the vlan in pfsense.  In order to setup a dhcp server for that vlan, it had to be assigned to an interface.  The tutorials I have followed in order to set this up told me to assign the vlan to the same interface as the LAN.  I appreciate your time looking at this.

    I attached a screenshot for what I have done in ESXI



  • Netgate

    vmnic2 is connected to a switch port.  What VLANs are on that switch port?

    vmnic3 is connected to a switch port.  What VLANs are on that switch port?

    I would not be using untagged interfaces (em0, em1) in the pfSense VM.  I would tag everything.  That would mean new VLANs for WAN and LAN.



  • vmnic2=ISP modem (provides internet)

    vmnic3=port 1 of the dlink switch.  VLAN 10 is tagged, vlan 1 (default vlan that is created by the factory) is untagged.

    Thanks for taking the time
    Bill


  • Rebel Alliance Global Moderator

    what is the point of the port group with a vlan 10 with nothing in it??

    Why would you set 4095 on your WAN?  Your not running multiple vlans over that are you?

    This not rocket science..

    I showed you the vswitch and port group that pfsense wlan vnic is attached is set with 4095.  Yes there  multiple vlans on this with their own IDs - see pfsense interfaces.  Then the switch port that is connected to physical nic in the esxi host is trunked.. It is that simple..  Then ports that are in those specific vlans on the switch are in those vlans..

    interface gigabitethernet3
    description "esxi wlan"
    switchport trunk allowed vlan add 100,200,300
    switchport trunk native vlan 20




  • I understand this is not "Rocket Science".

    That was a change I made after I saw your screen shot.  I thought what the hell nothing else I am trying seems to make a difference.  Nothing has changed.  I was hoping some people might have a few ideas as to why I can get dhcp from vlan 10 but I cannot ping the dafault gateway.  Nor do I have internet on Vlan 10.

    I have attempted several different firewall rules.  I just thought that people that have a lot of experience and have setup vlans before would provide some suggestions to help me out.  Maybe even help me troubleshoot the issue to help me determine if I did something incorrect in PFsense.
    This is my setup:
    pfsense - vnic – vswitch -- esxihostnic -- realswitch - realdevice

    It has been that way since I first posted the problem.

    I currently have a 192.168.2.0/24 network that can access the internet fine through PFsense.  The switch still has the default vlan "1".  I am currently attempting to transition to vlans.  My end goal will be to have a minimum of 2 vlans:

    vlan 10 w a subnet of 10.10.10.0/24

    vlan 20 w a subnet of 10.10.20.0/24

    Right now I am only concentrating on vlan 10.


  • Rebel Alliance Global Moderator

    exactly that is how pretty much every single setup would be using esxi

    pfsense - vnic – vswitch -- esxihostnic -- realswitch - realdevice

    So what is your configuration of the realswitch port that is connect to the esxihost nic?  What is the configuration of the real switch port that is connected to your client you want to be in vlan 10..

    I posted my config on my trunk port that connects esxi host to switch.  Here is config of for example port connected to my son's ps3 that I have in its own vlan

    interface gigabitethernet7
    description "ps3 powerline"
    switchport mode access
    switchport access vlan 100
    !

    So lets follow the packet.. devices sends broadcast dhcp discover -- hey dhcp server can I have an IP..  So that hits real switch port.  That is in access mode vlan 100 in my setup.  Now these packets that go down a trunk will be TAGGED with 100.. And since its broadcast will go to every other port that is in vlan 100.

    So it goes down the trunk that allows vlan 100, this hits the esxi nic, then the vswitch..  Vswitch that is set for 4095 is like a trunk and does not strip the tag..  So now it hits pfsense vnic with tag 100..  So this goes to the vlan 100 interface.  Dhcp server sees this and send back offer that goes back the same path.

    So we need to know what the setting are on your real switch for the port connected to your esxi nic and the port connected to your device.  If you don't allow the trunk to your esxi or don't have the vlan setup on the port connected to your device then no your dhcp discover will never get to your dhcp on your vlan or would go to just say the dhcp server running on the native or physical network without any tagging, etc.



  • My first post included 2 screenshots that show the config of the 2 physical ports being used.

    Dlink Port 1 is the port my esxi host is connected to.  The screenshot shows that the port in "tagged" in vlan 10 and untagged in vlan 1 (default vlan).  The post and screen shots also show the vlan interface I created and enabled in PFsense (including settings).

    Dlink port 24 is the port my laptop is connected to.  This screenshot shows that the port is "untagged" in vlan 10 and 1.

    Both screenshots shows the "native" vlan for the port.

    I provided a screenshot of the firewall rule I created.  The firewall rule was created on interface vlan 10.

    I did not any additional firewall rules under the following interfaces:

    "WAN"
    "LAN"
    "Floating"

    Bill


  • Rebel Alliance Global Moderator

    Yeah and those are not correct.. So you have your port connected to port connected to esxi no tagged at all.. And in a native vlan 1.. Not sure what hybrid vlan is?  So if that packet leaves the port going to the vswitch without a tag how would pfsense know to pick it up on its vlan interface ??  Would have to look up the manual for that switch..

    And then your port on your laptop is native vlan 10, but there is no tagging.. So where does it get tagged 10 so that switch sends it down port to esxi tagged?  You could prob tag it on the interface in your laptop.



  • To tell you the truth I am not sure what the Dlink "Hybrid" vlan is.  Their documentation is pretty sparse.  I did look at other mode but they did not work.  They also offer access mode and trunk mode.  Access mode seemed to offer either tagged or untagged (not both).  Trunk mode (in Dlink) terminology is not the same meaning as Cisco.  From what I read I should "tag" a port when another switch/router is connected to the port.  When a laptop/device is connected to a port then it needs to be untagged.  This setup was the only way I could get a DHCP address in vlan 10.

    Let me clarify:

    Doesn't traffic get tagged on Port 1?  Port 1 is tagged in vlan 10.  Due to different manufacturers using terms that mean different things this can get confusing.  Don't you think the traffic going to PFSense from vlan 10 is being scene as "tagged" in vlan 10 since I get a dhcp address in the range defined for vlan 10.  My laptop receives a dhcp address of 10.10.10.100.

    Here is a screen shot of my vswitches in esxi

    So I can ping 10.10.10.1 when I am connected to the 192.168.2.0/24 network.
    but
    When I am connected to and receive the vlan 10 dhcp address of 10.10.10.100.  I cannot ping 10.10.10.1  The default gateway.



  • Rebel Alliance Global Moderator

    So you get a dhcp address in vlan 10??  Well what are you rules on your vlan interface?  When you create new interfaces there are NO rules created.. Other than when you enable dhcp it creates some hidden rules that allow access to the dhcp server..  But until you create rules your not doing anything else

    So example I allow anything on ps3 network to talk to any port on pfsense ps3 address.  So ping, UPnP, dns, etc..  And as long as not trying to talk to other local networks it can go there..

    but your eth 1 setup is native 1 and untagged 1..  Why do you have it in there if you want it to tag 10 for traffic it sees?  Should that be native 10 and tagged 10 so that traffic it sees that is untagged will get tagged as 10 that is the way I read the hypbrid setup from dlink I just looked at.  Not sure why you have 1 listed in there at all if this is a access port you want in vlan 10??

    I agree other makers call things different.. I have an older netgear in the living room.  So the uplink to my cisco is on port 4..  So that is tagged 1 and 20.. Then ports are in untagged 1 and untagged 10.. So traffic it sees from untagged ports in the vlans with the pvid being set to the ports as 1 or 20..  Yeah yeah I know bad idea to use vlan 1…  But this is home setup and not real worried about it - makes it easier for setup.

    Normally vlan 1 should not be used and all ports should be removed from it, etc.  Use some other vlan as your native vlan, etc.








  • I posted a screenshot of the firewall rule in the first post called "Firewall rule.png".  I was trying to be as thorough as possible when explaining what I have done and what the problem is.

    I have also added a rule similar to your first rule with no change.  I still could not ping 10.10.10.1 nor access the internet.

    Bill



  • So I change Port 1 to be native to vlan 10.

    Same results.  I get DHCP from vlan 10 but cannot ping 10.10.10.1 nor can I access the internet.


  • Rebel Alliance Global Moderator

    Well what I would suggest then is sniff on psense for these pings.. So you see them?



  • Would you mind explaining that procedure?


  • Rebel Alliance Global Moderator

    click on diag in pfsense and do a packet capture.. Do you see the echo request, the replies?  do sniff on your actual interface in pfsense your lan, and then do on your vlan interface..  Where are you seeing the pings if at all?

    Could be firewall on client not sending them?



  • Here is a packet capture.  i captured ping traffic from 10.10.10.100 to the vlan 10 interface

    10:11:02.198128 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8799, length 64
    10:11:03.199463 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8800, length 64
    10:11:04.200840 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8801, length 64
    10:11:05.201992 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8802, length 64
    10:11:06.203285 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8803, length 64
    10:11:07.203872 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8804, length 64
    10:11:08.204284 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8805, length 64
    10:11:09.205265 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8806, length 64
    10:11:10.206324 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8807, length 64
    10:11:11.207325 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8808, length 64
    10:11:12.208180 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8809, length 64
    10:11:13.208670 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8810, length 64
    10:11:14.209595 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8811, length 64
    10:11:15.210582 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8812, length 64
    10:11:16.211597 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8813, length 64
    10:11:17.212543 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8814, length 64
    10:11:18.213504 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8815, length 64
    10:11:19.214646 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8816, length 64
    10:11:20.215250 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8817, length 64
    10:11:21.215954 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8818, length 64
    10:11:22.217061 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8819, length 64
    10:11:23.217931 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8820, length 64
    10:11:24.219165 IP 10.10.10.100 > 10.10.10.1: ICMP echo request, id 45428, seq 8821, length 64

    So I told 10.10.10.100 to go to www.google.com.  Here is the results of the packet capture

    10:15:05.969468 IP 10.10.10.100.56174 > 192.168.2.182.80: tcp 0
    10:15:20.102280 IP 10.10.10.100.56212 > 192.168.2.182.80: tcp 0
    10:15:21.168320 IP 10.10.10.100.56212 > 192.168.2.182.80: tcp 0
    10:15:22.075780 IP 10.10.10.100.56174 > 192.168.2.182.80: tcp 0
    10:15:22.175972 IP 10.10.10.100.56212 > 192.168.2.182.80: tcp 0
    10:15:22.176013 IP 10.10.10.100.56134 > 192.168.2.182.80: tcp 0
    10:15:23.186322 IP 10.10.10.100.56212 > 192.168.2.182.80: tcp 0
    10:15:24.195843 IP 10.10.10.100.56212 > 192.168.2.182.80: tcp 0
    10:15:25.203940 IP 10.10.10.100.56212 > 192.168.2.182.80: tcp 0
    10:15:27.220898 IP 10.10.10.100.56212 > 192.168.2.182.80: tcp 0
    10:15:31.279850 IP 10.10.10.100.56212 > 192.168.2.182.80: tcp 0

    So 192.168.2.182 is the Dlink Switch that the VLAN is running through.


  • Netgate

    Then it's your firewall rules on the pfSense interface.  What are those?



  • Here are screen shots of the WAN, LAN and VLAN 10 (aka Perk)

    Do you think it is defaulting to the switch since it can not ping 10.10.10.1 (default gateway)

    ![LAN Firewall Rules.PNG](/public/imported_attachments/1/LAN Firewall Rules.PNG)
    ![LAN Firewall Rules.PNG_thumb](/public/imported_attachments/1/LAN Firewall Rules.PNG_thumb)
    ![VLAN 10 Firewall Rules.PNG](/public/imported_attachments/1/VLAN 10 Firewall Rules.PNG)
    ![VLAN 10 Firewall Rules.PNG_thumb](/public/imported_attachments/1/VLAN 10 Firewall Rules.PNG_thumb)
    ![WAN Firewall Rules.PNG](/public/imported_attachments/1/WAN Firewall Rules.PNG)
    ![WAN Firewall Rules.PNG_thumb](/public/imported_attachments/1/WAN Firewall Rules.PNG_thumb)


  • Netgate

    So, to you, google is 192.168.2.182?  WTF?



  • Agreed


  • Rebel Alliance Global Moderator

    I don't see any response  Is your response going out different interface?

    And yeah why would you be going to 192.168.2??  Your switch?  So you have it doing L3 routing??  Makes no sense what so ever…  Even if you doing that - why would pfsense see that??

    Please draw up your network and connections..  And exactly is this switch?  You have it in layer 3 or layer 2 mode?  What are you using for name resolution..  So from your laptop you ping www.google.com what does it resolver to be it you get an answer or not?

    example
    C:>ping www.google.com

    Pinging www.google.com [173.194.219.104] with 32 bytes of data:
    Reply from 173.194.219.104: bytes=32 time=37ms TTL=43
    Reply from 173.194.219.104: bytes=32 time=31ms TTL=43

    See how it resolves to public IP.. How is google resolving and going to a 192.168 address..  So when you sniff that traffic you see it going to the PUBLIC IP not the layer 2 mac address of your gateway..

    Why are you blocking out stuff on your wan rules??  So why would it matter what 192.168 your forwarding too?  Or what port you have open on a public IP we don't even know.. etc.. Here are my wan rules.. What in there is of any use to you?  There is nothing there that could tell you what my public IP is.. And so what if you know I forward ntp to 192.168.9.40 etc. etc..






  • Will do.

    Not sure what it was capturing before but here is a current capture when I was trying to access Gmail and google

    11:24:57.292137 IP 10.10.10.100.26355 > 10.10.10.1.53: UDP, length 32
    11:24:58.610403 IP 10.10.10.100.61779 > 10.10.10.1.53: UDP, length 47
    11:24:58.610480 IP 10.10.10.100.60416 > 10.10.10.1.53: UDP, length 34
    11:24:58.610820 IP 10.10.10.100.53777 > 10.10.10.1.53: UDP, length 37
    11:24:58.611130 IP 10.10.10.100.34032 > 10.10.10.1.53: UDP, length 33
    11:24:59.467975 IP 10.10.10.100.55954 > 10.10.10.1.53: UDP, length 31
    11:24:59.468723 IP 10.10.10.100.48083 > 10.10.10.1.53: UDP, length 33
    11:24:59.619083 IP 10.10.10.100.12233 > 10.10.10.1.53: UDP, length 33
    11:24:59.683743 IP 10.10.10.100.61779 > 10.10.10.1.53: UDP, length 47
    11:24:59.683783 IP 10.10.10.100.60416 > 10.10.10.1.53: UDP, length 34
    11:24:59.683790 IP 10.10.10.100.53777 > 10.10.10.1.53: UDP, length 37
    11:25:00.476468 IP 10.10.10.100.16616 > 10.10.10.1.53: UDP, length 33
    11:25:00.543845 IP 10.10.10.100.55954 > 10.10.10.1.53: UDP, length 31
    11:25:01.634207 IP 10.10.10.100.62528 > 10.10.10.1.53: UDP, length 33


  • Rebel Alliance Global Moderator

    and that is 53.. So yeah its asking hey dns server 10.10.10.1 what is IP address of whatever it you were doing a query for.. Doesn't seem to be getting an answer.  Do you have dns listening on that IP?

    I don't see pfsense sending any answers not to ping or dns query.



  • So I am wondering if my issue could be linked to the following:

    My network has been a flat network with unmanaged switch.  Now I am implementing a managed switch with default vlan 1.  My flat network is 192.168.2.0/24.  It currently has internet utilizing pfsense.

    Port 24 that my laptop plugs into has vlan 10 "untagged' along with vlan 1.  Although the dlink allowed me to untagg both vlan's on the same port, I am wondering if this could be the source of my problem.

    Here is something new that I found this morning while troubleshooting:

    Earlier I reported that while my laptop was in vlan 10 w an address of 10.10.10.100 and a defult gateway of 10.10.10.1.  I could not ping 10.10.10.1.  When I am on the flat network 192.168.2.0/24, I can ping 10.10.10.1.  This morning I created a firewall rule that allows vlan10 to the lan.  I could then ping 10.10.10.1 when my laptop was in vlan10.

    Thoughts?

    Thanks for helping out
    Bill


  • Rebel Alliance Global Moderator

    if your laptop is suppose to be in 10.10.10 and this is vlan 10.. Then the only network on that switch port should be vlan 10.  You should be able to ping pfsense on 10.10.10.1 from vlan you sould need in any rules to allow vlan 10 to lan.. But pretty sure your rules was any any so that automatic gives access to any other lan or segments, etc.

    You should also be able to ping the 192.168.2 network from vlan 10..  Until you create such a rule to block it on the vlan 10 interface in pfsense.


  • Netgate

    @bhunter:

    Port 24 that my laptop plugs into has vlan 10 "untagged' along with vlan 1.  Although the dlink allowed me to untagg both vlan's on the same port, I am wondering if this could be the source of my problem.

    Whatever else might be going on, you cannot have two untagged VLANs on a port - trunk, access, hybrid, dual-mode, general or whatever.

    Access ports to your laptop should be untagged VLAN 10.



  • So I finally just resolve the issue.

    1.  Mistake I made was to have 2 vlan's untagged on port 24.  I changed the port to an access port untagged in vlan 10.

    2.  I did not have an outbound nat rule for 10.10.10.0/24 network.  This was probably my fault because at one time I set it to manual, so the route did not get auto created.

    3.  I had to reboot PFsense for the new nat rule to take affect.

    I wanted to thank you guys for helping out!
    Bill


  • Netgate

    I had to reboot PFsense for the new nat rule to take affect.

    No, you didn't but glad it's working.  What's with these switches allowing multiple VLANs untagged on a port? That's twice today.