Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Setting up a VLAN with pfSense, Ubiquiti, and ESXi

    L2/Switching/VLANs
    5
    66
    1830
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      pfSenseUser78 last edited by pfSenseUser78

      All,

      Trying to setup my first VLAN with pfSense. I've googled and searched and can't find where I'm going wrong.

      I've followed this guide:https://nguvu.org/pfsense/pfsense-baseline-setup/ and have no idea why it's not working.

      Short version is I enabled the VLAN in pfSense, added a DHCP range that isn't apart of my existing network, added the VLAN (90) as a newton in the Ubiquity software, and manually created Firewall rules. I still can't get the DHCP server running on the firewall to hand out IP addresses.

      The firewall is pfSense running on a ESXi instance. Switch and AP are both Ubiquiti.

      Please let me know whatever additional information needed to troubleshoot. I saw something about using VLAN 4095 on the ESXi host but I couldn't find where to do that. pfSense is currently running with one physical NIC for the WAN and one physical NIC for the LAN.

      Thanks!

      JKnott 1 Reply Last reply Reply Quote 0
      • JKnott
        JKnott @pfSenseUser78 last edited by

        @pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:

        added a DHCP range that isn't apart of my existing network,

        Did you just add a range to the existing DHCP config or create another config for the VLAN? Each interface requires it's own separate configuration. You should see the VLAN interface on the DHCP server tab.

        PfSense running on Qotom mini PC
        i5 CPU, 4 GB memory, 64 GB SSD & 4 Intel Gb Ethernet ports.
        UniFi AC-Lite access point

        I haven't lost my mind. It's around here...somewhere...

        P 1 Reply Last reply Reply Quote 0
        • P
          pfSenseUser78 last edited by

          This post is deleted!
          1 Reply Last reply Reply Quote 0
          • P
            pfSenseUser78 @JKnott last edited by

            @JKnott

            I created another config for the VLAN (I think). Internal DHCP is handled by a Window Server. VLAN 90 (for IoT) can be DHCP server from pfSense.

            JKnott 1 Reply Last reply Reply Quote 0
            • JKnott
              JKnott @pfSenseUser78 last edited by

              @pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:

              @JKnott

              I created another config for the VLAN (I think). Internal DHCP is handled by a Window Server. VLAN 90 (for IoT) can be DHCP server from pfSense.

              I thought you said the DHCP server was on the firewall. Now you're saying it's on the Windows server. Which is it? If it's on the Windows server, then the problem has nothing to do with pfSense.

              PfSense running on Qotom mini PC
              i5 CPU, 4 GB memory, 64 GB SSD & 4 Intel Gb Ethernet ports.
              UniFi AC-Lite access point

              I haven't lost my mind. It's around here...somewhere...

              P 1 Reply Last reply Reply Quote 0
              • johnpoz
                johnpoz LAYER 8 Global Moderator last edited by

                @pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:

                I saw something about using VLAN 4095 on the ESXi host but

                If you want to pass tags to pfsense virtual nic, then yes you would have to set the vlan ID on your esxi vswitch/port group to be 4095 or it will strip tags.. Pfsense can not see vlans then.

                An intelligent man is sometimes forced to be drunk to spend time with his fools
                If you get confused: Listen to the Music Play
                Please don't Chat/PM me for help, unless mod related
                2440 2.4.5p1 | 2x 3100 2.4.4p3 | 2x 3100 22.01 | 4860 22.05

                P 1 Reply Last reply Reply Quote 1
                • P
                  pfSenseUser78 @JKnott last edited by

                  @JKnott

                  For the internal NON-IoT network DHCP is handled by a windows domain controller. For the VLAN being setup for the IoT devices I do NOT need DHCP handled by a windows DC; I was saying that this can be handled by pfSense directly.

                  JKnott 1 Reply Last reply Reply Quote 0
                  • P
                    pfSenseUser78 @johnpoz last edited by pfSenseUser78

                    @johnpoz said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:

                    @pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:

                    I saw something about using VLAN 4095 on the ESXi host but

                    If you want to pass tags to pfsense virtual nic, then yes you would have to set the vlan ID on your esxi vswitch/port group to be 4095 or it will strip tags.. Pfsense can not see vlans then.

                    Ok, so, after finding where to make that change AND making it, I now get a DHCP IP address in the 192.168.90.x/24 range. Hurray!

                    My next issue is that DNS doesn't appear to be working. I'd like to point DNS to my two internal DNS servers in the 172.16.249.x/24 range as I'm not using pfSense for DNS either; how do I do that? If you can point me in the right direction it'd be much appreciated.

                    1 Reply Last reply Reply Quote 0
                    • johnpoz
                      johnpoz LAYER 8 Global Moderator last edited by johnpoz

                      To point your clients to other NS, just edit your dhcp scope to had out what you want.. out of the box pfsense if running dns, be it unbound (resolver or can be forwarder mode) or dnsmasq (forwarder) will point to itself for dns. If not running either, then it will hand out what is in its general tab.

                      To hand out something other, like your internal NS - just edit the dhcp scope..

                      example - here I hand out 192.168.3.10 to clients in this network
                      dns.jpg

                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                      If you get confused: Listen to the Music Play
                      Please don't Chat/PM me for help, unless mod related
                      2440 2.4.5p1 | 2x 3100 2.4.4p3 | 2x 3100 22.01 | 4860 22.05

                      P 1 Reply Last reply Reply Quote 0
                      • JKnott
                        JKnott @pfSenseUser78 last edited by

                        @pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:

                        @JKnott

                        For the internal NON-IoT network DHCP is handled by a windows domain controller. For the VLAN being setup for the IoT devices I do NOT need DHCP handled by a windows DC; I was saying that this can be handled by pfSense directly.

                        Then we're back to my original question, did you set up the DHCP config on the VLAN? In the DHCP config, there are pages for each interface. The way I read your original post, it sounded like you just added an address pool to the main interface.

                        PfSense running on Qotom mini PC
                        i5 CPU, 4 GB memory, 64 GB SSD & 4 Intel Gb Ethernet ports.
                        UniFi AC-Lite access point

                        I haven't lost my mind. It's around here...somewhere...

                        P 1 Reply Last reply Reply Quote 0
                        • P
                          pfSenseUser78 @johnpoz last edited by pfSenseUser78

                          @johnpoz

                          So this is what I have for DNS in the VLAN DHCP config:
                          DHCP w-DNS.png

                          I have this as a firewall rule:
                          Firewall.png

                          When I connect to the network on my iPhone I connect and get an IP address, however, the spinny thing next to the network name never goes away. If I try to load a website, nothing happens. If I manually specific a DNS server, nothing happens. Note that on my non-VLAN network I have a rule blocking all DNS traffic that doesn't originate from one of the two internal servers; not sure if this would be affecting a VLAN on a different IP range or not (the greyed out rule was a failed attempt to redirect errant DNS queries back to one of the two internal DNS servers at 105 or 106 - 78 & 79 are there for other reasons):
                          LAN Rules.png

                          And here is the DNS servers setup in general:DNS.png

                          Please let me know if there is any additional information I can provide; I greatly appreciate everything so far.

                          1 Reply Last reply Reply Quote 0
                          • P
                            pfSenseUser78 @JKnott last edited by

                            @JKnott said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:

                            @pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:

                            @JKnott

                            For the internal NON-IoT network DHCP is handled by a windows domain controller. For the VLAN being setup for the IoT devices I do NOT need DHCP handled by a windows DC; I was saying that this can be handled by pfSense directly.

                            Then we're back to my original question, did you set up the DHCP config on the VLAN? In the DHCP config, there are pages for each interface. The way I read your original post, it sounded like you just added an address pool to the main interface.

                            VLAN90-Setup.png

                            .
                            .

                            .
                            VLAN90-DHCP.png

                            Hope this helps - if not let me know what else I can provide a screenshot of.

                            1 Reply Last reply Reply Quote 0
                            • Derelict
                              Derelict LAYER 8 Netgate last edited by

                              You are not setting DNS servers in the DHCP configuration so this is what DNS servers will be sent to the clients (just like it says right there):

                              Leave blank to use the system default DNS servers: this interface's IP if DNS Forwarder or Resolver is enabled, otherwise the servers configured on the System / General Setup page.

                              So it depends on whether or not you have the resolver or forwarder configured, in which case the clients will get 192.168.90.200 as their DNS server. If DNS forwarder/resolver are NOT configured, they'll get whatever is configured in System > General.

                              Chattanooga, Tennessee, USA
                              The pfSense Book is free of charge!
                              DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                              Do Not Chat For Help! NO_WAN_EGRESS(TM)

                              P 1 Reply Last reply Reply Quote 0
                              • P
                                pfSenseUser78 @Derelict last edited by

                                @Derelict

                                Here's what's set as my DNS servers in System > General:

                                Screenshot from 2019-11-29 17-06-31.png

                                Could it be that the VLAN devices can't connect to the DNS servers?

                                1 Reply Last reply Reply Quote 0
                                • Derelict
                                  Derelict LAYER 8 Netgate last edited by

                                  Look at the DNS servers the clients are being assigned. Can the resolve names from them? If not, then yest that's a problem.

                                  Chattanooga, Tennessee, USA
                                  The pfSense Book is free of charge!
                                  DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                  P 1 Reply Last reply Reply Quote 0
                                  • P
                                    pfSenseUser78 @Derelict last edited by

                                    @Derelict DNS server being assigned to the clients is the firewall itself (172.16.249.200).

                                    1 Reply Last reply Reply Quote 0
                                    • Derelict
                                      Derelict LAYER 8 Netgate last edited by

                                      Can they resolve names using that?

                                      Chattanooga, Tennessee, USA
                                      The pfSense Book is free of charge!
                                      DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                      P 1 Reply Last reply Reply Quote 0
                                      • P
                                        pfSenseUser78 @Derelict last edited by

                                        @Derelict

                                        Ok. I can connect to the network (via WiFi) with my laptop. I get an IP address on the VLAN range. From there, I cannot ping 8.8.8.8 and I can't ping 192.168.90.200 (which should be the VLAN Firewall).

                                        Not being able to ping 8.8.8.8 tells me I might have a rule issue somewhere. I know I have one to block DNS queries on my main network (172.16.249.1/24) that don't originate from one of two DNS servers (on that same network) but that shouldn't block me from pinging 8.8.8.8.

                                        1 Reply Last reply Reply Quote 0
                                        • johnpoz
                                          johnpoz LAYER 8 Global Moderator last edited by johnpoz

                                          Vlans when created have zero rules on them - so not being able to do anything would be default.. What rules did you put on your vlan interface? Just that tcp rule, well no you wouldn't be able to do dns which is udp normally, and no you wouldn't be able to ping which is icmp..

                                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                                          If you get confused: Listen to the Music Play
                                          Please don't Chat/PM me for help, unless mod related
                                          2440 2.4.5p1 | 2x 3100 2.4.4p3 | 2x 3100 22.01 | 4860 22.05

                                          P 1 Reply Last reply Reply Quote 0
                                          • Derelict
                                            Derelict LAYER 8 Netgate last edited by Derelict

                                            Two things:

                                            1. Rules on the new interface passing traffic from the hosts on that interface.
                                            2. Outbound NAT on the WAN for the new interface source addresses if you strayed from Automatic or Hybrid outbound NAT.

                                            Those two things are installed by default on the LAN but not for any interfaces you might subsequently create.

                                            Chattanooga, Tennessee, USA
                                            The pfSense Book is free of charge!
                                            DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                            Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                            P 1 Reply Last reply Reply Quote 0
                                            • P
                                              pfSenseUser78 @johnpoz last edited by pfSenseUser78

                                              @johnpoz Makes sense, where would I find the rules I need to create? I've searched but I've not been able to find what they are.

                                              Edit: I've also looked at the default rules and don't see what get setup to try and duplicate on the VLAN.

                                              1 Reply Last reply Reply Quote 0
                                              • P
                                                pfSenseUser78 @Derelict last edited by

                                                @Derelict I've read and reread these a couple of times and am having a hard time understanding. I have an "ok" grasp of networking but these just don't make sense.

                                                Is there somewhere I can read more about these two points?

                                                1 Reply Last reply Reply Quote 0
                                                • johnpoz
                                                  johnpoz LAYER 8 Global Moderator last edited by johnpoz

                                                  They would be what you want - there is no possible way to show someone the rules that they would want.. Everyones setup is going to be different. I would start with an any any rule to validate connectivity works, and then set them how you want them.. My rules are going to be different than your rules, etc.

                                                  edit?
                                                  You don't know how to setup a any any rule??

                                                  Here is basic vlan rules that allows basic services but blocks access to all other vlans (rfc1918)..

                                                  basicrules.jpg

                                                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                                                  If you get confused: Listen to the Music Play
                                                  Please don't Chat/PM me for help, unless mod related
                                                  2440 2.4.5p1 | 2x 3100 2.4.4p3 | 2x 3100 22.01 | 4860 22.05

                                                  P 1 Reply Last reply Reply Quote 0
                                                  • P
                                                    pfSenseUser78 @johnpoz last edited by

                                                    @johnpoz Is this the any any rule you're referring to?

                                                    Any Any.png

                                                    If so, it's already in place.

                                                    1 Reply Last reply Reply Quote 0
                                                    • johnpoz
                                                      johnpoz LAYER 8 Global Moderator last edited by johnpoz

                                                      That is not an ANY rule - its only allowing tcp.. So no dns which is udp, and no ping which icmp.

                                                      any.jpg

                                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                                      If you get confused: Listen to the Music Play
                                                      Please don't Chat/PM me for help, unless mod related
                                                      2440 2.4.5p1 | 2x 3100 2.4.4p3 | 2x 3100 22.01 | 4860 22.05

                                                      P 1 Reply Last reply Reply Quote 0
                                                      • P
                                                        pfSenseUser78 @johnpoz last edited by

                                                        @johnpoz

                                                        Got it now.

                                                        I've created this an will test tomorrow; the goal is to allow the devices on this VLAN access to the internet and access to the specified DNS servers on the local LAN (I hope that term is correct):
                                                        NEW Any Any.png

                                                        1 Reply Last reply Reply Quote 0
                                                        • johnpoz
                                                          johnpoz LAYER 8 Global Moderator last edited by johnpoz

                                                          You have dns set to TCP only - while dns CAN use tcp, it defaults to UDP... You reallly should make those rules UDP/TCP

                                                          But since you have any rule there at the bottom those 2 allow rules for dns are pretty pointless.

                                                          And that any rule at the end is going to allow access to your lan as well.. You need to put a block rule above the any and below your dns rules, that blocks access your LAN net, if you dont want your vlan to talk to your lan..

                                                          Rules are evaluated as traffic enters the interface from the network its attached too, top down, first rule to trigger wins, no other rules are evaluated.

                                                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                                                          If you get confused: Listen to the Music Play
                                                          Please don't Chat/PM me for help, unless mod related
                                                          2440 2.4.5p1 | 2x 3100 2.4.4p3 | 2x 3100 22.01 | 4860 22.05

                                                          P 1 Reply Last reply Reply Quote 0
                                                          • P
                                                            pfSenseUser78 @johnpoz last edited by pfSenseUser78

                                                            @johnpoz

                                                            Ok, I think I'm getting closer. Here's the new rules:
                                                            VLan90 Firewall Rules 4.png

                                                            If I understand correctly the rules flow from the top down to the bottom. I'm allowing (on the VLAN90):

                                                            1. Ping responses anywhere (but should probably tighten this up to only allow responses out to the internet if I'm trying to segregate this traffic from my internal LAN)

                                                            2. DNS traffic ONLY to 172.16.249.139 and then .138

                                                            3. Blocking all communication from the VLAN90 to the local LAN

                                                            4. All connectivity out to the internet (although, like rule 1, I'm guessing that this rule is too broad and can be tightened up).

                                                            For rules 1 and 4, to "tighten" them up to be just applicable to the internet would I change the destination to "WAN net"?

                                                            1 Reply Last reply Reply Quote 0
                                                            • johnpoz
                                                              johnpoz LAYER 8 Global Moderator last edited by johnpoz

                                                              Wan net is not the internet - its just the network connect to your wan.. It would be something/mask like mine is 64.53.x.x/23 The internet is really anything.. There is no way to tighten that up... Can you put in all address blocks that make up the internet ;)

                                                              If you don't want clients on your vlan to ping stuff on your lan, then put a rule that blocks that before you allow ping any..

                                                              If you want to tighten it up more, you might want to use the "this firewall" alias to prevent access to your wan IP, this alias includes all IPs that firewall has, a block rule to that on your vlan would prevent access to say the pfsense gui via the wan IP.. See my example rules.

                                                              If you ever have a question to if something is allowed or deny, just run through the rules with where your going and what protocol.. Is it denied or allowed in the rules - if it gets to the end without being allowed then it would be denied.. Since the default is denied (not shown).

                                                              An intelligent man is sometimes forced to be drunk to spend time with his fools
                                                              If you get confused: Listen to the Music Play
                                                              Please don't Chat/PM me for help, unless mod related
                                                              2440 2.4.5p1 | 2x 3100 2.4.4p3 | 2x 3100 22.01 | 4860 22.05

                                                              P 2 Replies Last reply Reply Quote 0
                                                              • P
                                                                pfSenseUser78 @johnpoz last edited by

                                                                @johnpoz

                                                                THANK YOU for your help so far; I've almost got it. Here's my rules now (I plan on renaming the descriptions later):
                                                                VLAN 90 Firewall 6.png

                                                                My only remaining issue is that I can't get DNS to work with the rules the way they are now. From the VLAN I'm trying to use the DNS server on my LAN. I thought that rules 2 & 3 would allow the traffic to pass but after putting the 4th rule in place (to prevent the devices on VLAN90 form being able to see the devices on the LAN) DNS stops working entirely.

                                                                Should rules 2 & 3 just be the firewall IP address (which would then pass it off to the DNS servers on the LAN) or is there something I'm missing?

                                                                Also, I installed Avahi as one of my goals was to have my Apple devices on VLAN90 but still be able to Airplay to them; it seems to be working so far.

                                                                Thank you again.

                                                                1 Reply Last reply Reply Quote 0
                                                                • P
                                                                  pfSenseUser78 @johnpoz last edited by

                                                                  @johnpoz Would it make sense to start a new post at this point?

                                                                  Thank you again for all your (and everyone else's help).

                                                                  1 Reply Last reply Reply Quote 0
                                                                  • johnpoz
                                                                    johnpoz LAYER 8 Global Moderator last edited by johnpoz

                                                                    I see no hits on your rules for dns.. see the 0/0's - so your clients never sent anything to those IPs on port 53, or you would see hits there..

                                                                    Do you have anything in floating?

                                                                    VLAN90 form being able to see the devices on the LAN) DNS stops working entirely.

                                                                    Only way that would be is if your clients where actually asking pfsense for dns, or different IPs then what you have listed.. Yes your rfc1918 rule is getting hits. And so is your this firewall rule.

                                                                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                                                                    If you get confused: Listen to the Music Play
                                                                    Please don't Chat/PM me for help, unless mod related
                                                                    2440 2.4.5p1 | 2x 3100 2.4.4p3 | 2x 3100 22.01 | 4860 22.05

                                                                    1 Reply Last reply Reply Quote 0
                                                                    • M
                                                                      marvosa last edited by marvosa

                                                                      Is the goal for this to be an internet-only VLAN? Building on that question, is this for your own internal equipment or is this going to be a "guest" VLAN? In either case, many of these rules can be collapsed into a simpler ruleset, IMO:

                                                                      For an internet-only Guest VLAN:

                                                                      1. Configure DHCP to hand out PFsense (or public DNS) for DNS. Then collapse your rules down to:

                                                                      Block -> TCP/VL90 net/This firewall/port (alias for 22 and whatever port your GUI is listening on)
                                                                      Allow -> VL90 net/Invert match rfc1918 alias

                                                                      For an internal, internet-only VLAN where you want your devices to use your internal DNS servers:

                                                                      Allow -> (TCP/UDP)/VL90 net/Alias for DNS servers/port 53
                                                                      Block -> TCP/VL90 net/This firewall/port (alias for 22 and whatever port your GUI is listening on)
                                                                      Allow -> VL90 net/Invert match rfc1918 alias

                                                                      and TBH, unless there is a specific need for your internet-only traffic to use your internal DNS servers (which I assume are your DC's), I'd go with the first option.

                                                                      P 1 Reply Last reply Reply Quote 0
                                                                      • P
                                                                        pfSenseUser78 @marvosa last edited by pfSenseUser78

                                                                        @marvosa The idea is an internet only VLAN (it would be nice if Airplay worked between the VLAN and LAN but not necessary) HOWEVER DNS would be pointed to my two internal DNS servers. Anything and everything else on the VLAN would not have access to the LAN.

                                                                        This is my IoT VLAN; I'm looking to use my internal DNS servers for filtering purposes. For any VLAN I setup I'd want to point it to my internal DNS servers but everything else would be Internet only.

                                                                        M 1 Reply Last reply Reply Quote 0
                                                                        • M
                                                                          marvosa @pfSenseUser78 last edited by

                                                                          @pfSenseUser78 Then I'd go with the 2nd set of rules I posted. They'll be more streamlined for your use case.

                                                                          P 2 Replies Last reply Reply Quote 0
                                                                          • P
                                                                            pfSenseUser78 @marvosa last edited by pfSenseUser78

                                                                            @marvosa Ok. In the second setup you state "Alias for DNS servers"; I'm not sure what that means. Would I be putting in the 172.x.x.x addresses or am I creating something else?

                                                                            Thanks!

                                                                            Edit: Ok, found where to make the alias. Trying now.

                                                                            M 1 Reply Last reply Reply Quote 0
                                                                            • P
                                                                              pfSenseUser78 @marvosa last edited by

                                                                              @marvosa So like this?

                                                                              Firewall 34.png

                                                                              (I think I've got this correct)

                                                                              1 Reply Last reply Reply Quote 0
                                                                              • M
                                                                                marvosa @pfSenseUser78 last edited by marvosa

                                                                                @pfSenseUser78 Edited

                                                                                I first said yes, but the looked at your rules again...hold on

                                                                                P 1 Reply Last reply Reply Quote 0
                                                                                • P
                                                                                  pfSenseUser78 @marvosa last edited by pfSenseUser78

                                                                                  @marvosa SSH is enabled. I created the port alias but I can't figure out how to apply that to the second firewall rule. It doesn't appear in the list of ports to block.

                                                                                  Edit: Found it here: https://docs.netgate.com/pfsense/en/latest/book/firewall/aliases.html

                                                                                  Testing now!

                                                                                  M 1 Reply Last reply Reply Quote 0
                                                                                  • M
                                                                                    marvosa @pfSenseUser78 last edited by

                                                                                    @pfSenseUser78 The last rule should be

                                                                                    Protocol = any
                                                                                    Source = VL90_IOT net
                                                                                    Destination = Invert match rfc1918 alias

                                                                                    P Derelict 2 Replies Last reply Reply Quote 0
                                                                                    • First post
                                                                                      Last post