Setting up a VLAN with pfSense, Ubiquiti, and ESXi
-
All,
Trying to setup my first VLAN with pfSense. I've googled and searched and can't find where I'm going wrong.
I've followed this guide:https://nguvu.org/pfsense/pfsense-baseline-setup/ and have no idea why it's not working.
Short version is I enabled the VLAN in pfSense, added a DHCP range that isn't apart of my existing network, added the VLAN (90) as a newton in the Ubiquity software, and manually created Firewall rules. I still can't get the DHCP server running on the firewall to hand out IP addresses.
The firewall is pfSense running on a ESXi instance. Switch and AP are both Ubiquiti.
Please let me know whatever additional information needed to troubleshoot. I saw something about using VLAN 4095 on the ESXi host but I couldn't find where to do that. pfSense is currently running with one physical NIC for the WAN and one physical NIC for the LAN.
Thanks!
-
@pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:
added a DHCP range that isn't apart of my existing network,
Did you just add a range to the existing DHCP config or create another config for the VLAN? Each interface requires it's own separate configuration. You should see the VLAN interface on the DHCP server tab.
-
This post is deleted! -
I created another config for the VLAN (I think). Internal DHCP is handled by a Window Server. VLAN 90 (for IoT) can be DHCP server from pfSense.
-
@pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:
I created another config for the VLAN (I think). Internal DHCP is handled by a Window Server. VLAN 90 (for IoT) can be DHCP server from pfSense.
I thought you said the DHCP server was on the firewall. Now you're saying it's on the Windows server. Which is it? If it's on the Windows server, then the problem has nothing to do with pfSense.
-
@pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:
I saw something about using VLAN 4095 on the ESXi host but
If you want to pass tags to pfsense virtual nic, then yes you would have to set the vlan ID on your esxi vswitch/port group to be 4095 or it will strip tags.. Pfsense can not see vlans then.
-
For the internal NON-IoT network DHCP is handled by a windows domain controller. For the VLAN being setup for the IoT devices I do NOT need DHCP handled by a windows DC; I was saying that this can be handled by pfSense directly.
-
@johnpoz said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:
@pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:
I saw something about using VLAN 4095 on the ESXi host but
If you want to pass tags to pfsense virtual nic, then yes you would have to set the vlan ID on your esxi vswitch/port group to be 4095 or it will strip tags.. Pfsense can not see vlans then.
Ok, so, after finding where to make that change AND making it, I now get a DHCP IP address in the 192.168.90.x/24 range. Hurray!
My next issue is that DNS doesn't appear to be working. I'd like to point DNS to my two internal DNS servers in the 172.16.249.x/24 range as I'm not using pfSense for DNS either; how do I do that? If you can point me in the right direction it'd be much appreciated.
-
To point your clients to other NS, just edit your dhcp scope to had out what you want.. out of the box pfsense if running dns, be it unbound (resolver or can be forwarder mode) or dnsmasq (forwarder) will point to itself for dns. If not running either, then it will hand out what is in its general tab.
To hand out something other, like your internal NS - just edit the dhcp scope..
example - here I hand out 192.168.3.10 to clients in this network
-
@pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:
For the internal NON-IoT network DHCP is handled by a windows domain controller. For the VLAN being setup for the IoT devices I do NOT need DHCP handled by a windows DC; I was saying that this can be handled by pfSense directly.
Then we're back to my original question, did you set up the DHCP config on the VLAN? In the DHCP config, there are pages for each interface. The way I read your original post, it sounded like you just added an address pool to the main interface.
-
So this is what I have for DNS in the VLAN DHCP config:
I have this as a firewall rule:
When I connect to the network on my iPhone I connect and get an IP address, however, the spinny thing next to the network name never goes away. If I try to load a website, nothing happens. If I manually specific a DNS server, nothing happens. Note that on my non-VLAN network I have a rule blocking all DNS traffic that doesn't originate from one of the two internal servers; not sure if this would be affecting a VLAN on a different IP range or not (the greyed out rule was a failed attempt to redirect errant DNS queries back to one of the two internal DNS servers at 105 or 106 - 78 & 79 are there for other reasons):
And here is the DNS servers setup in general:
Please let me know if there is any additional information I can provide; I greatly appreciate everything so far.
-
@JKnott said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:
@pfSenseUser78 said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:
For the internal NON-IoT network DHCP is handled by a windows domain controller. For the VLAN being setup for the IoT devices I do NOT need DHCP handled by a windows DC; I was saying that this can be handled by pfSense directly.
Then we're back to my original question, did you set up the DHCP config on the VLAN? In the DHCP config, there are pages for each interface. The way I read your original post, it sounded like you just added an address pool to the main interface.
.
..
Hope this helps - if not let me know what else I can provide a screenshot of.
-
You are not setting DNS servers in the DHCP configuration so this is what DNS servers will be sent to the clients (just like it says right there):
Leave blank to use the system default DNS servers: this interface's IP if DNS Forwarder or Resolver is enabled, otherwise the servers configured on the System / General Setup page.
So it depends on whether or not you have the resolver or forwarder configured, in which case the clients will get 192.168.90.200 as their DNS server. If DNS forwarder/resolver are NOT configured, they'll get whatever is configured in System > General.
-
Here's what's set as my DNS servers in System > General:
Could it be that the VLAN devices can't connect to the DNS servers?
-
Look at the DNS servers the clients are being assigned. Can the resolve names from them? If not, then yest that's a problem.
-
@Derelict DNS server being assigned to the clients is the firewall itself (172.16.249.200).
-
Can they resolve names using that?
-
Ok. I can connect to the network (via WiFi) with my laptop. I get an IP address on the VLAN range. From there, I cannot ping 8.8.8.8 and I can't ping 192.168.90.200 (which should be the VLAN Firewall).
Not being able to ping 8.8.8.8 tells me I might have a rule issue somewhere. I know I have one to block DNS queries on my main network (172.16.249.1/24) that don't originate from one of two DNS servers (on that same network) but that shouldn't block me from pinging 8.8.8.8.
-
Vlans when created have zero rules on them - so not being able to do anything would be default.. What rules did you put on your vlan interface? Just that tcp rule, well no you wouldn't be able to do dns which is udp normally, and no you wouldn't be able to ping which is icmp..
-
Two things:
- Rules on the new interface passing traffic from the hosts on that interface.
- Outbound NAT on the WAN for the new interface source addresses if you strayed from Automatic or Hybrid outbound NAT.
Those two things are installed by default on the LAN but not for any interfaces you might subsequently create.
-
@johnpoz Makes sense, where would I find the rules I need to create? I've searched but I've not been able to find what they are.
Edit: I've also looked at the default rules and don't see what get setup to try and duplicate on the VLAN.
-
@Derelict I've read and reread these a couple of times and am having a hard time understanding. I have an "ok" grasp of networking but these just don't make sense.
Is there somewhere I can read more about these two points?
-
They would be what you want - there is no possible way to show someone the rules that they would want.. Everyones setup is going to be different. I would start with an any any rule to validate connectivity works, and then set them how you want them.. My rules are going to be different than your rules, etc.
edit?
You don't know how to setup a any any rule??Here is basic vlan rules that allows basic services but blocks access to all other vlans (rfc1918)..
-
-
That is not an ANY rule - its only allowing tcp.. So no dns which is udp, and no ping which icmp.
-
Got it now.
I've created this an will test tomorrow; the goal is to allow the devices on this VLAN access to the internet and access to the specified DNS servers on the local LAN (I hope that term is correct):
-
You have dns set to TCP only - while dns CAN use tcp, it defaults to UDP... You reallly should make those rules UDP/TCP
But since you have any rule there at the bottom those 2 allow rules for dns are pretty pointless.
And that any rule at the end is going to allow access to your lan as well.. You need to put a block rule above the any and below your dns rules, that blocks access your LAN net, if you dont want your vlan to talk to your lan..
Rules are evaluated as traffic enters the interface from the network its attached too, top down, first rule to trigger wins, no other rules are evaluated.
-
Ok, I think I'm getting closer. Here's the new rules:
If I understand correctly the rules flow from the top down to the bottom. I'm allowing (on the VLAN90):
-
Ping responses anywhere (but should probably tighten this up to only allow responses out to the internet if I'm trying to segregate this traffic from my internal LAN)
-
DNS traffic ONLY to 172.16.249.139 and then .138
-
Blocking all communication from the VLAN90 to the local LAN
-
All connectivity out to the internet (although, like rule 1, I'm guessing that this rule is too broad and can be tightened up).
For rules 1 and 4, to "tighten" them up to be just applicable to the internet would I change the destination to "WAN net"?
-
-
Wan net is not the internet - its just the network connect to your wan.. It would be something/mask like mine is 64.53.x.x/23 The internet is really anything.. There is no way to tighten that up... Can you put in all address blocks that make up the internet ;)
If you don't want clients on your vlan to ping stuff on your lan, then put a rule that blocks that before you allow ping any..
If you want to tighten it up more, you might want to use the "this firewall" alias to prevent access to your wan IP, this alias includes all IPs that firewall has, a block rule to that on your vlan would prevent access to say the pfsense gui via the wan IP.. See my example rules.
If you ever have a question to if something is allowed or deny, just run through the rules with where your going and what protocol.. Is it denied or allowed in the rules - if it gets to the end without being allowed then it would be denied.. Since the default is denied (not shown).
-
THANK YOU for your help so far; I've almost got it. Here's my rules now (I plan on renaming the descriptions later):
My only remaining issue is that I can't get DNS to work with the rules the way they are now. From the VLAN I'm trying to use the DNS server on my LAN. I thought that rules 2 & 3 would allow the traffic to pass but after putting the 4th rule in place (to prevent the devices on VLAN90 form being able to see the devices on the LAN) DNS stops working entirely.
Should rules 2 & 3 just be the firewall IP address (which would then pass it off to the DNS servers on the LAN) or is there something I'm missing?
Also, I installed Avahi as one of my goals was to have my Apple devices on VLAN90 but still be able to Airplay to them; it seems to be working so far.
Thank you again.
-
@johnpoz Would it make sense to start a new post at this point?
Thank you again for all your (and everyone else's help).
-
I see no hits on your rules for dns.. see the 0/0's - so your clients never sent anything to those IPs on port 53, or you would see hits there..
Do you have anything in floating?
VLAN90 form being able to see the devices on the LAN) DNS stops working entirely.
Only way that would be is if your clients where actually asking pfsense for dns, or different IPs then what you have listed.. Yes your rfc1918 rule is getting hits. And so is your this firewall rule.
-
Is the goal for this to be an internet-only VLAN? Building on that question, is this for your own internal equipment or is this going to be a "guest" VLAN? In either case, many of these rules can be collapsed into a simpler ruleset, IMO:
For an internet-only Guest VLAN:
- Configure DHCP to hand out PFsense (or public DNS) for DNS. Then collapse your rules down to:
Block -> TCP/VL90 net/This firewall/port (alias for 22 and whatever port your GUI is listening on)
Allow -> VL90 net/Invert match rfc1918 aliasFor an internal, internet-only VLAN where you want your devices to use your internal DNS servers:
Allow -> (TCP/UDP)/VL90 net/Alias for DNS servers/port 53
Block -> TCP/VL90 net/This firewall/port (alias for 22 and whatever port your GUI is listening on)
Allow -> VL90 net/Invert match rfc1918 aliasand TBH, unless there is a specific need for your internet-only traffic to use your internal DNS servers (which I assume are your DC's), I'd go with the first option.
-
@marvosa The idea is an internet only VLAN (it would be nice if Airplay worked between the VLAN and LAN but not necessary) HOWEVER DNS would be pointed to my two internal DNS servers. Anything and everything else on the VLAN would not have access to the LAN.
This is my IoT VLAN; I'm looking to use my internal DNS servers for filtering purposes. For any VLAN I setup I'd want to point it to my internal DNS servers but everything else would be Internet only.
-
@pfSenseUser78 Then I'd go with the 2nd set of rules I posted. They'll be more streamlined for your use case.
-
@marvosa Ok. In the second setup you state "Alias for DNS servers"; I'm not sure what that means. Would I be putting in the 172.x.x.x addresses or am I creating something else?
Thanks!
Edit: Ok, found where to make the alias. Trying now.
-
-
@pfSenseUser78 Edited
I first said yes, but the looked at your rules again...hold on
-
@marvosa SSH is enabled. I created the port alias but I can't figure out how to apply that to the second firewall rule. It doesn't appear in the list of ports to block.
Edit: Found it here: https://docs.netgate.com/pfsense/en/latest/book/firewall/aliases.html
Testing now!
-
@pfSenseUser78 The last rule should be
Protocol = any
Source = VL90_IOT net
Destination = Invert match rfc1918 alias
