Setting up a VLAN with pfSense, Ubiquiti, and ESXi

Derelict

@johnpoz said in Setting up a VLAN with pfSense, Ubiquiti, and ESXi:

I have removed my use of the ! rfc1918 rules and have adopted the explicit reject rfc1918 above my any as cleaner method.

If it saves a single rule set it's worth it.

Derelict

This post is deleted!

pfSenseUser78

@Derelict

I'd like to take your advice as you absolutely sound like you know what you're doing. What do I need to change (nothing has changed yet from the above screenshots)?

The DNS servers I'm attempting to use to do not have a firewall enabled that I know of. I do not know how they would have any configuration that prevents them from answering queries from the VLAN. They both work fine for any and all clients on the LAN.

Here's the output of dig:

dig @172.16.249.138 www.google.com

; <<>> DiG 9.11.3-1ubuntu1.11-Ubuntu <<>> @172.16.249.138 www.google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5224
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.google.com.			IN	A

;; ANSWER SECTION:
www.google.com.		220	IN	A	172.217.11.36

;; Query time: 11 msec
;; SERVER: 172.16.249.138#53(172.16.249.138)
;; WHEN: Sat Dec 14 16:33:46 EST 2019
;; MSG SIZE  rcvd: 59

And NSLookup:

nslookup www.google.com
Server:		127.0.0.53
Address:	127.0.0.53#53

** server can't find www.google.com: SERVFAIL

Devices DO get an IP address successfully in the 192.168.90.x range (which was setup for the VLAN). From the VLAN I can ping 8.8.8.8.

Please let me know any other information you need and thank you for your time and patience. Much appreciated.

johnpoz

Why would would nslookup be using 127.0.0.53???

pfSenseUser78

@johnpoz This is from a mint laptop joined to the VLAN90. Here's the output when joined to the LAN:

 nslookup www.google.com
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	www.google.com
Address: 172.217.6.196
Name:	www.google.com
Address: 2607:f8b0:4006:818::2004

Derelict

The way you have your rules set you need to set your clients to use 172.16.249.138 and 172.16.249.139 to resolve DNS. Based on the dig output that works fine. Figure out how to make that happen and stop trying to use what looks to me like unbound on the client (just a guess) and it will work fine.

127.0.0.53 is not 172.16.249.138 and 172.16.249.139

johnpoz

And where exactly whatever is listening on 127.0.0.53 sending the query when the laptop asks it for something?

So this laptop is running some sort of local caching service - yeah could be dnsmasq.. Client asks itself for www.something.tld, which that service forwards to where? Is the question, you need to make sure it forwards to your NS you have allowed on the other vlan.

pfSenseUser78

@Derelict

How do I get the VLAN set so that I don't have to manually assign the clients a specific DNS server? On some of these devices I can't manually set DNS.

Not using unbound at all, anywhere.

johnpoz

You can hand your clients whatever dns you want them to use via dhcp..

pfSenseUser78

@johnpoz

I have no idea. Everything is set to "automatic" for that wifi interface on the mint laptop I'm using (and yes, for all networks and not just the one on VLAN90). When I switch back to the LAN I don't have that problem.

johnpoz

In your dhcp scope you set on the vlan - set the NS you want the dhcp clients to use. Or setup a reservations for specific devices to use the dns you want those client to use.

Out of the box when you enable dhcp on an interface/vlan it hands out the interface/vlan as the dns..

pfSenseUser78

@johnpoz

GOT IT! IT WORKS!

Let me post my final VLAN rules to make sure there's nothing else I need to change (recommendations requested):
Firewall Final.png

marvosa

At this point, whether you choose to use blocks before an any/any or a streamlined allow rule that leverages the implicit deny... is now moot as it appears evident that the issue is on the client-side.

Either ruleset will work as soon as the clients are configured to use the correct DNS servers.

EDIT - Just saw the "IT WORKS!" post... glad it's working!

pfSenseUser78

@marvosa Now on to the next problem (which will be it's own post if I decide to continue) - HomeKit and WeMo don't talk to one another from the LAN to the VLAN. I found a few guides and attempted to open some ports but it's still not working.

At this point, I don't know if it's still worth it. I'd love to be able to have the IoT devices on their own network to avoid them compromising my LAN but it seems like a PITA to get them to talk across networks.