pfSense + Layer 3 + Access Point

Zipping8761

As a follow up: I forgot to make vlan 42 and vlan 44 into router interfaces and assign them an ip address. I also didn't have dhcp servers setup for them. Now I can connect to all 3 of the ssids and receive the proper ip address assigned to the vlan the ssid is on.

I believe for best security I should re-assign the mobile ssid to vlan 46 instead of vlan 40. Since the access point is managed by vlan 40, it should be given a /30 and not assign any other devices to it. This is how I have the connection to pfSense setup and I believe it is the proper way to go, never use the management vlan that 2 switches are connected to each other on as being used by other devices. Correct? I just need to figure out how to deny all dhcp leases on vlan 40 unless a static mapping is set. I only want the access points I assign to have dhcp leases. Easy to do in pfSense but not straightforward on the layer 3 switch.

Now the devices still have no internet access. I am able to ping from the devices 1.1.1.1 but it will not resolve the hostname so it looks like DNS is not working properly. DNS is being passed on to pfSense and the firewall is not blocking it. I can run a traceroute on the switch and it is able to process DNS. The DNS is not being passed along to the access point. I can see the DNS requests coming into pfSense through the states table.

Zipping8761

On the switch:

On the access point:

johnpoz

@zipping8761 google.com is invalid means what? Can you resolve google.com or not?

If you can not resolve it - then no your not going to be able to ping it

Zipping8761

I don't know. Maybe the access point is incapable of accepting the second option as a host and requires an ipv4/ipv6 address only. Or maybe it means it can't resolve it because it cannot connect to the DNS server. I wish I knew. I'm assuming it can't access the DNS server but as you can see above that it is able to ping 192.168.99.1 which is where the DNS resolver on pfSense lives. The switch has no problem resolving DNS.

While running traceroute from a proper linux laptop connected to the switch via vlan 10:

traceroute google.com
google.com: Temporary failure in name resolution
Cannot handle "host" cmdline arg `google.com` on position 1 (argc 1)

Traceroutes work with any ipv4 addresses but not host names which require DNS lookup. I don't know how pfSense is blocking DNS to not go beyond the switch to the hosts connected to the switch. I can change the DNS servers to 9.9.9.9 in /etc/resolv.conf and fix everything but that's not a solution to this problem. Something about switch routing or pfSense I'm not understanding here.

johnpoz

@zipping8761 if you have a downstream network, and you want this downstream network to use unbound, you need to added it to the acls

out of the box pfsense only auto adds its locally attached networks, and or tunnel networks of vpn server setups.

If you have say downstream network 10.0.0/24 via your 172.16.0.0/30 transit interface, you have to add 10.0.0/24 to your unbound ACL

Zipping8761

@johnpoz

Thank you for that. That was the magic setting that I didn't know about.

Zipping8761

I have successfully transferred my entire network over to my layer 3 switch. It was quite the chore to give all of my devices a new ip address. I had been using static ip addresses through pfsense's dhcp server. Most devices were unimportant but a few servers serve all the clients with their static ip addresses and were last to be transferred over. I am still using the dhcp server on the layer 3 switch and haven't decided to migrate from it yet. Maybe later.

I have managed to separate services on my server through a second network interface card and docker. My mistake was trying to use a /30 subnet where docker needed to have an ip address assigned from the same subnet but with a /30 that wasn't possible. So a workaround was to use a /24 subnet and block dhcp from assigning any more addresses. You can make up any ip address you want for docker as long as it has the same subnet setting as the NIC. I can share specifics if anyone else wants to try this.

I'm moving on to learning access lists. I couldn't understand the in and out directions. This video finally explained it in a way that I could understand. It's been a fun if not difficult learning experience so far. The only reason I think it's been so difficult is because I wanted to slowly transfer my network over and keep everything working during the whole process. If I had started from scratch I think all of my challenges would have been lessened.

Is it better or more efficient to have pfsense paired with a layer 2 switch instead of an layer 3 switch? Possibly. Your losing some functionality of pfsense. Time will tell how I like it. I know I am learning a lot of new things and that's good for my brain. Hopefully no one else is afraid to take on this challenge, I know I am glad I did.

johnpoz

@zipping8761 it can be a great learning experience.. But yeah its going to be more difficult than just clicking whatever rules you want in the pfsense gui ;)

And what are you using for dhcp - pfsense can not provide dhcp to clients that its not directly attached too.. So some downstream networks that are routed at your L3 switch would not be able to use the dhcpd on pfsense for their ips, etc.

Glad your having fun and learning..

Zipping8761

@johnpoz

I have an interface on pfsense connected to the layer 3 switch with a /30 subnet. I have the dhcp server enabled for that interface. There is no lease present in dhcp for the switch but there is an entry in arp. Since I would have had to create an interface for each vlan in order to use pfsense's dhcp server, I am currently using the downstream layer 3 switch for all dhcp.

I ran into an issue here with a smart tv that would not get an ip address. After a few days I had to let the wifi access point handle the dhcp for the smart tv. Since it's the only device on that vlan, it'll work for now. I couldn't figure out why it was failing. For the first few days I thought it was some wrong wifi band causing auth failure but once I was able to figure out how to view the access point's debug log it kept showing successful authentication. I am planning on running ISC DHCP at some point but as you stated it's still going to be downstream from pfsense.

I'm still using pfSense for DNS because I'm running pfBlockerNG and want that to continue to work.

I guess thinking about it now I am going to miss pfSense with Suricata acting as an intrusion protection system between the vlans (internal network) themselves. It almost makes me want to go back to a layer 2 switch mode. :) But for the most part all I ever really had between clients is false positives so will I really miss much? When I have my ACLs in place, none of the noisy devices will ever be able to access the desktops or servers. Plus everything is linux based with their own intrusion systems already so I may not miss the ips from pfSense in regards to that. It's still working for internet based blocking.

Now that everyone has a new static ip address assigned, I could easily switch over to layer 2 and setup vlan interfaces in pfsense in a few minutes should I decide I don't like layer 3.

Zipping8761

I'll probably go back to layer 2 eventually because these ACLs are harder to understand than originally thought. A simple ping test block for example. To block all pings from source 192.168.48.x devices to destination devices on 192.168.10.x would be:

ip access-list extended VLAN48-ACL
deny icmp any 192.168.10.0/24
permit ip any any
exit

interface ve 48
ip access-group VLAN48-ACL in
exit

As expected pings from 192.168.48.x devices to 192.168.10.x devices are blocked but pings from 192.168.10.x devices to 192.168.48.x devices are also blocked.

A device on vlan 48 sends packets in on vlan 48 and they go out on vlan 10 before hitting the destination device on vlan 10 without ever going out vlan 48 or in on vlan 10.

In order to achieve the desired results you also have to think about a reply from a ping coming back. The echo would be blocked by the deny icmp any 192.168.10.0/24 rule unless you add echo to the end of the rule.

deny icmp any 192.168.10.0/24 echo

How to deny only incoming traffic to a certain VLAN

johnpoz

@zipping8761 haha - I warned you, but it a good learning experience ;)