Set up issue with Proxmox, pfSense, and a dlink router

fr0zt · Jan 29, 2024, 5:46 PM

Hello,

I'm a relative newbie at this and have almost got my setup working, but can't figure out the last bit.

Goal: Run pfsense inside proxmox and use an old dlink DIR-819 router as an access point. I want to be able to send all traffic from the access point through a VPN

Current status: pfSense is set up inside proxmox, I have the access point connected. Wireless devices can reach other devices on the LAN and reach the internet, but only if I manually assign a static ip address on the device itself . DHCP from pfSense does not seem to be working for the wireless devices.

Details

Hardware is a 4 nic n5105 mini pc
I have 3 interfaces:
1. WAN (assigned to physical ethernet port 1, connected to my ISP modem)
2. LAN (not assigned to an ethernet port, just using for virtual machines)
3. OPT1 (assigned to physical ethernet port 4, connected to the LAN side of the dlink router)
LAN and OPT1 are bridged in pfSense
my virtual machine connected to LAN successfully gets a DHCP address (192.168.1.100)
my dlink router does not get a DHCP address. But if I manually enter a static ip (192.168.1.15, subnet 255.255.255.0, router 192.168.1.1) then it's connected to the network (gets internet and can ping the VM)
my laptop connected to the dlink router behaves the same. It doesn't get a DHCP address, but if I manually enter static ip, it connects (192.168.1.16)
I can see the 192.168.1.16 laptop in Diagnostics / ARP Table
I have disabled DHCP on the dlink router
I've opened firewalls wherever I've seen the options (ie firewalls open on the interfaces in proxmox, firewall rules open in the LAN and OPT1 pfsense interfaces, etc)

I'm happy to set a static ip on the dlink router, but don't want to have to set static ips and the wireless devices (laptop etc)

Also I'm not sure if I overcomplicated things by adding the third interface (maybe it's a bit simpler to just add the the ethernet port to the LAN interface and then it should work for VMs as well as the access point / connected devices?)

Any help would be greatly appreciated. Thanks!

fr0zt · Jan 29, 2024, 7:29 PM

Edit

I realized the the DHCP server was assigned to the LAN interface (and I'm not sure if that applies to devices connecting to OPT1 if they're bridged. So I followed this guide and:
1. Enabled the bridge interface as BR0 (I hadn't previously enabled it)
2. Disabled DHCP Service for LAN
3. Created an interface group, "Bridge", for LAN, OPT1, and BR0
4. Removed the static IP from LAN and added static ip to BR0 (192.168.1.1 / 24)
5. Added firewall rule for all traffic on Bridge
6. Enabled DHCP server on BR0 (192.168.1.10-192.168.1.254)
After doing this, I seem to be in still the same position as before (VM on LAN still has ip 192.168.1.100 and my wifi laptop still only works when manually assigning an ip, even after ipconfig release/renew)

stephenw10 · Jan 29, 2024, 9:05 PM

I would expect that to work as long as the DHCP server is on the assigned bridge interface itself. And the pass rules are explicit in passing everything.

fr0zt · Jan 29, 2024, 9:18 PM

Thanks for the reply @stephenw10

Confirmed that the DHCP server is on BR0, the bridge interface:

login-to-view

In DHCP logs I see:

Jan 29 13:03:11	dhcp6c	16939	Sending Solicit
Jan 29 13:03:11	dhcp6c	16939	advertise contains no address/prefix
Jan 29 13:05:14	dhcp6c	16939	Sending Solicit
Jan 29 13:05:14	dhcp6c	16939	advertise contains no address/prefix
Jan 29 13:07:09	dhcp6c	16939	Sending Solicit
Jan 29 13:07:09	dhcp6c	16939	advertise contains no address/prefix
Jan 29 13:09:02	dhcp6c	16939	Sending Solicit
Jan 29 13:09:02	dhcp6c	16939	advertise contains no address/prefix

In Firewall logs I see:

Jan 29 13:07:46	OPT1	Block IPv4 link-local (1000000101)	  169.254.204.233:49673	  239.255.255.250:3702	UDP
Jan 29 13:07:46	BR0	Block IPv4 link-local (1000000101)	  169.254.204.233:49673	  239.255.255.250:3702	UDP
Jan 29 13:07:46	OPT1	Block IPv4 link-local (1000000101)	  169.254.204.233:49673	  239.255.255.250:3702	UDP
Jan 29 13:07:46	WAN	Default deny rule IPv6 (1000000105)	  [fe80::4607:bff:fe6e:c4bd]:5353	  [ff02::fb]:5353	UDP
Jan 29 13:07:46	WAN	Default deny rule IPv6 (1000000105)	  [fe80::4607:bff:fe6e:c4bd]:5353	  [ff02::fb]:5353	UDP
Jan 29 13:07:46	WAN	Default deny rule IPv6 (1000000105)	  [fe80::22df:b9ff:fed3:fd63]:5353	  [ff02::fb]:5353	UDP
Jan 29 13:07:46	WAN	Default deny rule IPv6 (1000000105)	  [fe80::22df:b9ff:fe16:184c]:5353	  [ff02::fb]:5353	UDP
Jan 29 13:07:49	WAN	Default deny rule IPv6 (1000000105)	  [fe80::a018:1f77:e251:2728]	  [ff02::16]	Options
Jan 29 13:07:49	WAN	Default deny rule IPv6 (1000000105)	  [fe80::a018:1f77:e251:2728]	  [ff02::16]	Options
Jan 29 13:07:51	WAN	Default deny rule IPv6 (1000000105)	  [fe80::a018:1f77:e251:2728]	  [ff02::16]	Options
Jan 29 13:07:52	WAN	Default deny rule IPv6 (1000000105)	  [fe80::a018:1f77:e251:2728]	  [ff02::16]	Options
Jan 29 13:07:54	WAN	Default deny rule IPv6 (1000000105)	  [fe80::a018:1f77:e251:2728]	  [ff02::16]	Options
Jan 29 13:07:55	WAN	Default deny rule IPv6 (1000000105)	  [fe80::a018:1f77:e251:2728]	  [ff02::16]	Options
Jan 29 13:07:57	WAN	Default deny rule IPv6 (1000000105)	  [fe80::a018:1f77:e251:2728]	  [ff02::16]	Options

the ip address denied 169.254.204.233 is indeed the ip address that the laptop has, so I guess that suggests a firewall rule issue? Though if it were a dhcp issue, I'd expect the denials to happen on the dhcp port? (which google suggests are port 67/68?)

Firewall rules:
login-to-view

login-to-view

stephenw10 · Jan 29, 2024, 9:28 PM

That was straight after connecting a client to OPT1?

The 169.254 (APIPA) address is what the laptop gives itself when it can't find a DHCP server. It's a non-routable address so pfSense blocks it by default.

So either the client isn't trying to pull a DHCP lease or the server never sees it.

fr0zt · Jan 29, 2024, 9:35 PM

@stephenw10 said in Set up issue with Proxmox, pfSense, and a dlink router:

That was straight after connecting a client to OPT1?

It was straight after I ran ipconfig /release then ipconfig /renew on the laptop. (With the dlink router plugged into OPT1 using one of the dlink's LAN ports)

I guess 2 sources that I think I can rule out are:

Issue with proxmox forwarding the ethernet port / NIC to the pfsense VM?
Issue in the dlink router settings?

I think I can rule both those out, given that the laptop is able to connect to a VM on the LAN, when setting a static ip on the laptop

stephenw10 · Jan 29, 2024, 9:38 PM

Try connecting the laptop dircetly. The D-Link device could be blocking broadcast traffic dhcp uses.

fr0zt · Jan 29, 2024, 9:43 PM

oh *** It works now!! Thanks so much (your wording "straight after connecting a client to OPT1?"). Maybe ipconfig \renew didn't do what I was expecting. I disconnected from the wifi on the laptop, then reconnected to the dlink router. Now I'm succesfully getting internet access and can connect to the VM on the LAN (I started writing this before your previous message, so I haven't tried through wire connection).

Now to clean up - do I need all of those firewall allow rules on all of those interfaces? If I set "allow". Ie - there's the Bridge Interface Group, there's LAN, and OPT1, BR0. I'm fine to treat them all the same. Should I delete rules from all the interfaces except for the Bridge Interface Group? Will the rules then cascade down to the members?

stephenw10 · Jan 29, 2024, 10:04 PM

No you don't need all those rules. By default a bridge filters on the member interfaces not on the bridge itself. You've allowed for that with the group rule that gets applied to all members.

Alternatively you can move the bridge filtering onto the bridge itself and then you only need rules on BR0.

https://docs.netgate.com/pfsense/en/latest/bridges/firewall.html#bridging-and-firewalling

However if you want to filter traffic between LAN and OPT1 you need the rules on the members.

If you look at the rules you can see which ones have opened states and passed traffic. Anything that that doesn't likely doesn't need to be there. In your setup the rule(s) on the BridgeGroup override everything else so you'll likely only see states there.