LAN plus VLANs: device gets IP from the wrong DHCP-server

[sgw]

I am currently setting up and (already) debugging a network with these components:

A Protectli-Appliance with pfSense-CE-2.8.1

LAN .. igc1 ... plugged into Switch1, PVID 1
VLAN150 ... igc2.150 ... plugged into Switch2p1 ... no PVID, transporting VLANs

Switch1 and Switch2 are connected via a cable: PVID1, transporting VLANs (we're talking unifi-switches here)

Switch2p22 is native in VLAN150 and connects to a third switch (non-unifi) which doesn't talk VLAN-tags: it's just the physical switch for that second subnet.

So the idea is: the pfsense has 2 interfaces, one in LAN1, the other in LAN2 (=VLAN150).

I allow everything on the 2 interfaces. Maybe that is wrong already.

LAN1 should be served by a Windows-DC as DHCP-server.

I enabled DHCP guarding on the Unifi-Switches to only allow that one IP as DHCP-server.

should

When I plug a PC into a port with PVID1 on switch1 or switch2 I expect it to connect and talk to LAN1 only and receive a DHCP-Lease from the Windows-DC in LAN1.

is

It gets an IP from a DHCP server in LAN2! I tried that on several ports on both switches today.

questions

• Do I need to block anything on the pfsense interfaces to fix that?
  I need to have DHCP in VLAN150 going through there because I have VMs etc that should be served this way.

I assume there might be some unknown extra cable somewhere connecting their switch with the new ones. I can't look that up myself, I am located far away, another admin will do that asap.

• any ideas how to pinpoint this without for example unplugging the one wanted connection between the 2 worlds?

I have a proxmox server there, I can plug VMs into VLAN1(=no tags) or 150(=tag 150) and DHCP works accordingly. The physical ports on Switch1 should be way easier to set up (default network is VLAN1/untagged) ... but somehow there seem to be untagged packages from LAN2 there where they shouldn't be (and DHCP Guarding seems not to work).

I hope the picture is more or less understandable. Any hints welcome ...
Thanks in advance, Stefan

[sgw]

Should I block TCP/UDP ports 67-68 on the Interface for LAN2/VLAN150?
Would that be correct?

[patient0]

@sgw said in LAN plus VLANs: device gets IP from the wrong DHCP-server:

VLAN150 ... igc2.150 ... plugged into Switch2p1 ... no PVID, transporting VLANs

Can you show the config overview from the unifi controller for Switch1 and Switch2?

And how is the Windows-DC server connected, to what switch and what port?

Do you get the IP from the correct DHCP server if you connect a device to a port where PVID is set to 150 (or the 3rd switch)?

If you configured it correctly then a PC connected to a PVID1 port would not receive a DHCP from LAN2, no firewall deny rule necessary.

Btw: have you disable DHCP on LAN1 on pfSense?

[sgw]

@patient0 said in LAN plus VLANs: device gets IP from the wrong DHCP-server:

Can you show the config overview from the unifi controller for Switch1 and Switch2?

Hm, difficult, that's a lot. I will try to list the relevant ports again (second posting).

And how is the Windows-DC server connected, to what switch and what port?

It is connected to that third switch, non-unifi. Just without VLAN-tags.

Do you get the IP from the correct DHCP server if you connect a device to a port where PVID is set to 150 (or the 3rd switch)?

In a test VM on my promox: yes. I can connect it to "vmbr1" or "vmbr1.150" and get an IP from LAN1 or VLAN150 then OK.

We plan to test things with a physical client next week (I have to wait for their local IT guy to visit them again).

If you configured it correctly then a PC connected to a PVID1 port would not receive a DHCP from LAN2, no firewall deny rule necessary.

I agree :-)

Btw: have you disable DHCP on LAN1 on pfSense?

Yes.

[sgw]

I list the switches and ports now which I set up and play part in my setup.
I can only try to get my part right in the first place.

pfSense-CE 2.8.1

Port : Name
igc0 : WAN
igc1 : LAN
igc.3 : Guest (wifi)
igc.100 : Management
igc2.150 : LAN2

for switches:

(Port : Description : Native VLAN : Tagged VLANs)

Switch 1 (USW Pro Max 24 PoE)

2 : pfSense LAN - igc1 : 1 : Allow All
25 : trunk to Switch 2 : 1 : Allow All

Switch 2 (USW Pro XG 24)

1 : pfSense LAN2 - igc2.150 : None : Allow All
21 : Connect to Switch 3 : 150 : Allow All
22/23 : LAGG to Switch 4, p47/48 : 1 : Allow All

Switch 3 (HP? no admin access for me) - LAN2

x : connect to switch2, p21
y : Windows DC/DHCP

Switch 4 (USW Pro XG 48)

13 : PVE Host eno1 : None : Allow All
14: HPE ILO (PVE) : 100 : Allow All
47/48 : LAGG to Switch 2, p22/23 : 1 : Allow All
49 : PVE Host vmbr1 : 1 : Allow All

DHCP in LAN is done by a VM running in native LAN there (also placed on the PVE host behind Switch4p49

Switch4p13 is sketchy: this was initially used for installing first VMs before I had the faster connection on port49.

PVE

eno1 .. Switch2port13 .. is member of vmbr0, which is connected to NO VMs anymore

vmbr0.100 is used for the Management IP of the PVE host .. I'd like to have this extra cable for management ..

eno5np0 .. Switch4p49 .. member of vmbr1, with all the vms

Note: switch2p2 was plugged in there because there were no ports free on switch1 in that moment. It would be a bit more logical to have the pfSense connect with all its interfaces to switch1, at least for us humans.

So this is the setup I did. Do you see any mistake already? thanks!

[sgw]

Should I forbid all Tagged VLANs on Switch2p21?

Their native LAN should have no tag 1 ... but ...

[patient0]

@sgw said in LAN plus VLANs: device gets IP from the wrong DHCP-server:

It is connected to that third switch, non-unifi. Just without VLAN-tags.

I'm confused, you did write that LAN1 is served by the Windows-DC. But the 3rd switch doesn't do VLANs and the PVID is set to 150, LAN2. And the Windows-DC server is connected the 3rd switch, which means it runs on VLAN150.

And there is not much point in allowing all VLAN on Switch2p21 if it can't do VLANs.

[sgw]

@patient0 thanks

I was unclear:

There is a Windows DC/DHCP for one company/domain in LAN/LAN1 (running on a VM on that Proxmox host), and a second Windows DC/DHCP for the second company/domain in LAN2 attached to Switch3.

These two companies get merged .. and should be able to connect to services in each others subnets. Everything routed through and firewalled by the pfSense.

OK?

[sgw]

@patient0 said in LAN plus VLANs: device gets IP from the wrong DHCP-server:

And there is not much point in allowing all VLAN on Switch2p21 if it can't do VLANs.

correct. turned that off now. For sure this doesn't explain the DHCP issue.

[sgw]

corrected a mistake:

eno1 .. Switch2port13 .. is member of vmbr0 on the PVE

it only transports tagged VLANs .. so it can't transport the untagged DHCP traffic from LAN2 also

My try would be: connect PC as DHCP-client to switch1p20 (that's where the problems were reported first), let it get an address. Yesterday it pulled an IP in LAN2.

Then remove the connection between Switch2 and Switch3, retry.

We can only do that when we have announced some maintenance window (next week).

[sgw]

Just as a thought:

I don't have these 2 boxes ticked on the pfSense interfaces:

https://docs.netgate.com/pfsense/en/latest/interfaces/configure.html#reserved-networks

And the firewall rules have this "allow * from/to everywhere" for LAN and LAN2.

PCs should reach servers etc / but we don't want these DHCP-packages.

Should I block port 67/68 on LAN2 interface?

[patient0]

@sgw said in LAN plus VLANs: device gets IP from the wrong DHCP-server:

I don't have these 2 boxes ticked on the pfSense interfaces:
https://docs.netgate.com/pfsense/en/latest/interfaces/configure.html#reserved-networks

No, as mentioned on the page it is usually used on WAN interfaces to block RFC1918 traffic.

Should I block port 67/68 on LAN2 interface?

I don't see why that would be necessary. If you read DHCP discovery you see that the client sends a DHCPDISCOVER to 255.255.255.255 which is limited to the broadcast domain (LAN1).
I think it's best if you use Wireshark to sniff the traffic that you see on a port with PVID1 to check what DHCP traffic you see.

Btw, I assume you have set that port on the switch to PVID1 and no VLANs allowed?

[sgw]

@patient0 said in LAN plus VLANs: device gets IP from the wrong DHCP-server:

Btw, I assume you have set that port on the switch to PVID1 and no VLANs allowed?

pls detail which port is meant here, thx

[patient0]

@sgw I was referring to the port you put the client on which should get a LAN1 IP.

[sgw]

You talk about the access port for the PC? Yes, its PVID is 1 = LAN1 native. VLANs are allowed there per default, I assumed this wouldn't hurt as the PC should not talk tagged. But we will test that.

[sgw]

@patient0 said in LAN plus VLANs: device gets IP from the wrong DHCP-server:

I don't see why that would be necessary. If you read DHCP discovery you see that the client sends a DHCPDISCOVER to 255.255.255.255 which is limited to the broadcast domain (LAN1).
I think it's best if you use Wireshark to sniff the traffic that you see on a port with PVID1 to check what DHCP traffic you see.

Good to hear that my fw-rules aren't wrong in that way ... I run pfSenses with dozens of VLANs and interfaces in other sites and never had such an issue so far.

[patient0]

@sgw said in LAN plus VLANs: device gets IP from the wrong DHCP-server:

I assumed this wouldn't hurt as the PC should not talk tagged. But we will test that.

It shouldn't, you're right. But to narrow it down it would help.

I do use a Unifi Switch and have clients on ports configured like yours (native VLAN 1 and allow all VLAN) and it works as expected.

[sgw]

@patient0 I don't know about that PC. But it's very unlikely that it's configured to understand VLAN 150. This VLAN comes from me and exists only on my systems (pfSense, switches, PVE).

But sure, I will take away the VLANs from that port at first.

thanks so far

I wrote an issue on the german Proxmox-forum as well, to check my bridging setup on the PVE.

I link it here, maybe somebody is interested as well:
link