DHCP relay failing when VLANs on separate physical interfaces

[brainyron]

I'm sure I'm just missing something from having looked at this for too long but…

I have a pfSense router set up with multiple VLANs.  The Client and IoT VLANs get their IP from a DHCP server residing on the Server VLAN.  I'm trying to set it up to where the Server VLAN is on a separate physical interface from the other VLANs, but when I do this, DHCP relay does not work.  I can see in TCPdump that the dhcp packets are coming into the server VLAN interface, and going back out on the client VLAN interface, but the client never receives them.  As soon as I switch the configuration to where the server VLAN is bound to the same physical interface as the client VLAN, DHCP relay starts working as expected.  I've been working under the assumption that the firewall is dropping the traffic, but so far haven't found a working ruleset.

Right now, I have two floating rules in place (I've tried binding the same rules directly to the interfaces, without success):
1)  IPv4 UDP any/67-68 any/67-68 pass
2)  IPv4 UDP (local subnets alias)/any 255.255.255.255/any pass

Hopefully another pair of eyes on the problem can help me get to the bottom of this so that I can properly isolate traffic.

Thanks in advance!

[brainyron]

Update:  after some additional troubleshooting, it looks like the packets are making it to the switch, and the switch is electing to drop them:

From vlan interface coming from router:

17:34:54.387009 xx:xx:xx:xx:ad:97 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 346: vlan 100, p 0, ethertype IPv4, 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from d4:3d:7e:ea:ad:97, length 300
17:34:54.391498 xx:xx:xx:xx:b3:43 > xx:xx:xx:xx:ad:97, ethertype 802.1Q (0x8100), length 371: vlan 100, p 0, ethertype IPv4, 192.168.194.1.67 > 192.168.194.26.68: BOOTP/DHCP, Reply, length 325

From mirror of uplink to desk:
17:34:54.384985 xx:xx:xx:xx:ad:97 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from xx:xx:xx:xx:ad:97, length 300
17:34:55.496571 xx:xx:xx:xx:ae:fb > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 590: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from xx:xx:xx:xx:ae:fb, length 548
17:34:57.499810 xx:xx:xx:xx:ae:fb > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 590: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from xx:xx:xx:xx:ae:fb, length 548

xx:xx:xx:xx:ad:97 - The client machine I'm using to test
xx:xx:xx:xx:b3:43 - My pfSense router
xx:xx:xx:xx:ae:fb - Another machine that just happened to be requesting DHCP at the same time, shown to give timing.

My switch is a Ubiquiti UniFi US-24.  I've opened a support case with them to check on their end… Has anyone seen this particular behavior before?

[brainyron]

For future reference in case anyone else should hit this, it ended up being a bug in the ubiquiti switch:

https://community.ubnt.com/t5/UniFi-Routing-Switching/Disable-DHCP-Snooping-on-USW/m-p/1510229/highlight/true