Hello,
After months of work with @amassi, here is our feedback.
Multicast accross VLANs works with igmpproxy on pfSense <= 21.02.2-RELEASE but there are several cumulative constraints:
Only one upstream interface so only one VLAN can send multicast at once.
In theory, pimd (available in additionnal package) permits several upstream interfaces but it's totally buggy (when we start it, it tries to bind() on each network interface so it exceeds MAXVIFS kernel value - 32 - so it crashes. Obviously, it ignores its configuration file in which we have disabled unwanted network interfaces and it still tries to bind() on all interfaces).
Only 32 VLANs with multicast enabled at the same time (upstream + downstreams). Cause: MAXVIFS = 32 in FreeBSD kernel.
When we add CARP on each VLAN, the limit becomes 16 multicast-VLANs activable in igmpproxy. Cause: igmpproxy sees each VIP as a network interface so it tries to bind() on it and reaches MAXVIFS.
The more VIP we add on multicast-enabled interfaces, the less number of multicast-available interfaces we have. Same cause.
A multicast-enabled interface can't have more than six VIPs on it. Otherwise igmpproxy refuses to start.
A multicast-enabled VLAN must be in the XX first VLANs listed in Interface > Assignments > VLANs (all our VLANs are configured on lagg0). Otherwise igmpproxy don't bind() on it (no log message "adding VIF, Ix XX Fl 0x0 IP 0xXXXXXXXX lagg0.XXX).
On fresh install XX = 22. With CARP (for routing purpose) on all of our VLANs, XX = 21 (obviously, only VIPs on multicast-enabled VLANs are counted). With CARP for routing and destination NAT, XX = 20. If we add additionnal VIPs on these multicast-enabled VLANs, XX = 19.
We have moved our VLANs with a lot of VIPs at the end of the list => they are not counted.
We have added "parking" VLANs (unused VLAN IDs) in 17 th, 18 th, 19 th position in Interface > Assignments > VLANs. If we need to add VIPs on multicast-enabled VLANs, we will delete them. If we need to enable multicast on new VLAN, we will replace one ununsed VLAN ID by the new one and so VLAN will be in the 20 first multicast-activable VLANs.
In addition to these limits, we had an unknown problem with our FOG setup. We have installed a new storage node (in FOG terminology) and attached it to our existing FOG server => multicast works. New storage node has same OS and same FOG version (1.5.7) than the old one. For multicast, FOG uses the udpcast tool. sha256sum of updcast binaries are equal. So no idea of the root cause, but we now have a working inter-VLANs multicast FOG server with pfSense.
Finally, our XG-1541 reboot when we plug DAC cable in Chelsio's port and igmpproxy is enabled. At reboot, web gui prints core dump. Disable igmpproxy before plug DAC = no crash.
In summary: if you want to use inter-VLANs multicast with pfSense, you need to use igmpproxy + take previously-listed limits into account + maybe reinstalle your FOG storage node.
Bye