What site-to-site solution supports Avahi reflection?

ohmantics

Adding a second site with pfSense Plus.

We have an ever-increasing amount of gear that requires multicast to administer and only comes with closed-source phone applications. mDNS is traditionally used by Apple software and HomeKit devices, but is now also used for Matter devices, Elgato lights, etc. This isn't a choice — this is a requirement now.

I get that the site-to-site solution will need to support L2 and there seem to be a few choices (IPSec/L2TP, OpenVPN, VXLANs) but it's not clear which if any will work for Avahi reflection, which requires interfaces.

dennypage

@ohmantics said in What site-to-site solution supports Avahi reflection?:

mDNS is traditionally used by Apple software and HomeKit devices, but is now also used for Matter devices, Elgato lights, etc.

To answer your specific question, both Avahi and mDNS-Bridge require multicast enabled interfaces. In theory, VXLAN could work, but I believe this was withdrawn from pfSense due to stability issues in the upstream implementation.

FWIW, HomeKit/Matter devices are designed around the concept of having a controller in the local network. They are not designed with a remote controller in mind -- remote operations are expected to be performed through the network local controller.

Instead of trying to flood multicast DNS across sites, consider placing a controller in each network. You can run this as multi-site HomeKit, or if a unified single site view is important you can build this with something like Home Assistant.

YMMV

ohmantics

@dennypage I'm not asking for different solutions or to be told that I'm wrong for wanting this.

I'm asking for a site-to-site solution that supports multicast using pfSense.

Unifi and OPNsense both have this working and I'd rather not be forced to switch after all these years as I'm quite happy with pfSense otherwise.

Have the upstream issues with VXLAN been addressed?
Can OpenVPN be configured for L2 and does that work with Avahi reflection?
Does IPSec/L2TP work with Avahi reflection?

dennypage

@ohmantics said in What site-to-site solution supports Avahi reflection?:

Can OpenVPN be configured for L2 and does that work with Avahi reflection? Does IPSec/L2TP work with Avahi reflection?

Not that I am aware of.

dennypage

@ohmantics said in What site-to-site solution supports Avahi reflection?:

@dennypage I'm not asking for different solutions or to be told that I'm wrong for wanting this.

I'm asking for a site-to-site solution that supports multicast using pfSense.

There’s a good reason you can’t find one.

stephenw10

You can configure OpenVPN in TAP mode and bridge it to the local interfaces to carry L2 traffic. That can work OK as long as latency isn't too high. But, I agree, it's usually the wrong solution.

ohmantics

@stephenw10 I’ll give that a shot.

I notice that TNSR still has VXLAN support. Will that be coming back to pfSense soon? If so, is that a better choice?

stephenw10

The main reason we removed VXLANs last time was that bridging to them was broken at the time and that is/was required to use them is most common scenrios. Which a switch that supports them for example.

You would still need to encryption too.

There are no immediate plans to add VXLANs back but I believe the bridging issue is now resolved.

OpenVPN TAP mode works now and is encrypted by default.

ohmantics

@stephenw10 I'm not seeming to get OpenVPN tap mode to work.

It only wants to connect up in tap mode if I also have Remote Access set and not Peer to Peer. The client then complains about each remote network:

OpenVPN needs a gateway parameter for a --route option and no default was specified by either --route-gateway or --ifconfig options

Thus none of the remote network devices can be pinged. Ticking "Push Bridge Interface IPv4 address to connecting clients as a route gateway" produces new errors:

WARNING: OpenVPN was configured to add an IPv4 route. However, no IPv4 has been configured for ovpnc1, therefore the route installation may fail or may not work as expected.
ERROR: FreeBSD route add command failed: external program exited with error status: 1

But this feels like it shouldn't be enabled anyway.

stephenw10

Why are you trying to route across something at layer 2?

It is in fact possible to do that by just adding gateway as a custom command but if you need to route to remote subnets just use TUN mode. Or add the gateway outside OpenNPN as though it were some locally attached router.

Edit: Reading back I could be misunderstanding what you're trying to do here...

ohmantics

@stephenw10 As stated above, I have endless amounts of devices that use multicast and I want to use Avahi reflection.

The connection between sites is 1Gb, so the “oh no flooding traffic from mDNS” argument is moot. No, I can’t just know everything’s IP address to manage it; that’s not how this works since these aren’t just printers.

DBMandrake

@ohmantics Avahi should be possible with OpenVPN in Tap / Layer 2 mode as noted. (whether it's a good idea is a separate conversation)

We have one point to point OpenVPN link between two PFSense routers (main "office" and remote "office") with multiple VLAN subnets at each end traversing the same link to essentially replicate our VLAN structure at the remote site but with different subnets.

Initially I had this set up as a tun / layer3 link (as you normally would) however I wanted to do DHCP relay across the tunnel to the domain controllers at the main "office", and the DHCP relay agent needs to bind to an interface that supports broadcast traffic, including on the upstream interface, so that can't work with a layer 3 OpenVPN link.

So I changed the OpenVPN link to use Tap/Layer 2. About the only thing you need to do is you have to manually use route-gateway to tell it what the remote gateway is - it won't do it automatically for you as it does in layer 3 mode, without this you won't be able to route unicast traffic.

In advanced configuration custom options on the server side I have:

route-gateway 172.23.0.2
reneg-sec 0
fragment 1490
mssfix

And at the client end I have:

route-gateway 172.23.0.1
reneg-sec 0
fragment 1490
mssfix

The only thing I changed on the existing OpenVPN tunnel configuration when going from layer 3 to layer 2 was adding route-gateway with the remote peer's OpenVPN tunnel IP, the other options I was already using. All existing unicast traffic routes as normal once the route-gateway setting is in place.

I have the DHCP relay agent installed on the remote office PFSense pointing at the DHCP servers at the main office and it works great. In fact it also points at our PXE deployment server and it's possible to do a full PXE boot and deployment across the OpenVPN link.

Another thing I have working with the layer 2 OpenVPN link is wake on lan to the remote site - for this to work I have UDP broadcast relay installed on both firewalls and configured correctly - works great.

I did try binding Avahi to the OpenVPN tunnel interfaces at both ends of the link as a test and I'm pretty sure that worked as well, showing all the Airprint printers at the remote end of the link so providing you have IP routing between the subnets bonjour services like Airprint should work across the OpenVPN link.

I disabled it again however because the only bonjour services we use are Airprint and Airplay and while it's technically cool that it can operate across a 5 mile VPN link it's not very useful to print to a printer or screen mirror to a screen 5 miles away and just clutters up the list of printers and airplay devices on users devices with ones that aren't useful to them.

dennypage

@ohmantics said in What site-to-site solution supports Avahi reflection?:

The connection between sites is 1Gb, so the “oh no flooding traffic from mDNS” argument is moot. No, I can’t just know everything’s IP address to manage it; that’s not how this works since these aren’t just printers.

I know you are bound and determined to try and make this work, but it isn't that simple. What you are trying to do with mDNS would work with printers, but will not work with smart home protocols such as Matter.

Bandwidth between sites is rarely an issue. Latency between the controller and the devices would certainly be a concern, that isn't the core issue either.

Flooding mDNS between IP network segments doesn't make a smart home protocol work, even on-site. The mDNS packets in question contain IPv6 link local addresses (fe80::/10) which are used to talk between the controller and its devices. By definition, link local addresses are only reachable from within the same layer 2 segment.

And if you use something like Avahi or mDNS-Bridge to forward mDNS with the link local addresses removed, some of the protocols like Matter will not work because there is no IPv6 link local address in the mDNS packet.

In short, a local controller in each IP network segment is a requirement for modern smart home protocols such as Matter.

Of course, one could create a single layer 2 network that spans the multiple sites...

stephenw10

Yeah a single layer 2 is what I thought you were going for here initially. But that isn't that case.

If Avahi works to bridge local layer 2 segments though I'd expect it to also work between local and remote segments?

dennypage

@stephenw10 said in What site-to-site solution supports Avahi reflection?:

If Avahi works to bridge local layer 2 segments though I'd expect it to also work between local and remote segments?

mDNS-Bridge is a pure bridge, with filtering. Avahi is a blending of bridge and local cache, again with a filtering capability. Both operate at layer 3.

stephenw10

Mmm. Fun*. Yeah I still agree this all seems like a workaround to force something that wasn't designed to work that way. There must be a better way...