Site to site VPN with Multicast forwarding
I have searched but have not found too much info with multicast forwarding between subnets. I need it specifically for XBOX system link.
Cut-n-paste from http://www.semicomplete.com/projects/xboxproxy/
The xbox system link uses ethernet addresses (Layer 2) to indicate destination address and UDP (Layer 4) to send data. If you aren't familiar with the OSI model, then the layer information won't help you here. Basically, the 3 layers we care about for this system link proxy are ethernet (layer 2) and udp (layer 4). There's a special mention for the ip layer (layer 3) but that will be explained shortly.
System link packets come in two flavors: broadcast and unicast. In Halo 2, when you go to look for system link games, your xbox will send ethernet broadcast packets probing for available games. Broadcast packets are received by every network device on your layer 2 segment, this usually means your subnet or immediate network. Other xbox systems who are hosting games will respond directly to your xbox using your xbox's ethernet address (MAC address) as the destination. This process is called "discovery." After the discovery process completes and your xbox knows about other xboxes hosting games on the network, it begins direct communication to the known xboxes. When you try to join a game, your xbox sends packets directly to the other xbox you are connecting to. Direct communication continues until you quit the game.
A special note needs to be made, becuase you can't simple skip over layer 3 (the ip layer). We know now that addressed communication uses ethernet addresses, and we also know that the payloads are inside UDP packets, but what about the IP layer? The IP layer has addresses of its own, among other kinds of flags. Xboxes use the IP of 0.0.0.1 to communicate. This is nothing too special, but if you want to sniff only your xbox's traffic, then you can simply filter for that ip and you'll get it.
The proxy works by listening for broadcast packets from any xboxes on the immediate network. Any broadcast packets are forwarded to any known proxies over UDP. The proxy also keeps track of ethernet addresses by proxy. So if a packet from "my" xbox wants to talk to another xbox, the proxy will know which proxy that target xbox is on, and only forward the packet to that proxy.
This is a very simple system, and I don't have to know anything about the system link protocol beyond what the underlying layers are used for communication.
I later did some investigating into iTunes music shares. iTunes uses mdns (Apple calls it Rendezvous) for "discovery" of other iTunes music shares. The discovery is done over a protocol called multicast. Adding mdns support to the proxy/bridge program was quite trivial, and I have tested that it does in-fact work. You can use it to listen to iTunes music shares which are not on your immediate network.