Handling Multiple Interfaces on Client System?

mbelanger

I'm actually not sure whether this is the right forum for this, but I thought I'd give it a try.

I just set up the DNS Resolver on my pfSense boxen and I'm loving it. DNS for my LAN... Hooray! Whoever, I discovered an unintended consequence for laptops. Between WiFi, a Thunderbolt Display and an Ethernet dongle, there are three interfaces for said laptop. Obviously, I can use the same name in the hostname field, despite the fact that it is technically correct. What is the best practice for handling this. I certainly hope there's a solution more elegant than lappy-en0, lappy-en1 and lappy-en2...

Thoughts appreciated,
MB

JKnott

@mbelanger

What operating system are you running? With Linux, it automagically sorts things out. With Windows, I had to create a host name for each interface and use that, according to the interface I was using.

mbelanger

macOS, but the OS is irrelevant, because with the pfSense DHCP Server I'm handing out a IP per MAC address and then binding the hostname to said IP via the option within DNS Resolver. This naturally leads to the following result:

puter:~ $ dig mac-lappy
10.0.1.40
10.0.1.41
10.0.1.42
puter:~ $

Having one hostname associated with three different IPs leads to all kinds of hell, but I can't think of a clean way to associate the hostname/same IP with three different MAC addresses (which definitely is hell when two+ interfaces are up at the same time).

I think I have to endure having three separate hostnames (mac-lappy, mac-lappy-wifi, mac-lappy-dock) and just move on.

JKnott

@mbelanger said in Handling Multiple Interfaces on Client System?:

I think I have to endure having three separate hostnames (mac-lappy, mac-lappy-wifi, mac-lappy-dock) and just move on.

Since macOS is based on BSD, I expect it will work the same as Linux, in that it will route appropriately. In my case, I have a notebook that can be connected via WiFi or Ethernet. When I'm at home, and connected via Ethernet, WiFi is also connected, so that I have 2 connections to my network. However, due to something called metric, the Ethernet connection will be used rather than WiFi. However, WiFi still has an IP address, which the host name points to. No matter which connection is used, that address will always work, as Linux knows enough to route to it. I expect BSD will do similar. Windows will not. Give it a try and see what happens.

mbelanger

I must not be making myself clear.

It routes fine. I can get out over any of the three interfaces. The problem I'm trying to solve is heading the other direction, like via ssh. Ideally, I could just type

% ssh mac-lappy

from another machine and through some kind of DNS magic, I could get in regardless of which interface was active. If this is even possible, especially with ssh in the mix, I'm not sure it is, I doubt pfSense's Untangle could handle it.

Ultimately, it's not so complicated that I can't remember what interface said laptop is on, so I'm not going to pursue it.

JKnott

@mbelanger said in Handling Multiple Interfaces on Client System?:

Ultimately, it's not so complicated that I can't remember what interface said laptop is on, so I'm not going to pursue it.

As I said, with Linux and presumably macOS, it just works. It's only with Windows that you have to worry about how you're connected. Try creating a DNS entry that points to the WiFi address. Then connect the computer with Ethernet. Assuming the WiFi connection is up, connecting to the WiFi address will go through the Ethernet connection to that computer. It just works. I do that all the time here with my notebook computer. I have the DNS configured to point to the WiFi and, regardless of whether the notebook is connected via Ethernet or WiFi, it works fine.

Try it and see what happens.

johnpoz

@mbelanger said in Handling Multiple Interfaces on Client System?:

ssh mac-lappy

That is not a fqdn, so it would not resolve anyway.. You could broadcast for a host address, or your client could add domain to that in a search suffix..

mbelanger

This is getting out of hand. I'm good. Nothing to solve here...

That said...

@johnpoz If you click the "Static DHCP" checklist, the pfSense domainname is appended to the hostnames listed within the static DHCP hostname fields, so it is a FQDN. I just didn't type it out, because I assumed it was self-evident. And before you tell me it won't route over the Internet, it's not intended to, as I'm using foo.home for my LAN DNS.

@JKnott I must be a terrible communicator, because I don't even understand what you're suggesting.

Here's what happens when any laptop comes online, macOS, Linux or otherwise:

if WiFi interface is enabled, the pfSense sees MAC addr 00:00:00:00:00:01 and assigns IP 10.0.1.10.
if Dongle Ethernet is enabled, the pfSense sees MAC addr 00:00:00:00:00:02 and assigns IP 10.0.1.11.
if Display Ethernet is enabled, the pfSense sees MAC addr 00:00:00:00:00:03 and assigns IP 10.0.1.12.

Each interface has it's own MAC address and thus gets its own IP. If you assign the same hostname to each of those interfaces, you will always wind up at .11, regardless of whether the interface is even up.

How does Linux magically solve this problem when the DNS is assigned absent any knowledge of interface status?

(Mods: is there a way to close a thread?)

JKnott

@mbelanger said in Handling Multiple Interfaces on Client System?:

Each interface has it's own MAC address and thus gets its own IP. If you assign the same hostname to each of those interfaces, you will always wind up at .11, regardless of whether the interface is even up.

I never said assign the same host name to each interface. I said use the host name for the WiFi interface only. If both Ethernet and Wifi are connected, you can connect to the WiFi address, even though the traffic will pass through the Ethernet interface. This is how it's worked here with my notebook for years with Linux. As I mentioned, it will not work with Windows, but I expect the BSD under macOS will work just like Linux. Yes, each interface will have it's own IP address and can also have a host name, but only the WiFi name & address will be used, when trying to connect to it.

When you do what I describe, both interfaces will have an address in the same subnet. There is something called "metric", which Linux uses to determine which of the 2 interfaces to use. A lower metric is the preferred connection. Ethernet will have a lower metric than WiFi and will be used. This is basic routing.

johnpoz

For the life of me why would you have both wifi and ethernet on at the same time in the first place? Normally you would setup the box to disable wifi when ethernet connected..

That is how my laptop works.

I have to on purpose enable the wifi after connecting ethernet if want to do something odd. Also you should hand your display its own name vs a reservation. Why in the world world would say ssh be listening on your display ;)

From dns point of view you would need to use different name for different IP if you want to distinguish which one you connect to. If you have multiple IPs for the same record, then you would get them back round robin style or all of them depending. Normally you would have wifi and wired on different networks, using different domains - so for example on my network wired is local.lan, while one of the wifi networks is wlan.local.lan and there is dmz.local.lan and psk.local.lan, etc. So If want to talk to a devices wlan IP would use host.wlan.local.lan to resolve it.

JKnott

@johnpoz said in Handling Multiple Interfaces on Client System?:

For the life of me why would you have both wifi and ethernet on at the same time in the first place? Normally you would setup the box to disable wifi when ethernet connected..
That is how my laptop works.

WiFi connects automagically, when the computer is powered up. If I expect to be doing some serious file transfer, I will plug in the Ethernet cable. The WiFi is still up, resulting in both interfaces connected to the same network at the same time. As I mentioned, the metric will be used to determine which interface is used. There is no need to turn off the WiFi when Ethernet is connected. With Linux, the WiFi address is still reachable, when Ethernet is connected. So, I can ping, ssh or whatever to the WiFi host name and it just works fine.

Here's what the "ip route show" command shows:
default via 172.16.0.1 dev eth0 proto dhcp metric 100
default via 172.16.0.1 dev wlan0 proto dhcp metric 600
172.16.0.0/24 dev eth0 proto kernel scope link src 172.16.0.42 metric 100
172.16.0.0/24 dev wlan0 proto kernel scope link src 172.16.0.40 metric 600

WiFi has the metric 600 and Ethernet 100. This means that Ethernet is the preferred connection. If it wasn't connected, then WiFi would be used. This is basic routing, where routers pick the lowest cost (metric) when choosing a route.

The WiFi address is 172.16.0.40 and Ethernet 172.16.0.42.

I can ping either address.

The MAC for WiFi is 38:59:f9:e0:7d:5d and Ethernet f0f1:8c:dc:99. If I ping the WiFi address and watch with Wireshark, I can see the packets for the WiFi IP address 172.16.0.40 contain the Ethernet MAC address f0f1:8c:dc:99.

This means the problem described by the OP does not happen with Linux, and I expect macOS, but it does with Windows.

Now, perhaps instead of arguing about this, he could simply try it and let us know the results. That is connect via Ethernet, with WiFi up, and see if he can ping the WiFi ip address. If he can, his problem is solved.

BTW, I just noticed this site likes to insert a German flag in the Ethernet MAC address. I guess this is because the 2 hidden characters are "de".

JKnott

@johnpoz said in Handling Multiple Interfaces on Client System?:

That is how my laptop works.

Windows perhaps?

I have to on purpose enable the wifi after connecting ethernet if want to do something odd. Also you should hand your display its own name vs a reservation. Why in the world world would say ssh be listening on your display ;)
From dns point of view you would need to use different name for different IP if you want to distinguish which one you connect to. If you have multiple IPs for the same record, then you would get them back round robin style or all of them depending. Normally you would have wifi and wired on different networks, using different domains - so for example on my network wired is local.lan, while one of the wifi networks is wlan.local.lan and there is dmz.local.lan and psk.local.lan, etc. So If want to talk to a devices wlan IP would use host.wlan.local.lan to resolve it.

As I mentioned, the only relevant IP address/host name is the WiFi. Linux is able to figure things out. On the other hand on the same computer, running Windows 10, WiFi is disconnected when Ethernet is connected, which means a different IP address/host name has to be used.

Since the OP is running macOS, I expect it will behave the same as Linux. However, the way to find out is for him to try it. It's not hard to do and will take only a few seconds.

johnpoz

dude not sure why your having a hard time understanding his point..

Yeah client will USE the best interface per a metric for outbound connections. He point is he on on a different machine.. And wants to ssh to said box with multiple IPs.. And wants to connect to specific IP..

His box has 3 IPs... A B and C... he wants to create a connection to A via name.. How would linux magically handle this when dns hands back C or hands back All 3 of them??

JKnott

@johnpoz

Unless I'm mistaken, he has multiple interfaces on the same system, just like I have here. I thought he didn't want to use different host names/IPs when connecting. If that is correct, then what I described is exactly what he wants. As long as WiFi is up he does not have to worry about the Ethernet address or host name. I expect, with his computer, WiFi is always up and would have an IP address. BSD is likely able to handle that, just like Linux does. So, all he has to do is plug in Ethernet and then try pinging the WiFi address. If that works then his problem is solved. He can also use the ifconfig command, to see what the metric is for each interface. As long as WiFi has the highest metric, then this will work fine.

Again, this will not work with Windows.

JKnott

@johnpoz said in Handling Multiple Interfaces on Client System?:

How would linux magically handle this when dns hands back C or hands back All 3 of them??

I have attached 2 files, one of pings and the other for arp. Please note the MAC addresses. It's the same for both the WiFi and Ethernet IP addresses, which shows that the same interface, Ethernet, is being used for both. I'll have to try again later, after the MAC addresses have expired from the cache, to show what happens from the first arp broadcast. Regardless, at the moment, my desktop computer arp cache lists the same MAC for both Ethernet and WiFi IP addresses.

172.16.0.40 dev eth0 lladdr f0f1:8c:dc:99 STALE
172.16.0.42 dev eth0 lladdr f0f1:8c:dc:99 STALE

Substitute "de" for the German flags.

arp.pcapng ping.pcapng

JKnott

@JKnott said in Handling Multiple Interfaces on Client System?:

I'll have to try again later, after the MAC addresses have expired from the cache, to show what happens from the first arp broadcast.

Further on this by reflecting on how arp works. When there is no arp cache entry for an IP address, the device will send the arp request to the broadcast address. Then the device that has that IP address receives the request will reply. In the case I've been describing, where there are 2 interfaces, it will reply through the one with the lowest metric, even though the broadcast was heard on both interfaces. After this, the arp cache now has an entry for that IP through the interface that has that lowest metric. Further requests go to the same MAC, rather than broadcast. As a result, the WiFi address is listed as having the same MAC as Ethernet and so all traffic for it go through the Ethernet port.

mbelanger

Wow. I didn't anticipate my question would generate such a discussion. Thank you to those who chimed in and helped me better understand how Linux networking works in this specific instance.

@JKnott got my situation. At various times, I might connect via three different interfaces (Wi-Fi, Display Ethernet, Thunderbolt Ethernet dongle). WiFi is always on, but I'll connect to hardline when pulling down distribution ISOs or backing up my Blu-ray rips to b2. Using pfSense Static DHCP mappings has a field for hostname, in which I can put "lappy" for all three interfaces or "lappy-en0, -en2, -en3" as appropriate. The latter doesn't help solve my problem, but the former doesn't really work either, as DNS will return all three addresses regardless of whether they are up. Ultimately, this problem is beyond the scope pfSense to solve. Since I leave WiFi on most of the time, I opted to set that interface to "lappy" and the other two by appending their interface suffix.

One clarification about Mac networking, though. In typical Apple fashion, they don't use metric to determine interface to go through. Instead, one sets the preferred service order either via the GUI or the CL tool networksetup, which I suppose is simpler to grok than using metric. FYI.

JKnott

@mbelanger said in Handling Multiple Interfaces on Client System?:

One clarification about Mac networking, though. In typical Apple fashion, they don't use metric to determine interface to go through. Instead, one sets the preferred service order either via the GUI or the CL tool networksetup, which I suppose is simpler to grok than using metric. FYI.

That's curious, as Mac OS is based on a BSD, just like pfSense. The FreeBSD under pfSense does show a metric. However, since I don't have multiple connections to the same network, I can't check it.

So, if you connect via Ethernet and then ping the WiFi address it doesn't work? It certainly does with Linux, but not Windows.

mbelanger

@JKnott said in Handling Multiple Interfaces on Client System?:

So, if you connect via Ethernet and then ping the WiFi address it doesn't work? It certainly does with Linux, but not Windows.

They have different IP addresses assigned to them, so it depends which address you ping. I'll try changing the hostnames to match in pfSense later and report back.

That said, the DNS client story within macOS isn't that great, so it's difficult to troubleshoot. Apple-built tools and POSIX tools don't use the same DNS lookup mechanisms. As such, nslookup, dig, dscacheutil and scutil can return different results. Some third-parties, like Google, use their own implementations. A few OS revisions ago, Apple released a new solution for DNS lookups, but it was fraught with problems and they ultimately gave up and restored the older solution as the primary.

JKnott

@mbelanger said in Handling Multiple Interfaces on Client System?:

They have different IP addresses assigned to them, so it depends which address you ping.

Yes, I know they have different addresses assigned. That's the whole point of what I've been saying. On Linux, if I ping the WiFi address, when Ethernet is connected, I still get an reply, even though it goes through the Ethernet port. Have you tried what I asked, that is ping the WiFi address? Forget anything about host names at this point, we're focusing on addresses only. For example, on my notebook, WiFi is 172.16.0.40 and Ethernet is 172.16.0.42. When Ethernet is connected and WiFi is up, pinging 172.16.0.40 will get a reply. Have you tried that? You seem to keep going back to host names, when at this point they're irrelevant to the discussion.