Weird DNS issue with Android Wifi Calling



  • Weird one, it's probably not strictly a pfsense problem, but maybe you guys can point me in the right direction.

    There is some oddity with android wifi calling and my network - I realised this by connecting to a quick network I setup that bypasses pfsense etc and it worked fine. After investigating it seems there's a weird conflict between Android Wifi Calling and my Microsoft DNS (my GF's iPhone works fine on Wifi Calling on my network)

    Before I had my phone connecting via DHCP to my pfsense box which then gave it DNS servers on my network, those DNS Servers end up forwarding to Google's DNS servers. For some reason, despite my whole network working with this, and my android phone generally working with this, it seems that this destroys wifi calling. If I connect to the wifi network with a static IP and use Google's DNS servers directly, it works.

    Has anyone got any ideas? This is a very strange one.



  • That's very odd but it sounds like you'll learn something new and interesting through resolving this! :)

    From some slight perusing, the only thing I was able to find that seem relevant was this T-Mobile issued statement on WiFi-Calling issues:

    https://support.t-mobile.com/docs/DOC-4950

    The REG99: Unable to Connect error may appear when Wi-Fi Calling is active and connected to certain routers or devices and is usually caused by the following:

    Device is not sending the request out due to a software error.
      It may be an issue with the DNS response from the DNS server when the device tries to resolve the domain name configured in the IMS client.
        Router is blocking the register request from the device before it is sent to the ISP.
      This could be a firewall or other router related issue. Sometimes, this will happen with public networks because their firewall is configured to block this type of traffic.
        ISP is not forwarding the request to the T-Mobile network. The request may not reach the T-Mobile network due to an internal issue. This indicates the device is not able to connect to Wi-Fi Calling.
        Device can make calls over the network, but not via Wi-Fi Calling

    From the PFSense forums, this looked to be a similar topic that was resolved by other users: https://forum.pfsense.org/index.php?topic=93506.msg566074#msg566074
    Have you had a chance to take a look there?

    I was hoping to find more information on WiFi calling at a mid-technical tier that had information on both the current usage and expected results but a lot of it was Android programmer Q&A

    Are you getting an error message or will the calls just fail with no other information? Also what is your carrier?



  • This is the problem, there are no error logs or messages, it either works or it doesn't.

    The only change that seems to matter is which DNS it uses, if I point it to my windows DNS servers, it won't work, if I point it to Googles it works fine. This is why I'm thinking it's a DNS issue.

    I've done a lot of reading over the past few days, and my understanding is that the EE wifi calling does a ipsec vpn tunnel using ipv6. My home network is completely ipv4, and my ISP is  ipv4 only. The apn is:
    <apn carrier="EE IMS" <br="">mcc="234"
    mnc="30"
    apn="ims"
    type="ims"
    protocol="IPV6"
    roaming_protocol="IPV6"
    />

    This is where I get fuzzy, I'm not sure how this ipv6 stuff works on an ipv4 network, and my DNS server doesn't have am ipv6 ip. Could this be the source of my troubles?

    My carrier is EE (Everything Everywhere) in the UK</apn>


  • LAYER 8 Netgate

    I would packet capture and look at the DNS queries and answers.



  • Hi,

    I've just done this, and I think I've found the culprit.

    For some reason my DNS isn't responding to this DNS query, you can see it trying my backup DNS, whereas Google responds first time. Anyone have any idea why?! All other DNS queries look fine.





  • LAYER 8 Netgate

    $ dig epdg.epc.mnc030.mcc234.pub.3gppnetwork.org a

    ; <<>> DiG 9.8.3-P1 <<>> epdg.epc.mnc030.mcc234.pub.3gppnetwork.org a
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64656
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 6, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;epdg.epc.mnc030.mcc234.pub.3gppnetwork.org. IN A

    ;; ANSWER SECTION:
    epdg.epc.mnc030.mcc234.pub.3gppnetwork.org. 300 IN A 109.249.188.56
    epdg.epc.mnc030.mcc234.pub.3gppnetwork.org. 300 IN A 109.249.180.0
    epdg.epc.mnc030.mcc234.pub.3gppnetwork.org. 300 IN A 109.249.186.72

    ;; AUTHORITY SECTION:
    mnc030.mcc234.pub.3gppnetwork.org. 86400 IN NS j4.nstld.com.
    mnc030.mcc234.pub.3gppnetwork.org. 86400 IN NS k4.nstld.com.
    mnc030.mcc234.pub.3gppnetwork.org. 86400 IN NS g4.nstld.com.
    mnc030.mcc234.pub.3gppnetwork.org. 86400 IN NS l4.nstld.com.
    mnc030.mcc234.pub.3gppnetwork.org. 86400 IN NS f4.nstld.com.
    mnc030.mcc234.pub.3gppnetwork.org. 86400 IN NS a4.nstld.com.

    ;; Query time: 282 msec
    ;; SERVER: 192.168.223.1#53(192.168.223.1)
    ;; WHEN: Mon Mar 28 17:29:26 2016
    ;; MSG SIZE  rcvd: 219

    Resolves here.

    What do you get? You'll want to be sure you're pointing drill at the resolver these clients are using if doing it from pfSense.

    eg. $ drill @127.0.0.1 epdg.epc.mnc030.mcc234.pub.3gppnetwork.org a



  • Thats the weird thing. If I do a NSLOOKUP on one of my machines it resolves fine - it's simply the request from the android device that isn't being responded to. I setup a log on my DNS Server to show it's received and sent packets, and the DNS service doesn't seem to be getting these DNS queries. I don't understand, there's no firewall on the DNS server :/



  • I found a tool that did a NSLOOKUP style thing on Android, and that got a response from my DNS server so I analysed the differences in packets. I've found it.

    The MAC address seems to be wrong on the DNS request.

    The ones that work have this:

    Dst: Universa_08:67:34 (WINDOWS MAC ADDRESS)

    The ones that get no response have this:

    Dst: Microsof_01:f4:01 (00:15:5d:01:f4:01) - which appears to be Google DNS's mac address.

    Why is the MAC address wrong?!



  • Sorry to post (again), but I changed my DNS to OpenDNS to test, and it still sends it the wrong MAC address, but it responds anyway. Seems to be an android bug, any way to setup my windows server to ignore checking the MAC address and accept the packet?



  • MAC addresses are only locally relevant, only visible on the same broadcast domain. That's not Google DNS's MAC, it's your default gateway's MAC. Sounds like you have an IP conflict for your gateway IP. Only one MAC should ever answer for a given IP.



  • @cmb:

    MAC addresses are only locally relevant, only visible on the same broadcast domain. That's not Google DNS's MAC, it's your default gateway's MAC. Sounds like you have an IP conflict for your gateway IP. Only one MAC should ever answer for a given IP.

    Thanks for the response, thanks for the correction on MAC addresses. I'm not saying you're incorrect, but I'm just confused as to why only my android device (and only on that one DNS request) is using my gateway (pfsense) mac while talking to my internal DNS server. Im 99% positive my pfsense server and DNS server are the only computers using their ip addresses. I suppose the easy way to fix this (can't test currently as now in bed) is to set google DNS as my secondary DNS, as I see it always checks that after not getting a response from the primary, but it'd be nice to fix the problem properly.

    What do you think I should try


  • LAYER 8 Netgate

    Look at android. What are its exact, configured name servers when connected to this network? The different MAC addresses could also also be queries to two different local hosts.


  • LAYER 8 Global Moderator

    "I'm not saying you're incorrect"

    Your kind of trying to hint at it ;)  Sorry to inform you but Derelict is correct here.. (as always) MAC are local, so unless your in the DC where one of googles namerservers are connected to the same L2 you are that is not googles mac address ;)

    I never get why anyone would set a 2nd name server that do not have the same data..  That fine if you want to point to say google and opendns out on the public internet.  But if you want too resolve your local stuff you have to point at a name server that resolves your local stuff.  So it is pointless to point at pfsense, and then 2nd point to google..

    If you need failover then you should be pointing to 2 local nameservers that both resolve your local stuff, and then either forward and resolve stuff that is not local.  I would concur with Derelict as well to the assumption that you have a duplicate IP issue.  You should never get 2 different mac back for the same IP.. So clearly you got something wrong on your network..


  • LAYER 8 Netgate

    It's right up there in the windns.PNG file. First it asks 10.0.1.30 then 10.0.1.22. Neither of which return answers. Look at your arp table and see if the two MACs match those two IP addresses. The question is why those do not return answers.

    A better question is what local DNS servers did the person who designed this network intend to be used by local hosts?

    Read the post above again. All configured name servers need to return the same answers to the same queries. At least for local records. For internet results it's possible they could get and cache different answers.



  • I assumed you were using Google's DNS given the comment re: the MAC. Source of the problem still sounds the same, just on a diff IP. If the DNS server in question is a local IP, then it's that local IP that has the IP conflict.



  • I feel like I need to explain myself a bit better.

    My gateway is pfsense on 10.0.1.19
    I have two local DNS servers, both Windows DNS on an active domain which are synced. These are 10.0.1.30 and 10.0.1.22.

    These DNS servers both forward to Google DNS, if it can't resolve it.

    This has been my setup for months, all my machines work fine, always resolving websites and local services.

    The only issue I have noticed is this wifi calling DNS request made by android. While monitoring it with wireshark all over DNS queries were made to 10.0.1.30 with the correct mac address. I downloaded a nslookup tool for Android and when requesting the same host through that the packet was correct and went to the correct DNS server with the correct mac address.

    When I said I'm not saying you're incorrect but earlier, I wasn't disputing the MACs only working locally, just the duplicate ips.

    I honestly think that this is a bug in Androids implementation of Wifi calling, because my girlfriend iPhone has always consistently worked (with my local DNS servers) with EE Wifi Calling. It's only been my android phone that's had any issues. It seems there's a bug where this one call gets the ip of the DNS server and the mac of the gateway server mixed up and uses the wrong combination. I think this one call is special because it's a core built in service.

    I'm really happy to be wrong and to find a solution though. But that's my thinking at the moment. This bug could be easily missed as 90% of home users won't run a local DNS so the DNS MAC will match the gateway MAC.

    Here is my ARP cache in pfsense. https://imgur.com/6X6LU5g

    Honestly I want to be wrong because an Android bug is going to be a PITA to get fixed. Anyone think I've misunderstood or am wrong?

    Cheers

    (also sorry for bad spelling, on mobile)


  • LAYER 8 Netgate

    Well if your Windows DNS Servers forward to google for global resolution I don't see where pfSense fits into the picture.

    MAC addresses really have nothing to do with your problem unless, as has been said, you have an IP address conflict. Just forget about MAC addresses.

    It really sounds like you need to realize a DNS strategy, get it working, then make sure all your devices are configured to use it.

    Is the working iPhone configured to use the same DNS servers as the broken android phone?



  • @Derelict:

    Well if your Windows DNS Servers forward to google for global resolution I don't see where pfSense fits into the picture.

    MAC addresses really have nothing to do with your problem unless, as has been said, you have an IP address conflict. Just forget about MAC addresses.

    It really sounds like you need to realize a DNS strategy, get it working, then make sure all your devices are configured to use it.

    Is the working iPhone configured to use the same DNS servers as the broken android phone?

    Yes, the iPhone has 10.0.1.30 and 10.0.1.22 as DNS servers and works perfectly, as does every other computer on the network. The only reason I bring up mac addresses is because the broken DNS packet as seen via wireshark has the wrong mac address so the DNS server isn't recieving the packet (i'm right in the assumption that no matter what the ip address is, NIC will ignore packets sent to it with a different mac address aren't I?)


  • LAYER 8 Global Moderator

    "NIC will ignore packets sent to it with a different mac address aren't I?)"

    What??  Post up this sniff showing what you think is a wrong mac..

    In your sniff there was NO answer to your queries..  No shit if your dns does not answer you can not look up stuff..



  • @johnpoz:

    "NIC will ignore packets sent to it with a different mac address aren't I?)"

    What??  Post up this sniff showing what you think is a wrong mac..

    In your sniff there was NO answer to your queries..  No shit if your dns does not answer you can not look up stuff..

    Ive attached the traces

    In the Onlythatdnsrequestbeingignored.png you can see that the DNS server is happily responding to every other DNS request coming from the android phone.

    The workingdns image shows a packet being sent to 10.0.1.30 with its correct mac address, so the dns server responds.

    The brokendns image shows the only packet thats broken. The only difference is the MAC address (which is my gateway not the dns server).

    I put Microsoft DNS in debugging mode, which logs all requests, and it never even sees the broken DNS packet (because, i assume, the NIC ignores it as it has the incorrect MAC address)

    See what I mean? This seems like unusual behaviour from my android device.







  • LAYER 8 Global Moderator

    What device do you own that is this mac?

    6C:0B:84:08:67:34 I show it as Universal Global Scientific Industrial Co.,Ltd.

    Clearly if your windows dns server is at 00:15:5d:01:f4:01 then no shit another box that is not running dns is not going to answer.  You most likely have something that has the same IP 10.0.1.30 as your windows dns server is the most likely issue.

    Is it this company http://www.usish.com/english/overview.asp  look up what they make and what device you have of theirs.



  • @johnpoz:

    What device do you own that is this mac?

    6C:0B:84:08:67:34 I show it as Universal Global Scientific Industrial Co.,Ltd.

    Clearly if your windows dns server is at 00:15:5d:01:f4:01 then no shit another box that is not running dns is not going to answer.  You most likely have something that has the same IP 10.0.1.30 as your windows dns server is the most likely issue.

    Is it this company http://www.usish.com/english/overview.asp  look up what they make and what device you have of theirs.

    No, the windows box is at 6C:0B:84:08:67:34 (10.0.1.30). Thats the mac of the integrated Intel NIC on the motherboard.

    The 00:15:5d:01:f4:01 mac address is that of pfsense (gateway)  which is running on ip 10.0.1.19.

    There is no ip conflict between them, both their IPs are static, and I posted my ARP Cache earlier and no MACs are duplicated.

    That's why I'm at a loss


  • LAYER 8 Global Moderator

    Dude clearly 00:15:5d is a microsoft mac…

    MAC Address Details

    Company
        Microsoft Corporation
    Address
        Redmond WA 98052-8300
        UNITED STATES
    Range
        00:15:5D:00:00:00 - 00:15:5D:FF:FF:FF
    Type
        IEEE MA-L

    And clearly from your sniff your sending traffic for 10.0.1.30 to that MAC...

    if your saying your windows box nic is that global universal company... That what is the microsoft mac device??

    Post up your ifconfig from your pfsense box, and y our ipconfig /all from your windows machine running dns..



  • @johnpoz:

    Dude clearly 00:15:5d is a microsoft mac…

    MAC Address Details

    Company
        Microsoft Corporation
    Address
        Redmond WA 98052-8300
        UNITED STATES
    Range
        00:15:5D:00:00:00 - 00:15:5D:FF:FF:FF
    Type
        IEEE MA-L

    And clearly from your sniff your sending traffic for 10.0.1.30 to that MAC...

    if your saying your windows box nic is that global universal company... That what is the microsoft mac device??

    Post up your ifconfig from your pfsense box, and y our ipconfig /all from your windows machine running dns..

    The reason why pfsense has a microsoft MAC is because it's a VM and that's a software NIC which is teamed with 3 NICS and LACP'ed at the router level. The 10.0.1.30 DNS Server is a physical server. 10.0.1.22 is a VM.

    pfsense:

    $ ifconfig
    pflog0: flags=100 <promisc>metric 0 mtu 33144
    pfsync0: flags=0<> metric 0 mtu 1500
    	syncpeer: 224.0.0.240 maxupd: 128 defer: on
    	syncok: 1
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
    	options=600003 <rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6>inet 127.0.0.1 netmask 0xff000000 
    	inet6 ::1 prefixlen 128 
    	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
    	nd6 options=21 <performnud,auto_linklocal>enc0: flags=0<> metric 0 mtu 1536
    	nd6 options=21 <performnud,auto_linklocal>hn0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=18 <vlan_mtu,vlan_hwtagging>ether 00:15:5d:01:f4:00
    	inet6 fe80::215:5dff:fe01:f400%hn0 prefixlen 64 scopeid 0x5 
    	nd6 options=21 <performnud,auto_linklocal>hn1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=18 <vlan_mtu,vlan_hwtagging>ether 00:15:5d:01:f4:01
    	inet6 fe80::215:5dff:fe01:f401%hn1 prefixlen 64 scopeid 0x6 
    	inet 10.0.1.19 netmask 0xffffff00 broadcast 10.0.1.255 
    	nd6 options=21 <performnud,auto_linklocal>pppoe2: flags=88d1 <up,pointopoint,running,noarp,simplex,multicast>metric 0 mtu 1492
    	inet6 fe80::215:5dff:fe01:f400%pppoe2 prefixlen 64 scopeid 0x7 
    	inet <<myip>> --> 195.166.130.211 netmask 0xffffffff 
    	nd6 options=21 <performnud,auto_linklocal>ovpns1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
    	<<omitted>>
    ovpns2: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
    	<<omitted>>
    ovpnc3: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
    	<<omitted>>
    ovpnc4: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
    	<<omitted>>
    ovpnc5: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
    	<<omitted>></omitted></up,pointopoint,running,multicast></omitted></up,pointopoint,running,multicast></omitted></up,pointopoint,running,multicast></omitted></up,pointopoint,running,multicast></omitted></up,pointopoint,running,multicast></performnud,auto_linklocal></myip></up,pointopoint,running,noarp,simplex,multicast></performnud,auto_linklocal></vlan_mtu,vlan_hwtagging></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></vlan_mtu,vlan_hwtagging></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></performnud,auto_linklocal></rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6></up,loopback,running,multicast></promisc>
    

    10.0.1.30

    
    C:\Windows\system32>ipconfig /all
    
    Windows IP Configuration
    
       Host Name . . . . . . . . . . . . : Gibson
       Primary Dns Suffix  . . . . . . . : mydomain.private
       Node Type . . . . . . . . . . . . : Hybrid
       IP Routing Enabled. . . . . . . . : No
       WINS Proxy Enabled. . . . . . . . : No
       DNS Suffix Search List. . . . . . : mydomain.private
    
    Ethernet adapter vEthernet (Internal): (this is a private VM network to some VMs)
    
       Connection-specific DNS Suffix  . :
       Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter #2
       Physical Address. . . . . . . . . : 00-15-5D-01-F4-0A
       DHCP Enabled. . . . . . . . . . . : No
       Autoconfiguration Enabled . . . . : Yes
       IPv4 Address. . . . . . . . . . . : 192.168.0.1(Preferred)
       Subnet Mask . . . . . . . . . . . : 255.255.255.0
       Default Gateway . . . . . . . . . :
       NetBIOS over Tcpip. . . . . . . . : Enabled
    
    Ethernet adapter Integrated: (this is a physical onboard NIC just for 10.0.1.30)
    
       Connection-specific DNS Suffix  . :
       Description . . . . . . . . . . . : Intel(R) Ethernet Connection I217-LM
       Physical Address. . . . . . . . . : 6C-0B-84-08-67-34
       DHCP Enabled. . . . . . . . . . . : No
       Autoconfiguration Enabled . . . . : Yes
       IPv4 Address. . . . . . . . . . . : 10.0.1.30(Preferred)
       Subnet Mask . . . . . . . . . . . : 255.255.255.0
       Default Gateway . . . . . . . . . : 10.0.1.19
       DNS Servers . . . . . . . . . . . : 127.0.0.1
                                           10.0.1.22
       NetBIOS over Tcpip. . . . . . . . : Enabled
    
    Tunnel adapter isatap.{CD48BF58-A900-4A8C-917E-6E0083ABD4E4}:
    
       Media State . . . . . . . . . . . : Media disconnected
       Connection-specific DNS Suffix  . :
       Description . . . . . . . . . . . : Microsoft ISATAP Adapter
       Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
       DHCP Enabled. . . . . . . . . . . : No
       Autoconfiguration Enabled . . . . : Yes
    
    Tunnel adapter isatap.{D7362D7E-EA03-4337-8329-58F19F230FE7}:
    
       Media State . . . . . . . . . . . : Media disconnected
       Connection-specific DNS Suffix  . :
       Description . . . . . . . . . . . : Microsoft ISATAP Adapter #3
       Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
       DHCP Enabled. . . . . . . . . . . : No
       Autoconfiguration Enabled . . . . : Yes
    
    


  • I just tried (for testing purposes, I don't want to keep it like this) switching on the DNS Forwarder on the pfsense box to 10.0.1.30, and then pointing my android phone to 10.0.1.19 (pfsense) and that works perfectly. But obviously this isn't ideal, I don't get why it's querying the windows dns server with the pfsense mac address


  • LAYER 8 Global Moderator

    So clearly your machine doing the query to 10.0.1.30 thinks that this address is at pfsense MAC..  But that is not what pfsense shows.

    Now your saying pfsense is a VM, that uses the same hardware nic as your physical machine nic.  So you have a problem in your virtual setup.

    Look on your sniffs for when your machine arps for 10.0.1.30.. For some reason it is getting the wrong mac for this..  So you need to figure out why.. You do not have any sort of static arps setup on device doing the queries.



  • @johnpoz:

    So clearly your machine doing the query to 10.0.1.30 thinks that this address is at pfsense MAC..  But that is not what pfsense shows.

    Now your saying pfsense is a VM, that uses the same hardware nic as your physical machine nic.  So you have a problem in your virtual setup.

    Look on your sniffs for when your machine arps for 10.0.1.30.. For some reason it is getting the wrong mac for this..  So you need to figure out why.. You do not have any sort of static arps setup on device doing the queries.

    Seems to be fine :/



  • LAYER 8 Global Moderator

    Yes that arp is fine, what about all the other arps?

    Dude if your never seeing arp come back with that mac.. Then how/why would you client send traffic to it..  It would be impossible for that to happen.

    That is not a normal arp, that is directed.. A normal arp is to ff:ff:ff:ff:ff:ff



  • @johnpoz:

    Yes that arp is fine, what about all the other arps?

    Dude if your never seeing arp come back with that mac.. Then how/why would you client send traffic to it..  It would be impossible for that to happen.

    That is not a normal arp, that is directed..

    I hear you, but this is why I think it's a bug with Android, because all other DNS requests from android work fine, it's just that one DNS request. I rebooted the phone, connected to wifi, and thats the only ARP I see.


  • LAYER 8 Global Moderator

    see my edit, that is not a correct arp.. An arp is to all F's


Log in to reply