Losing WAN when receiving VOIP call



  • I've recently been having an issue that I'm struggling to figure out. I use Ooma and sometimes when I answer the phone, my WAN connection drops. I do see Pfsense report that the WAN is disconnected, but cannot figure out why.

    This lasts for 2 minutes, then it all starts working again. This only happens periodically and I'm not sure it isn't my ISPs fault either. I'm just trying to debug a bit before I call him.

    Any ideas what I could look at when this happens to help figure out why?

    In case it matter, I did update to 2.4.4 to see if that helped, but it made no difference.

    thanks
    david


  • Netgate Administrator

    What log entries are you actually seeing?

    Is the WAN connection actually going down or just the gateway monitoring showing an alarm?

    Steve



  • Not exactly what logs to look at. I do see this:

    Oct 19 14:25:04 pfsense rc.gateway_alarm[46339]: >>> Gateway alarm: WAN (Addr:9.9.9.9 Alarm:1 RTT:22.887ms RTTsd:2.871ms Loss:21%)
    Oct 19 14:25:04 pfsense check_reload_status: updating dyndns WAN
    Oct 19 14:25:04 pfsense check_reload_status: Restarting ipsec tunnels
    Oct 19 14:25:04 pfsense check_reload_status: Restarting OpenVPN tunnels/interfaces
    Oct 19 14:25:04 pfsense check_reload_status: Reloading filter
    Oct 19 14:25:05 pfsense php-fpm[59963]: /rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. ''
    Oct 19 14:25:05 pfsense php-fpm[59963]: /rc.openvpn: Static Routes: Gateway IP could not be found for 50.18.0.0/16
    Oct 19 14:25:05 pfsense php-fpm[59963]: /rc.openvpn: Static Routes: Gateway IP could not be found for 184.169.0.0/16
    Oct 19 14:25:05 pfsense php-fpm[59963]: /rc.openvpn: Static Routes: Gateway IP could not be found for 184.72.0.0/16
    Oct 19 14:25:05 pfsense php-fpm[59963]: /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN.
    Oct 19 14:25:05 pfsense php-fpm[1634]: /rc.dyndns.update: MONITOR: WAN is down, omitting from routing group WAN_Group1 9.9.9.9|xxx.xxx.xxx.xxx|WAN|22.892ms|2.899ms|23%|down
    Oct 19 14:25:05 pfsense php-fpm[1634]: /rc.dyndns.update: Gateways status could not be determined, considering all as up/active. (Group: WAN_Group1)
    Oct 19 14:25:05 pfsense php-fpm[1635]: /rc.filter_configure_sync: Gateways status could not be determined, considering all as up/active. (Group: WAN_Group1)
    Oct 19 14:25:05 pfsense php-fpm[1634]: /rc.dyndns.update: Gateways status could not be determined, considering all as up/active. (Group: WAN_Group1)
    Oct 19 14:25:05 pfsense php-fpm[1634]: /rc.dyndns.update: Gateways status could not be determined, considering all as up/active. (Group: WAN_Group1)
    Oct 19 14:25:05 pfsense php-fpm[1634]: /rc.dyndns.update: Gateways status could not be determined, considering all as up/active. (Group: WAN_Group1)
    Oct 19 14:25:05 pfsense php-fpm[1634]: /rc.dyndns.update: Gateways status could not be determined, considering all as up/active. (Group: WAN_Group1)
    Oct 19 14:25:05 pfsense php-fpm[1634]: /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Oct 19 14:25:06 pfsense php-fpm[1635]: /rc.filter_configure_sync: The gateway: WAN_DHCP is invalid or unknown, not using it.
    Oct 19 14:25:06 pfsense php-fpm[1635]: /rc.filter_configure_sync: The gateway: WAN_DHCP is invalid or unknown, not using it.
    Oct 19 14:25:06 pfsense php-fpm[1635]: /rc.filter_configure_sync: The gateway: WAN_DHCP is invalid or unknown, not using it.
    Oct 19 14:25:25 pfsense check_reload_status: Linkup starting re2
    Oct 19 14:25:25 pfsense kernel: re2: link state changed to DOWN
    Oct 19 14:25:26 pfsense php-fpm[54512]: /rc.linkup: Hotplug event detected for WAN(wan) static IP (xxx.xxx.xxx.xxx )
    Oct 19 14:25:26 pfsense check_reload_status: Reloading filter
    Oct 19 14:25:26 pfsense kernel: arpresolve: can't allocate llinfo for xxx.xxx.xxx.xxx on re2
    

    The arpresolve message continue for the entire time it is down

    I see this sometimes:

    Oct 19 14:25:55 pfsense php-fpm[1634]: /rc.filter_configure_sync: The gateway: WAN_DHCP is invalid or unknown, not using it.
    O
    

    Note sure where that is from as I don't have a WAN_DHCP gateway defined.

    And then when it starts working I see

    Oct 19 14:27:33 pfsense rc.gateway_alarm[27204]: >>> Gateway alarm: WAN (Addr:9.9.9.9 Alarm:0 RTT:22.751ms RTTsd:3.137ms Loss:5%)
    Oct 19 14:27:33 pfsense check_reload_status: updating dyndns WAN
    Oct 19 14:27:33 pfsense check_reload_status: Restarting ipsec tunnels
    Oct 19 14:27:33 pfsense check_reload_status: Restarting OpenVPN tunnels/interfaces
    Oct 19 14:27:33 pfsense check_reload_status: Reloading filter
    Oct 19 14:27:34 pfsense php-fpm[59963]: /rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. ''
    Oct 19 14:27:34 pfsense php-fpm[1635]: /rc.dyndns.update: 1635MONITOR: WAN is available now, adding to routing group WAN_Group1 9.9.9.9|xxx.xxx.xxx.xxx|WAN|22.733ms|3.117ms|3%|none
    Oct 19 14:27:34 pfsense php-fpm[59963]: /rc.openvpn: Static Routes: Gateway IP could not be found for 50.18.0.0/16
    Oct 19 14:27:34 pfsense php-fpm[59963]: /rc.openvpn: Static Routes: Gateway IP could not be found for 184.169.0.0/16
    Oct 19 14:27:34 pfsense php-fpm[59963]: /rc.openvpn: Static Routes: Gateway IP could not be found for 184.72.0.0/16
    Oct 19 14:27:34 pfsense php-fpm[59963]: /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN.
    Oct 19 14:27:34 pfsense php-fpm[1635]: /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Oct 19 14:27:35 pfsense php-fpm[1634]: /rc.filter_configure_sync: The gateway: WAN_DHCP is invalid or unknown, not using it.
    Oct 19 14:27:35 pfsense php-fpm[1634]: /rc.filter_configure_sync: The gateway: WAN_DHCP is invalid or unknown, not using it.
    Oct 19 14:27:35 pfsense php-fpm[1634]: /rc.filter_configure_sync: The gateway: WAN_DHCP is invalid or unknown, not using it.
    

    Hopefully this helps,
    david


  • Netgate Administrator

    @lovinghdtv said in Losing WAN when receiving VOIP call:

    Oct 19 14:25:25 pfsense kernel: re2: link state changed to DOWN
    Oct 19 14:25:26 pfsense php-fpm[54512]: /rc.linkup: Hotplug event detected for WAN(wan) static IP (xxx.xxx.xxx.xxx )

    That implies that the link was actually lost on re2. That is 20s after it started showing packet loss.

    I would try swapping that interface with another one and see if the fault stays on WAN or follows re2.

    Steve



  • Is there a method I should use to swap two interfaces that are already in service?

    I tried just swapping them in the assignments page, save->apply, but could never reestablish WAN connectivity. I had to set them back to their original settings and reboot to get it to work.

    thanks


  • Netgate Administrator

    That should work. Obviously you would also have to swap the cables to the NICs to use the new assignments.

    Steve



  • hmm, did swap the cables as well. I'll have to wait for later when everyone doesn't get mad at me and try again.

    thanks for your help


  • Netgate Administrator

    Best to avoid angry users 😉



  • Hmm, if I change my WAN to a different interface (re2->re1)it can no longer connect to the internet. As soon as I change it back, it connects just fine.

    Things I've done.

    1. verified with my ISP that there is no MAC address limitation
    2. tried changing via GUI
      3 tried changing via command prompt
    3. rebooting after every change both router and modem

    any other idea on why I can't change from re2 -> re1? I know that re1 works as I use it as my failover interface.

    thanks
    davdi



  • @lovinghdtv

    What kind of internet connection are you using and what modem?



  • That's one of the issue I experienced in the past when trying to use Realtek NICs in FreeBSD, a sudden increase in small packets crashed the interface. You can try to use the driver from Realtek instead of the FreeBSD included one, check the hardware forum for it. Best would be to use Intel or Broadcom based NICs.



  • @chpalmer said in Losing WAN when receiving VOIP call:

    @lovinghdtv

    What kind of internet connection are you using and what modem?

    It is a fixed wireless modem from Ubiquiti. I don't have the name of it, though I could ask if that helps.

    david



  • @grimson said in Losing WAN when receiving VOIP call:

    That's one of the issue I experienced in the past when trying to use Realtek NICs in FreeBSD, a sudden increase in small packets crashed the interface. You can try to use the driver from Realtek instead of the FreeBSD included one, check the hardware forum for it. Best would be to use Intel or Broadcom based NICs.

    I have a 4 port Intel card, just don't have a low-profile bracket for it. Ordered one so when it comes in I'll swap it over.

    thanks for the suggestion.
    david


  • Netgate Administrator



  • how do I verify I have a Realtek chipset?

    NM: google helped me:
    pciconf -lv

    reports all three are Realtek


  • Netgate Administrator

    Yes, the reX interface names confirm that.

    Steve



  • @lovinghdtv

    Was just ruling out a Puma6 equipped modem issue.. UDP traffic on those can cause this.



  • It happened again.

    I did verify that the new drivers are loaded

    
    [2.4.4-RELEASE][admin@pfsense.home]/var/log: kldstat
    Id Refs Address            Size     Name
     1   27 0xffffffff80200000 2d9a7d0  kernel
     2    1 0xffffffff82f9c000 7d2c0    if_re.ko
     3    1 0xffffffff833f9000 10a0     cpuctl.ko
     4    1 0xffffffff833fb000 72b8     aesni.ko
     5    1 0xffffffff83403000 2ec0     cryptodev.ko
     6    1 0xffffffff83406000 11a0     amdtemp.ko
     7    1 0xffffffff83408000 648      amdsmn.ko
     8    1 0xffffffff83409000 10800    dummynet.ko
    
    

    Here is what I see in the system.log

    Oct 28 18:41:23 pfsense check_reload_status: Linkup starting re2
    Oct 28 18:41:23 pfsense kernel: re2: link state changed to DOWN
    Oct 28 18:41:24 pfsense php-fpm[50605]: /rc.linkup: Hotplug event detected for WAN(wan) static IP (xxx.xxx.xxx.xxx )
    Oct 28 18:41:24 pfsense check_reload_status: Reloading filter
    Oct 28 18:41:25 pfsense php-fpm[50605]: /rc.filter_configure_sync: Gateways status could not be determined, considering all as up/active. (Group: WAN_Group1)
    Oct 28 18:41:26 pfsense check_reload_status: Linkup starting re2
    Oct 28 18:41:26 pfsense kernel: re2: link state changed to UP
    Oct 28 18:41:27 pfsense php-fpm[64170]: /rc.linkup: Hotplug event detected for WAN(wan) static IP (xxx.xxx.xxx.xxx )
    Oct 28 18:41:27 pfsense check_reload_status: rc.newwanip starting re2
    Oct 28 18:41:27 pfsense check_reload_status: Reloading filter
    Oct 28 18:41:28 pfsense php-fpm[64170]: /rc.newwanip: rc.newwanip: Info: starting on re2.
    Oct 28 18:41:28 pfsense php-fpm[64170]: /rc.newwanip: rc.newwanip: on (IP address: xxx.xxx.xxx.xxx) (interface: WAN[wan]) (real interface: re2).
    Oct 28 18:41:28 pfsense check_reload_status: Reloading filter
    Oct 28 18:41:28 pfsense php-fpm[42540]: /rc.filter_configure_sync: Gateways status could not be determined, considering all as up/active. (Group: WAN_Group1)
    Oct 28 18:41:30 pfsense php-fpm[64170]: /index.php: Successful login for user 'admin' from: 192.168.1.175 (Local Database)
    

    Not sure if that helps figure out what is going on.

    thanks
    david



  • What brand of wireless bridge radio do you have?

    This comment- "I've recently been having an issue that I'm struggling to figure out"

    Does this mean that it started to occur suddenly or from the first time you started using a VOIP service?


  • Netgate Administrator

    Not really. It shows the link went down and then came back up after 3 seconds.

    I would still try to swap out re2 for a different NIC to verify the fault stays on the NIC.

    What is it actually connected to? Does that log anything?

    You might try adding a switch in between the WAN and whatever it's connected to. The link should never go down with that in place.

    Steve



  • @chpalmer said in Losing WAN when receiving VOIP call:

    What brand of wireless bridge radio do you have?

    This comment- "I've recently been having an issue that I'm struggling to figure out"

    Does this mean that it started to occur suddenly or from the first time you started using a VOIP service?

    I've had voip on this router over a year, it just started happening.

    david



  • Then ask your ISP if anything about your circuit has changed on their end. Just to cover your bases.



  • @stephenw10 said in Losing WAN when receiving VOIP call:

    Not really. It shows the link went down and then came back up after 3 seconds.

    I would still try to swap out re2 for a different NIC to verify the fault stays on the NIC.

    What is it actually connected to? Does that log anything?

    You might try adding a switch in between the WAN and whatever it's connected to. The link should never go down with that in place.

    Steve

    I ordered a low profile bracket and when it comes I'll remove the 2 port realtek and replace with a 4 port Intel.

    I have tried swapping re1 for re2, but it never works. I tried it several times and for whatever reason re1 never works. I tired in the GUi and via command line. Now re1 works great through my failover WAN. I don't know why I can't reassign the interface. By that I mean get it to work. Pfsense says it is reassign, but it never assigns the static IP and doesn't work.

    The WAN is only connected to my modem/antenna. I run pingplotter and can see that the WAN drops as soon as I answer the phone call. Interestingly I've never had it drop if I let it ring more than once :)

    david



  • @chpalmer said in Losing WAN when receiving VOIP call:

    Then ask your ISP if anything about your circuit has changed on their end. Just to cover your bases.

    I already texted him :)