WAN interface dropping connections after upgrade to 2.3.2
-
Hi All,
After updating our firewall to version 2.3.2 WAN interface is dropping connections sometimes, only a reboot is solving the issue. This happens at irregular times and never happened before the upgrade to 2.3.2.
Below are the logs, if others have a solution for this issue, please let me know. If more information is needed I'm more than happy to give that, hopefully we can solve this issue.
In the logs we see the firewall starting with check_reload, then a couple xinetd messages, afterwards a repeating set of logs starting with a watchdog timeout:
Sep 22 11:55:59 hostname check_reload_status: updating dyndns GW
Sep 22 11:55:59 hostname check_reload_status: Restarting ipsec tunnels
Sep 22 11:55:59 hostname check_reload_status: Restarting OpenVPN tunnels/interfaces
Sep 22 11:55:59 hostname check_reload_status: Reloading filter
Sep 22 11:56:00 hostname xinetd[6391]: Starting reconfiguration
Sep 22 11:56:00 hostname xinetd[6391]: Swapping defaults
Sep 22 11:56:00 hostname xinetd[6391]: readjusting service 6969-udp
Sep 22 11:56:00 hostname xinetd[6391]: Reconfigured: new=0 old=1 dropped=0 (services)Sep 22 11:56:16 hostname kernel: em1: Watchdog timeout Queue[0]– resetting
Sep 22 11:56:16 hostname kernel: Interface is RUNNING and ACTIVE
Sep 22 11:56:16 hostname kernel: em1: TX Queue 0 ------
Sep 22 11:56:16 hostname kernel: em1: hw tdh = 375, hw tdt = 335
Sep 22 11:56:16 hostname kernel: em1: Tx Queue Status = -2147483648
Sep 22 11:56:16 hostname kernel: em1: TX descriptors avail = 40
Sep 22 11:56:16 hostname kernel: em1: Tx Descriptors avail failure = 971
Sep 22 11:56:16 hostname kernel: em1: RX Queue 0 ------
Sep 22 11:56:16 hostname kernel: em1: hw rdh = 440, hw rdt = 438
Sep 22 11:56:16 hostname kernel: em1: RX discarded packets = 0
Sep 22 11:56:16 hostname kernel: em1: RX Next to Check = 439
Sep 22 11:56:16 hostname kernel: em1: RX Next to Refresh = 438
Sep 22 11:56:16 hostname kernel: em1: link state changed to DOWN
Sep 22 11:56:16 hostname check_reload_status: Linkup starting em1
Sep 22 11:56:17 hostname php-fpm[57455]: /rc.linkup: Hotplug event detected for WAN(wan) static IP (x.x.x.x)
Sep 22 11:56:17 hostname check_reload_status: Reloading filter
Sep 22 11:56:18 hostname xinetd[6391]: Starting reconfiguration
Sep 22 11:56:18 hostname xinetd[6391]: Swapping defaults
Sep 22 11:56:18 hostname xinetd[6391]: readjusting service 6969-udp
Sep 22 11:56:18 hostname xinetd[6391]: Reconfigured: new=0 old=1 dropped=0 (services)
Sep 22 11:56:19 hostname check_reload_status: Linkup starting em1
Sep 22 11:56:19 hostname kernel: em1: link state changed to UP
Sep 22 11:56:20 hostname php-fpm[81403]: /rc.linkup: Hotplug event detected for WAN(wan) static IP (x.x.x.x)
Sep 22 11:56:20 hostname check_reload_status: rc.newwanip starting em1
Sep 22 11:56:20 hostname check_reload_status: Reloading filter
Sep 22 11:56:21 hostname php-fpm[81403]: /rc.newwanip: rc.newwanip: Info: starting on em1.
Sep 22 11:56:21 hostname php-fpm[81403]: /rc.newwanip: rc.newwanip: on (IP address: x.x.x.x) (interface: WAN[wan]) (real interface: em1).
Sep 22 11:56:21 hostname check_reload_status: Reloading filter
Sep 22 11:56:21 hostname xinetd[6391]: Starting reconfiguration
Sep 22 11:56:21 hostname xinetd[6391]: Swapping defaults
Sep 22 11:56:21 hostname xinetd[6391]: readjusting service 6969-udp
Sep 22 11:56:21 hostname xinetd[6391]: Reconfigured: new=0 old=1 dropped=0 (services)
Sep 22 11:56:22 hostname xinetd[6391]: Starting reconfiguration
Sep 22 11:56:22 hostname xinetd[6391]: Swapping defaults
Sep 22 11:56:22 hostname xinetd[6391]: readjusting service 6969-udp
Sep 22 11:56:22 hostname xinetd[6391]: Reconfigured: new=0 old=1 dropped=0 (services)Sep 22 11:56:46 hostname kernel: em1: Watchdog timeout Queue[0]– resetting
Sep 22 11:56:46 hostname kernel: Interface is RUNNING and ACTIVE
Sep 22 11:56:46 hostname kernel: em1: TX Queue 0 ------
Sep 22 11:56:46 hostname kernel: em1: hw tdh = 0, hw tdt = 984
Sep 22 11:56:46 hostname kernel: em1: Tx Queue Status = -2147483648
Sep 22 11:56:46 hostname kernel: em1: TX descriptors avail = 40
Sep 22 11:56:46 hostname kernel: em1: Tx Descriptors avail failure = 991
Sep 22 11:56:46 hostname kernel: em1: RX Queue 0 ------
Sep 22 11:56:46 hostname kernel: em1: hw rdh = 0, hw rdt = 1023
Sep 22 11:56:46 hostname kernel: em1: RX discarded packets = 0
Sep 22 11:56:46 hostname kernel: em1: RX Next to Check = 0
Sep 22 11:56:46 hostname kernel: em1: RX Next to Refresh = 0
Sep 22 11:56:46 hostname kernel: em1: link state changed to DOWN
Sep 22 11:56:46 hostname check_reload_status: Linkup starting em1
Sep 22 11:56:47 hostname php-fpm[84921]: /rc.linkup: Hotplug eventted for WAN(wan) static IP (x.x.x.x)
Sep 22 11:56:47 hostname check_reload_status: Reloading filter
Sep 22 11:56:48 hostname xinetd[6391]: Starting reconfiguration
Sep 22 11:56:48 hostname xinetd[6391]: Swapping defaults
Sep 22 11:56:48 hostname xinetd[6391]: readjusting service 6969-udp
Sep 22 11:56:48 hostname xinetd[6391]: Reconfigured: new=0 old=1=0 (services)
Sep 22 11:56:49 hostname check_reload_status: Linkup starting em1
Sep 22 11:56:49 hostname kernel: em1: link state changed to UP