Помогите решить проблему пропадания сети в определенном vlan и отвал вебинтерфейса

Jktu79

pfsense
2.4.3
В сети 3 vlan основных, но на одном из них периодически выборочно отваливаются от сети несколько адресов (хаотично), помогает ребут pfsensa.

Так же отваливается веб интерфейс.

Логи выкладываю ниже при последнем пропадании вебинтерфеса:

Feb 11 16:03:18  php-fpm  87353  /index.php: Successful login for user 'admin' from: 192.168.0.40  
Feb 11 16:03:11  tv-router.stv   nginx: 2019/02/11 16:03:11 [error] 36354#100130: *18054 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.0.93, server: , request: "POST /widgets/widgets/interfaces.widget.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.0.254", referrer: "http://192.168.0.254/"  
Feb 11 16:03:00  check_reload_status   check_reload_status is starting.  
Feb 11 16:03:00  rc.php-fpm_restart  86607  >>> Restarting php-fpm  
Feb 11 16:01:46  sshd  30800  Accepted keyboard-interactive/pam for admin from 192.168.0.40 port 57422 ssh2  
Feb 11 16:00:08  tv-router.stv   nginx: 2019/02/11 16:00:08 [error] 36354#100130: *18054 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.0.93, server: , request: "POST /widgets/widgets/interfaces.widget.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.0.254", referrer: "http://192.168.0.254/"  
Feb 11 15:58:05  check_reload_status   Linkup starting em0.30  
Feb 11 15:58:05  check_reload_status   Linkup starting em0.45  
Feb 11 15:58:05  check_reload_status   Linkup starting em0.10  
Feb 11 15:58:05  check_reload_status   Linkup starting em0.24  
Feb 11 15:58:05  check_reload_status   Linkup starting em0.20  
Feb 11 15:58:05  check_reload_status   Linkup starting em0.1  
Feb 11 15:58:05  check_reload_status   Linkup starting em0.50  
Feb 11 15:58:05  kernel   em0.30: link state changed to UP  
Feb 11 15:58:05  kernel   em0.45: link state changed to UP  
Feb 11 15:58:05  kernel   em0.10: link state changed to UP  
Feb 11 15:58:05  kernel   em0.24: link state changed to UP  
Feb 11 15:58:05  kernel   em0.20: link state changed to UP  
Feb 11 15:58:05  kernel   em0.1: link state changed to UP  
Feb 11 15:58:05  kernel   em0.50: link state changed to UP  
Feb 11 15:58:05  kernel   em0: link state changed to UP  
Feb 11 15:58:05  check_reload_status   Linkup starting em0  
Feb 11 15:58:02  check_reload_status   Linkup starting em0.30  
Feb 11 15:58:02  check_reload_status   Linkup starting em0.45  
Feb 11 15:58:02  check_reload_status   Linkup starting em0.10  
Feb 11 15:58:02  check_reload_status   Linkup starting em0.24  
Feb 11 15:58:02  check_reload_status   Linkup starting em0.20  
Feb 11 15:58:02  check_reload_status   Linkup starting em0.1  
Feb 11 15:58:02  check_reload_status   Linkup starting em0.50  
Feb 11 15:58:02  kernel   em0.30: link state changed to DOWN  
Feb 11 15:58:02  kernel   em0.45: link state changed to DOWN  
Feb 11 15:58:02  kernel   em0.10: link state changed to DOWN  
Feb 11 15:58:02  kernel   em0.24: link state changed to DOWN  
Feb 11 15:58:02  kernel   em0.20: link state changed to DOWN  
Feb 11 15:58:02  kernel   em0.1: link state changed to DOWN  
Feb 11 15:58:02  kernel   em0.50: link state changed to DOWN  
Feb 11 15:58:02  kernel   em0: link state changed to DOWN  
Feb 11 15:58:02  check_reload_status   Linkup starting em0  
Feb 11 15:57:06  tv-router.stv   nginx: 2019/02/11 15:57:06 [error] 36354#100130: *18054 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.0.93, server: , request: "POST /widgets/widgets/thermal_sensors.widget.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.0.254", referrer: "http://192.168.0.254/"  
Feb 11 15:54:16  check_reload_status   Linkup starting em0.30  
Feb 11 15:54:16  check_reload_status   Linkup starting em0.45  
Feb 11 15:54:16  check_reload_status   Linkup starting em0.10  
Feb 11 15:54:16  check_reload_status   Linkup starting em0.24  
Feb 11 15:54:16  check_reload_status   Linkup starting em0.20  
Feb 11 15:54:16  check_reload_status   Linkup starting em0.1  
Feb 11 15:54:16  check_reload_status   Linkup starting em0.50  
Feb 11 15:54:16  kernel   em0.30: link state changed to UP  
Feb 11 15:54:16  kernel   em0.45: link state changed to UP  
Feb 11 15:54:16  kernel   em0.10: link state changed to UP  
Feb 11 15:54:16  kernel   em0.24: link state changed to UP  
Feb 11 15:54:16  kernel   em0.20: link state changed to UP  
Feb 11 15:54:16  kernel   em0.1: link state changed to UP  
Feb 11 15:54:16  kernel   em0.50: link state changed to UP  
Feb 11 15:54:16  kernel   em0: link state changed to UP  
Feb 11 15:54:16  check_reload_status   Linkup starting em0  
Feb 11 15:54:13  check_reload_status   Linkup starting em0.30  
Feb 11 15:54:13  check_reload_status   Linkup starting em0.45  
Feb 11 15:54:13  check_reload_status   Linkup starting em0.10  
Feb 11 15:54:13  check_reload_status   Linkup starting em0.24  
Feb 11 15:54:13  check_reload_status   Linkup starting em0.20  
Feb 11 15:54:13  check_reload_status   Linkup starting em0.1  
Feb 11 15:54:13  check_reload_status   Linkup starting em0.50  
Feb 11 15:54:13  kernel   em0.30: link state changed to DOWN  
Feb 11 15:54:13  kernel   em0.45: link state changed to DOWN  
Feb 11 15:54:13  kernel   em0.10: link state changed to DOWN  
Feb 11 15:54:13  kernel   em0.24: link state changed to DOWN  
Feb 11 15:54:13  kernel   em0.20: link state changed to DOWN  
Feb 11 15:54:13  kernel   em0.1: link state changed to DOWN  
Feb 11 15:54:13  kernel   em0.50: link state changed to DOWN  
Feb 11 15:54:13  kernel   em0: link state changed to DOWN  
Feb 11 15:54:13  check_reload_status   Linkup starting em0  
Feb 11 15:54:05  tv-router.stv   nginx: 2019/02/11 15:54:05 [error] 36354#100130: *18054 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.0.93, server: , request: "POST /widgets/widgets/interfaces.widget.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.0.254", referrer: "http://192.168.0.254/"  
Feb 11 15:54:05  tv-router.stv   nginx: 2019/02/11 15:54:05 [error] 36697#100101: *17998 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.0.93, server: , request: "POST /ifstats.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.0.254", referrer: "http://192.168.0.254/"  
Feb 11 15:51:58  check_reload_status   Linkup starting em0.30  
Feb 11 15:51:58  check_reload_status   Linkup starting em0.45  
Feb 11 15:51:58  check_reload_status   Linkup starting em0.10  
Feb 11 15:51:58  check_reload_status   Linkup starting em0.24  
Feb 11 15:51:58  check_reload_status   Linkup starting em0.20  
Feb 11 15:51:58  check_reload_status   Linkup starting em0.1  
Feb 11 15:51:58  check_reload_status   Linkup starting em0.50  
Feb 11 15:51:58  kernel   em0.30: link state changed to UP  
Feb 11 15:51:58  kernel   em0.45: link state changed to UP  
Feb 11 15:51:58  kernel   em0.10: link state changed to UP  
Feb 11 15:51:58  kernel   em0.24: link state changed to UP  
Feb 11 15:51:58  kernel   em0.20: link state changed to UP  
Feb 11 15:51:58  kernel   em0.1: link state changed to UP  
Feb 11 15:51:58  kernel   em0.50: link state changed to UP  
Feb 11 15:51:58  kernel   em0: link state changed to UP  
Feb 11 15:51:58  check_reload_status   Linkup starting em0  
Feb 11 15:51:55  check_reload_status   Linkup starting em0.30  
Feb 11 15:51:55  check_reload_status   Linkup starting em0.45  
Feb 11 15:51:55  check_reload_status   Linkup starting em0.10  
Feb 11 15:51:55  check_reload_status   Linkup starting em0.24  
Feb 11 15:51:55  check_reload_status   Linkup starting em0.20  
Feb 11 15:51:55  check_reload_status   Linkup starting em0.1  
Feb 11 15:51:55  check_reload_status   Linkup starting em0.50  
Feb 11 15:51:55  kernel   em0.30: link state changed to DOWN  
Feb 11 15:51:55  kernel   em0.45: link state changed to DOWN  
Feb 11 15:51:55  kernel   em0.10: link state changed to DOWN  
Feb 11 15:51:55  kernel   em0.24: link state changed to DOWN  
Feb 11 15:51:55  kernel   em0.20: link state changed to DOWN  
Feb 11 15:51:55  kernel   em0.1: link state changed to DOWN  
Feb 11 15:51:55  kernel   em0.50: link state changed to DOWN  
Feb 11 15:51:55  kernel   em0: link state changed to DOWN  
Feb 11 15:51:55  check_reload_status   Linkup starting em0  
Feb 11 15:51:04  check_reload_status   rc.newwanip starting em0.20  
Feb 11 15:51:04  php-fpm  62046  /rc.linkup: Hotplug event detected for OPT2(opt2) static IP (192.168.20.254 )  
Feb 11 15:51:04  check_reload_status   rc.newwanip starting em0.1  
Feb 11 15:51:04  php-fpm  62046  /rc.linkup: Hotplug event detected for OPT4(opt4) static IP (192.168.0.254 )  
Feb 11 15:51:04  check_reload_status   Reloading filter  
Feb 11 15:51:04  check_reload_status   rc.newwanip starting em0.50  
Feb 11 15:51:04  php-fpm  62046  /rc.linkup: Hotplug event detected for OPT5(opt5) static IP (192.168.50.254 )  
Feb 11 15:51:04  check_reload_status   Reloading filter  
Feb 11 15:51:04  php-fpm  62046  /rc.linkup: Linkup detected on disabled interface...Ignoring  
Feb 11 15:51:03  check_reload_status   Linkup starting em0.30  
Feb 11 15:51:03  check_reload_status   Linkup starting em0.45  
Feb 11 15:51:03  check_reload_status   Linkup starting em0.10  
Feb 11 15:51:03  check_reload_status   Linkup starting em0.24  
Feb 11 15:51:03  check_reload_status   Linkup starting em0.20  
Feb 11 15:51:03  check_reload_status   Linkup starting em0.1  
Feb 11 15:51:03  check_reload_status   Linkup starting em0.50  
Feb 11 15:51:03  kernel   em0.30: link state changed to UP  
Feb 11 15:51:03  kernel   em0.45: link state changed to UP  
Feb 11 15:51:03  kernel   em0.10: link state changed to UP  
Feb 11 15:51:03  kernel   em0.24: link state changed to UP  
Feb 11 15:51:03  kernel   em0.20: link state changed to UP  
Feb 11 15:51:03  kernel   em0.1: link state changed to UP  
Feb 11 15:51:03  kernel   em0.50: link state changed to UP  
Feb 11 15:51:03  kernel   em0: link state changed to UP  
Feb 11 15:51:03  check_reload_status   Linkup starting em0  
Feb 11 15:51:02  xinetd  29845  Reconfigured: new=0 old=1 dropped=0 (services)  
Feb 11 15:51:02  xinetd  29845  readjusting service 6969-udp  
Feb 11 15:51:02  xinetd  29845  Swapping defaults  
Feb 11 15:51:02  xinetd  29845  Starting reconfiguration  
Feb 11 15:51:02  xinetd  29845  Reconfigured: new=0 old=1 dropped=0 (services)  
Feb 11 15:51:02  xinetd  29845  readjusting service 6969-udp  
Feb 11 15:51:02  xinetd  29845  Swapping defaults  
Feb 11 15:51:02  xinetd  29845  Starting reconfiguration  
Feb 11 15:51:01  php-fpm  62046  /rc.linkup: Linkup detected on disabled interface...Ignoring  
Feb 11 15:51:01  php-fpm  62046  /rc.linkup: Hotplug event detected for OPT6(opt6) static IP (192.168.45.254 )  
Feb 11 15:51:01  php-fpm  62046  /rc.linkup: Hotplug event detected for OPT1(opt1) static IP (192.168.10.254 )  
Feb 11 15:51:01  php-fpm  54139  /rc.linkup: Hotplug event detected for OPT2(opt2) static IP (192.168.20.254 )  
Feb 11 15:51:01  check_reload_status   Reloading filter  
Feb 11 15:51:01  php-fpm  62046  /rc.linkup: Hotplug event detected for OPT4(opt4) static IP (192.168.0.254 )  
Feb 11 15:51:01  php-fpm  54139  /rc.linkup: Hotplug event detected for OPT5(opt5) static IP (192.168.50.254 )  
Feb 11 15:51:01  check_reload_status   Reloading filter  
Feb 11 15:51:01  php-fpm  62046  /rc.linkup: Linkup detected on disabled interface...Ignoring  
Feb 11 15:51:00  check_reload_status   Linkup starting em0.30  
Feb 11 15:51:00  check_reload_status   Linkup starting em0.45  
Feb 11 15:51:00  check_reload_status   Linkup starting em0.10  
Feb 11 15:51:00  check_reload_status   Linkup starting em0.24  
Feb 11 15:51:00  check_reload_status   Linkup starting em0.20  
Feb 11 15:51:00  check_reload_status   Linkup starting em0.1  
Feb 11 15:51:00  check_reload_status   Linkup starting em0.50  
Feb 11 15:51:00  check_reload_status   Linkup starting em0  
Feb 11 15:51:00  kernel   em0.30: link state changed to DOWN  
Feb 11 15:51:00  kernel   em0.45: link state changed to DOWN  
Feb 11 15:51:00  kernel   em0.10: link state changed to DOWN  
Feb 11 15:51:00  kernel   em0.24: link state changed to DOWN  
Feb 11 15:51:00  kernel   em0.20: link state changed to DOWN  
Feb 11 15:51:00  kernel   em0.1: link state changed to DOWN  
Feb 11 15:51:00  kernel   em0.50: link state changed to DOWN  
Feb 11 15:51:00  kernel   em0: link state changed to DOWN  
Feb 11 15:50:05  php-fpm  54139  /rc.newwanip: rc.newwanip: on (IP address: 192.168.0.254) (interface: OPT4[opt4]) (real interface: em0.1).  
Feb 11 15:50:05  php-fpm  54139  /rc.newwanip: rc.newwanip: Info: starting on em0.1.  
Feb 11 15:50:05  php-fpm  54139  /rc.newwanip: rc.newwanip: on (IP address: 192.168.45.254) (interface: OPT6[opt6]) (real interface: em0.45).  
Feb 11 15:50:05  php-fpm  54139  /rc.newwanip: rc.newwanip: Info: starting on em0.45.  
Feb 11 15:50:05  xinetd  29845  Reconfigured: new=0 old=1 dropped=0 (services)  
Feb 11 15:50:05  xinetd  29845  readjusting service 6969-udp  
Feb 11 15:50:05  xinetd  29845  Swapping defaults  
Feb 11 15:50:05  xinetd  29845  Starting reconfiguration  
Feb 11 15:50:05  php-fpm  54139  /rc.newwanip: rc.newwanip: on (IP address: 192.168.10.254) (interface: OPT1[opt1]) (real interface: em0.10).  
Feb 11 15:50:05  php-fpm  54139  /rc.newwanip: rc.newwanip: Info: starting on em0.10.  
Feb 11 15:50:05  php-fpm  54139  /rc.newwanip: rc.newwanip: on (IP address: 192.168.50.254) (interface: OPT5[opt5]) (real interface: em0.50).  
Feb 11 15:50:05  php-fpm  54139  /rc.newwanip: rc.newwanip: Info: starting on em0.50.  
Feb 11 15:50:05  check_reload_status   Reloading filter  
Feb 11 15:50:05  php-fpm  62046  /rc.newwanip: rc.newwanip: on (IP address: 192.168.20.254) (interface: OPT2[opt2]) (real interface: em0.20).  
Feb 11 15:50:05  php-fpm  62046  /rc.newwanip: rc.newwanip: Info: starting on em0.20.  
Feb 11 15:50:05  xinetd  29845  Reconfigured: new=0 old=1 dropped=0 (services)  
Feb 11 15:50:05  xinetd  29845  readjusting service 6969-udp  
Feb 11 15:50:05  xinetd  29845  Swapping defaults  
Feb 11 15:50:05  xinetd  29845  Starting reconfiguration  
Feb 11 15:50:04  check_reload_status   rc.newwanip starting em0.1  
Feb 11 15:50:04  php-fpm  62046  /rc.linkup: Linkup detected on disabled interface...Ignoring  
Feb 11 15:50:04  check_reload_status   rc.newwanip starting em0.45  
Feb 11 15:50:04  php-fpm  62046  /rc.linkup: Hotplug event detected for OPT6(opt6) static IP (192.168.45.254 )  
Feb 11 15:50:04  check_reload_status   rc.newwanip starting em0.10  
Feb 11 15:50:04  php-fpm  62046  /rc.linkup: Hotplug event detected for OPT1(opt1) static IP (192.168.10.254 )  
Feb 11 15:50:04  check_reload_status   rc.newwanip starting em0.50  
Feb 11 15:50:04  check_reload_status   Reloading filter  
Feb 11 15:50:04  check_reload_status   rc.newwanip starting em0.20  
Feb 11 15:50:04  php-fpm  87597  /rc.linkup: Hotplug event detected for OPT2(opt2) static IP (192.168.20.254 )  
Feb 11 15:50:04  check_reload_status   Reloading filter  
Feb 11 15:50:04  php-fpm  87597  /rc.linkup: Linkup detected on disabled interface...Ignoring  
Feb 11 15:50:04  php-fpm  62046  /rc.linkup: Hotplug event detected for OPT5(opt5) static IP (192.168.50.254 )  
Feb 11 15:50:04  php-fpm  54139  /rc.linkup: Hotplug event detected for OPT4(opt4) static IP (192.168.0.254 )  
Feb 11 15:50:03  check_reload_status   Linkup starting em0.30  
Feb 11 15:50:03  check_reload_status   Linkup starting em0.45  
Feb 11 15:50:03  check_reload_status   Linkup starting em0.10  
Feb 11 15:50:03  check_reload_status   Linkup starting em0.24  
Feb 11 15:50:03  check_reload_status   Linkup starting em0.20  
Feb 11 15:50:03  check_reload_status   Linkup starting em0.1  
Feb 11 15:50:03  check_reload_status   Linkup starting em0.50  
Feb 11 15:50:03  kernel   em0.30: link state changed to UP

Или подскажите в каком направлении смотреть?

Konstanti

@jktu79

Привет
Нашел тут одну статью англоязычную , тоже у человека отваливалась карта с vlan-ми , потом поднималась ( тоже PF установлен был )
Правда , карта у него была Realtek ( у Вас Intel)

Ему помогло /system/advanced/networking/disable hardware checksum offloading
и перезагрузка

p.s. многие рекомендуют и LRO отключить и TSO тоже

некоторые грешат на MSIX ( из доков на pf)
это как их отключить
0_1549917600720_b7bd8d09-4dde-48c8-b74a-a9532154d32f-image.png

плюс к этому можно в этом же файле установить вот такую переменную
hw.em.msix = 0

hw.em.msix
Enable or disable MSI-X style interrupts. Default 1 (enabled).

https://www.freebsd.org/cgi/man.cgi?em%284%29

Konstanti

@jktu79
Еще хотел задать уточняющий вопрос - проверьте , с чего начинается падение em0 по логам ? Нет ли там таких строчек em0: watchdog timeout -- resetting ? Вы выложили лог , в котором не видно начала проблемы . Если такая строчка есть , также кабель рекомендуют проверить

Jktu79

@konstanti

Mar 5 20:15:19  php-fpm  338  /index.php: Successful login for user 'admin' from: 192.168.0.40  
Mar 5 20:11:49  sshlockout  68462  sshlockout/webConfigurator v3.0 starting up  
Mar 5 20:11:49  sshd  67867  Accepted keyboard-interactive/pam for admin from 192.168.0.40 port 54603 ssh2  
Mar 5 19:13:54  xinetd  40208  Reconfigured: new=0 old=1 dropped=0 (services)  
Mar 5 19:13:54  xinetd  40208  readjusting service 6969-udp  
Mar 5 19:13:54  xinetd  40208  Swapping defaults  
Mar 5 19:13:54  xinetd  40208  Starting reconfiguration  
Mar 5 19:13:54  php-fpm  21757  /rc.newwanip: rc.newwanip: on (IP address: 192.168.0.254) (interface: OPT4[opt4]) (real interface: em0.1).  
Mar 5 19:13:54  php-fpm  21757  /rc.newwanip: rc.newwanip: Info: starting on em0.1.  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: on (IP address: 192.168.24.254) (interface: IKS24[opt7]) (real interface: em0.24).  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: Info: starting on em0.24.  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: on (IP address: 192.168.45.254) (interface: OPT6[opt6]) (real interface: em0.45).  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: Info: starting on em0.45.  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: on (IP address: 192.168.10.254) (interface: OPT1[opt1]) (real interface: em0.10).  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: Info: starting on em0.10.  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: on (IP address: 10.17.92.254) (interface: OPT8[opt8]) (real interface: em0.92).  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: Info: starting on em0.92.  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: on (IP address: 192.168.50.254) (interface: OPT5[opt5]) (real interface: em0.50).  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: Info: starting on em0.50.  
Mar 5 19:13:53  check_reload_status   Reloading filter  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: on (IP address: 192.168.20.254) (interface: OPT2[opt2]) (real interface: em0.20).  
Mar 5 19:13:53  php-fpm  21757  /rc.newwanip: rc.newwanip: Info: starting on em0.20.  
Mar 5 19:13:53  xinetd  40208  Reconfigured: new=0 old=1 dropped=0 (services)  
Mar 5 19:13:53  xinetd  40208  readjusting service 6969-udp  
Mar 5 19:13:53  xinetd  40208  Swapping defaults  
Mar 5 19:13:53  xinetd  40208  Starting reconfiguration  
Mar 5 19:13:53  check_reload_status   rc.newwanip starting em0.1  
Mar 5 19:13:52  check_reload_status   rc.newwanip starting em0.24  
Mar 5 19:13:52  php-fpm  338  /rc.linkup: Linkup detected on disabled interface...Ignoring  
Mar 5 19:13:52  check_reload_status   rc.newwanip starting em0.45  
Mar 5 19:13:52  php-fpm  338  /rc.linkup: Hotplug event detected for OPT6(opt6) static IP (192.168.45.254 )  
Mar 5 19:13:52  check_reload_status   rc.newwanip starting em0.10  
Mar 5 19:13:52  php-fpm  338  /rc.linkup: Hotplug event detected for OPT1(opt1) static IP (192.168.10.254 )  
Mar 5 19:13:52  check_reload_status   rc.newwanip starting em0.92  
Mar 5 19:13:52  php-fpm  339  /rc.linkup: Hotplug event detected for IKS24(opt7) static IP (192.168.24.254 )  
Mar 5 19:13:52  php-fpm  338  /rc.linkup: Hotplug event detected for OPT8(opt8) static IP (10.17.92.254 )  
Mar 5 19:13:52  check_reload_status   rc.newwanip starting em0.50  
Mar 5 19:13:52  check_reload_status   Reloading filter  
Mar 5 19:13:52  check_reload_status   rc.newwanip starting em0.20  
Mar 5 19:13:52  php-fpm  338  /rc.linkup: Hotplug event detected for OPT2(opt2) static IP (192.168.20.254 )  
Mar 5 19:13:52  check_reload_status   Reloading filter  
Mar 5 19:13:52  php-fpm  338  /rc.linkup: Linkup detected on disabled interface...Ignoring  
Mar 5 19:13:52  php-fpm  339  /rc.linkup: Hotplug event detected for OPT5(opt5) static IP (192.168.50.254 )  
Mar 5 19:13:52  php-fpm  21757  /rc.linkup: Hotplug event detected for OPT4(opt4) static IP (192.168.0.254 )  
Mar 5 19:13:51  check_reload_status   Linkup starting em0.30  
Mar 5 19:13:51  check_reload_status   Linkup starting em0.45  
Mar 5 19:13:51  check_reload_status   Linkup starting em0.10  
Mar 5 19:13:51  check_reload_status   Linkup starting em0.24  
Mar 5 19:13:51  check_reload_status   Linkup starting em0.92  
Mar 5 19:13:51  check_reload_status   Linkup starting em0.20  
Mar 5 19:13:51  check_reload_status   Linkup starting em0.1  
Mar 5 19:13:51  check_reload_status   Linkup starting em0.50  
Mar 5 19:13:51  kernel   em0.30: link state changed to UP  
Mar 5 19:13:51  kernel   em0.45: link state changed to UP  
Mar 5 19:13:51  kernel   em0.10: link state changed to UP  
Mar 5 19:13:51  kernel   em0.24: link state changed to UP  
Mar 5 19:13:51  kernel   em0.92: link state changed to UP  
Mar 5 19:13:51  kernel   em0.20: link state changed to UP  
Mar 5 19:13:51  kernel   em0.1: link state changed to UP  
Mar 5 19:13:51  kernel   em0.50: link state changed to UP  
Mar 5 19:13:51  kernel   em0: link state changed to UP  
Mar 5 19:13:51  check_reload_status   Linkup starting em0  
Mar 5 19:13:51  xinetd  40208  Reconfigured: new=0 old=1 dropped=0 (services)  
Mar 5 19:13:51  xinetd  40208  readjusting service 6969-udp  
Mar 5 19:13:51  xinetd  40208  Swapping defaults  
Mar 5 19:13:51  xinetd  40208  Starting reconfiguration  
Mar 5 19:13:50  xinetd  40208  Reconfigured: new=0 old=1 dropped=0 (services)  
Mar 5 19:13:50  xinetd  40208  readjusting service 6969-udp  
Mar 5 19:13:50  xinetd  40208  Swapping defaults  
Mar 5 19:13:50  xinetd  40208  Starting reconfiguration  
Mar 5 19:13:49  php-fpm  339  /rc.linkup: Linkup detected on disabled interface...Ignoring  
Mar 5 19:13:49  php-fpm  339  /rc.linkup: Hotplug event detected for OPT6(opt6) static IP (192.168.45.254 )  
Mar 5 19:13:49  php-fpm  339  /rc.linkup: Hotplug event detected for OPT1(opt1) static IP (192.168.10.254 )  
Mar 5 19:13:49  php-fpm  21757  /rc.linkup: Hotplug event detected for IKS24(opt7) static IP (192.168.24.254 )  
Mar 5 19:13:49  php-fpm  339  /rc.linkup: Hotplug event detected for OPT8(opt8) static IP (10.17.92.254 )  
Mar 5 19:13:49  check_reload_status   Reloading filter  
Mar 5 19:13:49  php-fpm  21757  /rc.linkup: Hotplug event detected for OPT2(opt2) static IP (192.168.20.254 )  
Mar 5 19:13:49  php-fpm  338  /rc.linkup: Hotplug event detected for OPT4(opt4) static IP (192.168.0.254 )  
Mar 5 19:13:49  check_reload_status   Reloading filter  
Mar 5 19:13:49  php-fpm  21757  /rc.linkup: Linkup detected on disabled interface...Ignoring  
Mar 5 19:13:49  php-fpm  339  /rc.linkup: Hotplug event detected for OPT5(opt5) static IP (192.168.50.254 )  
Mar 5 19:13:48  check_reload_status   Linkup starting em0.30  
Mar 5 19:13:48  check_reload_status   Linkup starting em0.45  
Mar 5 19:13:48  check_reload_status   Linkup starting em0.10  
Mar 5 19:13:48  check_reload_status   Linkup starting em0.24  
Mar 5 19:13:48  check_reload_status   Linkup starting em0.92  
Mar 5 19:13:48  check_reload_status   Linkup starting em0.20  
Mar 5 19:13:48  check_reload_status   Linkup starting em0.1  
Mar 5 19:13:48  check_reload_status   Linkup starting em0.50  
Mar 5 19:13:48  kernel   em0.30: link state changed to DOWN  
Mar 5 19:13:48  kernel   em0.45: link state changed to DOWN  
Mar 5 19:13:48  kernel   em0.10: link state changed to DOWN  
Mar 5 19:13:48  kernel   em0.24: link state changed to DOWN  
Mar 5 19:13:48  kernel   em0.92: link state changed to DOWN  
Mar 5 19:13:48  kernel   em0.20: link state changed to DOWN  
Mar 5 19:13:48  kernel   em0.1: link state changed to DOWN  
Mar 5 19:13:48  kernel   em0.50: link state changed to DOWN  
Mar 5 19:13:48  kernel   em0: link state changed to DOWN  
Mar 5 19:13:48  check_reload_status   Linkup starting em0  
Mar 5 18:18:22  tv-router.stv   nginx: 2019/03/05 18:18:22 [error] 46881#100111: send() failed (54: Connection reset by peer)  
Mar 5 18:17:11  xinetd  40208  Reconfigured: new=0 old=1 dropped=0 (services)  
Mar 5 18:17:11  xinetd  40208  readjusting service 6969-udp  
Mar 5 18:17:11  xinetd  40208  Swapping defaults  
Mar 5 18:17:11  xinetd  40208  Starting reconfiguration  
Mar 5 18:17:10  check_reload_status   Reloading filter  
Mar 5 18:17:10  check_reload_status   Restarting OpenVPN tunnels/interfaces  
Mar 5 18:17:10  check_reload_status   Restarting ipsec tunnels  
Mar 5 18:17:10  check_reload_status   updating dyndns GW_WAN  
Mar 5 18:17:10  rc.gateway_alarm  23320  >>> Gateway alarm: GW_WAN (Addr:78.30.192.1 Alarm:0 RTT:13945ms RTTsd:38286ms Loss:12%)  
Mar 5 18:17:02  tv-router.stv   nginx: 2019/03/05 18:17:02 [error] 46809#100139: send() failed (54: Connection reset by peer)  
Mar 5 18:16:54  kernel   done.  
Mar 5 18:16:54  php-fpm  21757  /rc.start_packages: Restarting/Starting all packages.  
Mar 5 18:16:54  kernel   done.  
Mar 5 18:16:54  syslogd   kernel boot file is /boot/kernel/kernel  
Mar 5 18:16:54  syslogd   exiting on signal 15  
Mar 5 18:16:54  kernel   done.  
Mar 5 18:16:54  syslogd   Logging subprocess 73774 (exec /usr/local/sbin/sshlockout_pf 15) exited due to signal 15.  
Mar 5 18:16:54  php-cgi   rc.bootup: Creating rrd update script  
Mar 5 18:16:54  php-cgi   rc.bootup: The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/system-memory.rrd N:U:U:U:U:U' returned exit code '1', the output was 'ERROR: This RRD was created on another architecture'  
Mar 5 18:16:54  php-cgi   rc.bootup: The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/system-processor.rrd N:U:U:U:U:U' returned exit code '1', the output was 'ERROR: This RRD was created on another architecture'  
Mar 5 18:16:54  php-cgi   rc.bootup: The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/system-states.rrd N:U:U:U:U:U' returned exit code '1', the output was 'ERROR: This RRD was created on another architecture'  
Mar 5 18:16:54  php-cgi   rc.bootup: The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/ipsec-packets.rrd N:U:U:U:U:U:U:U:U' returned exit code '1', the output was 'ERROR: This RRD was created on another architecture'  
Mar 5 18:16:54  php-cgi   rc.bootup: The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/ipsec-traffic.rrd N:U:U:U:U:U:U:U:U' returned exit code '1', the output was 'ERROR: This RRD was created on another architecture'  
Mar 5 18:16:54  snmpd  22895  disk_OS_get_disks: adding device 'ada0' to device list  
Mar 5 18:16:54  snmpd  22895  disk_OS_get_disks: adding device 'cd0' to device list  
Mar 5 18:16:53  php-cgi   rc.bootup: The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/wan-packets.rrd N:U:U:U:U:U:U:U:U' returned exit code '1', the output was 'ERROR: This RRD was created on another architecture'  
Mar 5 18:16:53  php-cgi   rc.bootup: The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/wan-traffic.rrd N:U:U:U:U:U:U:U:U' returned exit code '1', the output was 'ERROR: This RRD was created on another architecture'  
Mar 5 18:16:49  xinetd  40208  Reconfigured: new=0 old=1 dropped=0 (services)  
Mar 5 18:16:49  xinetd  40208  readjusting service 6969-udp  
Mar 5 18:16:49  xinetd  40208  Swapping defaults  
Mar 5 18:16:49  xinetd  40208  Starting reconfiguration  
Mar 5 18:16:49  kernel   re0: link state changed to UP  
Mar 5 18:16:49  check_reload_status   Linkup starting re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   ..  
Mar 5 18:16:49  xinetd  40208  Reconfigured: new=0 old=1 dropped=0 (services)  
Mar 5 18:16:49  xinetd  40208  readjusting service 6969-udp  
Mar 5 18:16:49  xinetd  40208  Swapping defaults  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  xinetd  40208  Starting reconfiguration  
Mar 5 18:16:49  check_reload_status   Updating all dyndns  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0  
Mar 5 18:16:49  kernel   arpresolve: can't allocate llinfo for 78.30.192.1 on re0

Konstanti

@jktu79 said

Из документации ARP
arpresolve: can't allocate llinfo for %d.%d.%d.%d: The route for the ref-
erenced host points to a device upon which ARP is required, but ARP was
unable to allocate a routing table entry in which to store the host's MAC
address. This usually points to a misconfigured routing table. It can
also occur if the kernel cannot allocate memory.

Возможно , уши торчат отсюда
Или маршрутизация или что-то подъедает память

Jktu79

@konstanti
Спасибо за ответ,

Подскажите где проанализировать, куда в первую очередь обратить внимание?

Konstanti

@jktu79
Сложно сказать . Судя по тому , что пауза между постами 23 дня , то получается , что в этот период все работало нормально .
Я бы мониторил память
Например , top или vmstat -m
Посмотрите , нет ли ошибок настройке маршрутов , масок подсетей и тд и тп
Почитайте про эту ошибку в инете , постов много
Есть ли какие-нибудь запущенные нестандартные приложения ???
Что такое 78.30.192.1 ???