Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Router Locking Up (maybe due to excessive lan traffic?)

    Scheduled Pinned Locked Moved General pfSense Questions
    64 Posts 6 Posters 5.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • X
      Ximulate @Ximulate
      last edited by 

      EFI systbl not available
firmware: 'iwm3160fw' version 0: 918268 bytes loaded at 0xffffffff81b196f6
module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff807652e0, 0) error 1
iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
firmware: 'rt2860fw' version 0: 8192 bytes loaded at 0xffffffff82acc5ce
firmware: 'rt2661fw' version 0: 8192 bytes loaded at 0xffffffff82aca5ce
firmware: 'rt2561sfw' version 0: 8192 bytes loaded at 0xffffffff82ac85ce
firmware: 'rt2561fw' version 0: 8192 bytes loaded at 0xffffffff82ac65ce
module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff80765230, 0) error 1
iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80765180, 0) error 1
iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80747700, 0) error 1
ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff80747650, 0) error 1
ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff807475a0, 0) error 1
ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
firmware: 'runfw' version 0: 8192 bytes loaded at 0xffffffff82b18392
firmware: 'rsu-rtl8712fw' version 120: 122328 bytes loaded at 0xffffffff82afa5ba
firmware: 't6fw' version 0: 730624 bytes loaded at 0xffffffff819491f2
firmware: 't6fw_cfg_uwire' version 0: 22693 bytes loaded at 0xffffffff8194394d
firmware: 't6fw_cfg' version 0: 5960 bytes loaded at 0xffffffff81942205
firmware: 't5fw' version 0: 678912 bytes loaded at 0xffffffff8189c605
firmware: 't5fw_cfg_uwire' version 0: 21859 bytes loaded at 0xffffffff818970a2
firmware: 't5fw_cfg' version 0: 5902 bytes loaded at 0xffffffff81895994
wlan: <802.11 Link Layer>
firmware: 't4fw' version 0: 570880 bytes loaded at 0xffffffff8180a394
firmware: 't4fw_cfg_uwire' version 0: 21433 bytes loaded at 0xffffffff81804fdb
firmware: 't4fw_cfg' version 0: 4523 bytes loaded at 0xffffffff81803e30
firmware: 'iwn6050fw' version 0: 469780 bytes loaded at 0xffffffff82a3bef6
firmware: 'iwn6000g2bfw' version 0: 679436 bytes loaded at 0xffffffff829960ea
firmware: 'iwn6000g2afw' version 0: 677296 bytes loaded at 0xffffffff828f0b3a
firmware: 'iwn6000fw' version 0: 454608 bytes loaded at 0xffffffff82881b6a
firmware: 'iwn5150fw' version 0: 337400 bytes loaded at 0xffffffff8282f572
firmware: 'iwn5000fw' version 0: 340696 bytes loaded at 0xffffffff827dc29a
wlan: mac acl policy registered
firmware: 'mxge_rss_ethp_z8e' version 544468: 152837 bytes loaded at 0xffffffff82cafd20
firmware: 'mxge_rss_eth_z8e' version 534724: 151912 bytes loaded at 0xffffffff82c8ab00
firmware: 'mxge_ethp_z8e' version 387604: 121317 bytes loaded at 0xffffffff82c6d060
firmware: 'mxge_eth_z8e' version 377284: 120629 bytes loaded at 0xffffffff82c4f870
firmware: 'mwlboot' version 0: 2280 bytes loaded at 0xffffffff82ac5ce6
firmware: 'mw88W8363fw' version 0: 94940 bytes loaded at 0xffffffff82aaea0a
feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=2 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25
snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024]
firmware: 'iwn4965fw' version 0: 187972 bytes loaded at 0xffffffff827ae456
firmware: 'iwn2030fw' version 0: 707392 bytes loaded at 0xffffffff82701916
firmware: 'iwn2000fw' version 0: 695876 bytes loaded at 0xffffffff82657ad2
firmware: 'iwn135fw' version 0: 701228 bytes loaded at 0xffffffff825ac7a6
firmware: 'iwn105fw' version 0: 689680 bytes loaded at 0xffffffff82504196
firmware: 'iwn100fw' version 0: 337572 bytes loaded at 0xffffffff824b1af2
firmware: 'iwn1000fw' version 0: 337520 bytes loaded at 0xffffffff8245f482
firmware: 'iwm8265fw' version 0: 1811984 bytes loaded at 0xffffffff822a4e72
firmware: 'rtwn-rtl8821aufw' version 111: 27804 bytes loaded at 0xffffffff82af391e
firmware: 'rtwn-rtl8812aufw' version 111: 32060 bytes loaded at 0xffffffff82aebbe2
firmware: 'rtwn-rtl8192eufw' version 111: 31818 bytes loaded at 0xffffffff82ae3f98
firmware: 'rtwn-rtl8192cfwU' version 111: 16126 bytes loaded at 0xffffffff82ae009a
firmware: 'rtwn-rtl8192cfwT' version 111: 16126 bytes loaded at 0xffffffff82adc19c
firmware: 'rtwn-rtl8192cfwE_B' version 111: 16332 bytes loaded at 0xffffffff82ad81d0
firmware: 'wpifw' version 153229: 150100 bytes loaded at 0xffffffff82b1a392
firmware: 'rtwn-rtl8192cfwE' version 111: 14818 bytes loaded at 0xffffffff82ad47ee
firmware: 'rtwn-rtl8188eufw' version 111: 13904 bytes loaded at 0xffffffff82ad119e
firmware: 'rtwn-rtl8188eefw' version 111: 11216 bytes loaded at 0xffffffff82ace5ce
firmware: 'iwm8000Cfw' version 0: 2351636 bytes loaded at 0xffffffff82066c5e
firmware: 'iwm7265Dfw' version 0: 1383604 bytes loaded at 0xffffffff81f14faa
firmware: 'iwm7265fw' version 0: 1180412 bytes loaded at 0xffffffff81df4cae
firmware: 'iwm7260fw' version 0: 1049340 bytes loaded at 0xffffffff81cf49b2
firmware: 'iwm3168fw' version 0: 1028032 bytes loaded at 0xffffffff81bf99f2
random: entropy device external interface
TCP_ratelimit: Is now initialized
SMP: passed TSC synchronization test
elvt0: 0x00010000 elvt1: 0x00010000 elvt2: 0x00010000 elvt3: 0x00010000
AMD ext features: 0x00040007
timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
ID: 0x01000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff
cpu1 AP:
SMP: AP CPU #1 Launched!
elvt0: 0x00010000 elvt1: 0x00010000 elvt2: 0x00010000 elvt3: 0x00010000
AMD ext features: 0x00040007
timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
ID: 0x03000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff
cpu3 AP:
SMP: AP CPU #3 Launched!
elvt0: 0x00010000 elvt1: 0x00010000 elvt2: 0x00010000 elvt3: 0x00010000
AMD ext features: 0x00040007
timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
ID: 0x02000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff
cpu2 AP:
SMP: AP CPU #2 Launched!
elvt0: 0x00010000 elvt1: 0x000000f2 elvt2: 0x00010000 elvt3: 0x00010000
AMD ext features: 0x00040007
timer: 0x000300ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff
cpu0 BSP:
ioapic1 <Version 2.1> irqs 24-55
ioapic0 <Version 2.1> irqs 0-23
lapic: LINT1 polarity: high
lapic: LINT1 trigger: edge
lapic: Routing NMI -> LINT1
ioapic0: intpin 9 polarity: low
ioapic0: intpin 9 trigger: level
MADT: Interrupt override: source 9, irq 9
ioapic0: Routing IRQ 0 -> intpin 2
MADT: Interrupt override: source 0, irq 2
ioapic1: ver 0x21 maxredir 0x1f
ioapic1: MADT APIC ID 5 != hw id 0
MADT: Found IO APIC ID 5, Interrupt 24 at 0xfec20000
ioapic0: Routing external 8259A's -> intpin 0
ioapic0: ver 0x21 maxredir 0x17
MADT: Found IO APIC ID 4, Interrupt 0 at 0xfec00000
ACPI: HPET 0x00000000DFFBEB10 000038 (v01 CORE COREBOOT 00000000 CORE 00000000)
ACPI: SSDT 0x00000000DFFBE340 0007C8 (v01 AMD AGESA 00000001 AMD 00000001)
ACPI: SSDT 0x00000000DFFB9A90 0048A6 (v02 AMD AGESA 00000002 MSFT 04000000)
ACPI: HEST 0x00000000DFFB98C0 0001D0 (v01 CORE COREBOOT 00000000 CORE 00000000)
ACPI: APIC 0x00000000DFFB9840 00007E (v01 CORE COREBOOT 00000000 CORE 00000000)
ACPI: SSDT 0x00000000DFFB97F0 000045 (v02 CORE COREBOOT 0000002A CORE 0000002A)
ACPI: FACS 0x00000000DFFB8210 000040
ACPI: DSDT 0x00000000DFFB8250 001496 (v02 AMD COREBOOT 00010001 INTL 20140114)
ACPI: FACP 0x00000000DFFB96F0 0000F4 (v04 CORE COREBOOT 00000000 CORE 00000000)
ACPI: XSDT 0x00000000DFFB80E0 00005C (v01 CORE COREBOOT 00000000 CORE 00000000)
ACPI: RSDP 0x00000000000F4840 000024 (v02 CORE )
ULE: setup cpu 3
ULE: setup cpu 2
ULE: setup cpu 1
ULE: setup cpu 0
hostuuid: using 00000000-0000-0000-0000-000000000000
VIMAGE (virtualized network stack) enabled
random: unblocking device.
random: no platform bootloader entropy
random: read 4096 bytes from preloaded cache
x86bios: ROM 0x0a0000-0x0fefff at 0xfffff800000a0000
x86bios: EBDA 0x09f000-0x09ffff at 0xfffff8000009f000
x86bios: SSEG 0x09e000-0x09efff at 0xfffffe0008531000
x86bios: IVT 0x000000-0x0004ff at 0xfffff80000000000
Pentium Pro MTRR support enabled
APIC: CPU 3 has ACPI ID 3
APIC: CPU 2 has ACPI ID 2
APIC: CPU 1 has ACPI ID 1
APIC: CPU 0 has ACPI ID 0
CPU3 (AP): APIC ID: 3
Core HW ID = 3
CPU2 (AP): APIC ID: 2
Core HW ID = 2
CPU1 (AP): APIC ID: 1
Core HW ID = 1
CPU0 (BSP): APIC ID: 0
Core HW ID = 0
Package HW ID = 0
FreeBSD/SMP: 1 package(s) x 4 core(s)
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
INTR: Adding local APIC 3 as a target
INTR: Adding local APIC 2 as a target
INTR: Adding local APIC 1 as a target
AP boot address 0x9e000
Core ID shift: 0
L1 cache ID shift: 0
L2 cache ID shift: 2
Package ID shift: 3
ACPI APIC Table: <CORE COREBOOT>
LAPIC: ipi_wait() us multiplier 5 (r 176182 tsc 998139400)
Event timer "LAPIC" quality 100
SMP: Added CPU 3 (AP)
MADT: Found CPU APIC ID 3 ACPI ID 3: enabled
SMP: Added CPU 2 (AP)
MADT: Found CPU APIC ID 2 ACPI ID 2: enabled
SMP: Added CPU 1 (AP)
MADT: Found CPU APIC ID 1 ACPI ID 1: enabled
SMP: Added CPU 0 (AP)
MADT: Found CPU APIC ID 0 ACPI ID 0: enabled
avail memory = 4071268352 (3882 MB)
0x0000000100000000 - 0x000000011efe7fff, 519995392 bytes (126952 pages)
0x00000000dfe00000 - 0x00000000dff8cfff, 1626112 bytes (397 pages)
0x0000000004002000 - 0x00000000d87a5fff, 3564781568 bytes (870308 pages)
0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages)
0x0000000000010000 - 0x000000000009efff, 585728 bytes (143 pages)
Physical memory chunk(s):
real memory = 4815060992 (4592 MB)
L2 unified cache: 2048 kbytes, 64 bytes/line, 1 lines/tag, 16-way associative
L2 4KB instruction TLB: 512 entries, 4-way associative
L2 4KB data TLB: 512 entries, 4-way associative
L2 2MB instruction TLB: 0 entries, 2-way associative
L2 2MB data TLB: 256 entries, 2-way associative
L1 instruction cache: 32 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative
L1 data cache: 32 kbytes, 64 bytes/line, 1 lines/tag, 8-way associative
L1 4KB instruction TLB: 32 entries, fully associative
L1 4KB data TLB: 40 entries, fully associative
L1 2MB instruction TLB: 8 entries, fully associative
L1 2MB data TLB: 8 entries, fully associative
TSC: P-state invariant, performance statistics
Revision=1, ASIDs=8
SVM: Features=0x1cdf<NP,LbrVirt,SVML,NRIPS,TscRateMsr,FlushByAsid,DecodeAssist,PauseFilter,EncryptedMcodePatch,PauseFilterThreshold>
XSAVE Features=0x1<XSAVEOPT>
Structured Extended Features=0x8<BMI1>
AMD Features2=0x1d4037ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT,Topology,PNXC,DBE,PTSC,PL2I>
AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
Features2=0x3ed8220b<SSE3,PCLMULQDQ,MON,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C>
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Origin="AuthenticAMD" Id=0x730f01 Family=0x16 Model=0x30 Stepping=1
CPU: AMD GX-412TC SOC (998.14-MHz K8-class CPU)
Early TSC frequency 998139400Hz calibrated from 8254 PIT
Preloaded TSLOG data "TSLOG" at 0xffffffff83cd9b80.
Preloaded boot_entropy_cache "/boot/entropy" at 0xffffffff83cd9b28.
Preloaded elf obj module "/boot/kernel/zfs.ko" at 0xffffffff83cd92c0.
Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff83cd0000.
VT(vga): resolution 640x480
pmap: large map 8 PML4 slots (4096 GB)
PPIM 0: PA=0xa0000, VA=0xffffffff84410000, size=0x10000, mode=0
FreeBSD clang version 16.0.6 (https://github.com/llvm/llvm-project.git llvmorg-16.0.6-0-g7cbf1a259152)
root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/obj/amd64/StdASW5b/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/sources/FreeBSD-src-RELENG_2_7_2/amd64.amd64/sys/pfSense amd64
FreeBSD 14.0-CURRENT amd64 1400094 #1 RELENG_2_7_2-n255948-8d2b56da39c: Wed Dec 6 20:45:47 UTC 2023
FreeBSD is a registered trademark of The FreeBSD Foundation.
The Regents of the University of California. All rights reserved.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Copyright (c) 1992-2023 The FreeBSD Project.
      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Yeah that all looks normal. I wouldn't expect it to show anything until stops responding. If it's a driver/hardware issue it might show additional output at that point.

        X 1 Reply Last reply Reply Quote 0
        • X
          Ximulate @stephenw10
          last edited by

          @stephenw10 I thought I nipped this issue.. running will for several days until just now.

          I have DNS redirection configured according to this recipe:
          https://docs.netgate.com/pfsense/en/latest/recipes/dns-redirect.html
          except I noticed I had set Destination: !This Firewall instead of LAN Address. I'm not sure the difference, but I changed it to LAN Address several days ago.

          Just now, the network failed again. GUI via browser reported 504 Gateway Timeout. I was able to RDP to my laptop which is connected to the pfSense console via serial. I pinged:
          8.8.8.8 - OK
          9.9.9.9 - OK
          1.1.1.1 - 100% packet lose
          10.0.11.11 - OK

          Via Console menu, I tried restarting PHP-FPM, and I was then able to open the GUI in the browser. Only issue I saw was the primary gateway offline. Via console, I tried playback restartallwan... waited probably about 5 mins but no response from the console. Went back to the GUI & rebooted. The router didn't come up. Had to power cycle.

          Excerpt from the System log shows:

          Mar 13 21:47:07 router kernel: module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff80747650, 0) error 1

          For context...

          Mar 13 21:47:07 router kernel: ---<<BOOT>>---
Mar 13 21:47:07 router kernel: Copyright (c) 1992-2023 The FreeBSD Project.
Mar 13 21:47:07 router kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Mar 13 21:47:07 router kernel: 	The Regents of the University of California. All rights reserved.
Mar 13 21:47:07 router kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
Mar 13 21:47:07 router kernel: FreeBSD 14.0-CURRENT amd64 1400094 #1 RELENG_2_7_2-n255948-8d2b56da39c: Wed Dec  6 20:45:47 UTC 2023
Mar 13 21:47:07 router kernel:     root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/obj/amd64/StdASW5b/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/sources/FreeBSD-src-RELENG_2_7_2/amd64.amd64/sys/pfSense amd64
Mar 13 21:47:07 router kernel: FreeBSD clang version 16.0.6 (https://github.com/llvm/llvm-project.git llvmorg-16.0.6-0-g7cbf1a259152)
Mar 13 21:47:07 router kernel: VT(vga): resolution 640x480
Mar 13 21:47:07 router kernel: CPU: AMD GX-412TC SOC                                (998.28-MHz K8-class CPU)
Mar 13 21:47:07 router kernel:   Origin="AuthenticAMD"  Id=0x730f01  Family=0x16  Model=0x30  Stepping=1
Mar 13 21:47:07 router kernel:   Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Mar 13 21:47:07 router kernel:   Features2=0x3ed8220b<SSE3,PCLMULQDQ,MON,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C>
Mar 13 21:47:07 router kernel:   AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
Mar 13 21:47:07 router kernel:   AMD Features2=0x1d4037ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT,Topology,PNXC,DBE,PTSC,PL2I>
Mar 13 21:47:07 router kernel:   Structured Extended Features=0x8<BMI1>
Mar 13 21:47:07 router kernel:   XSAVE Features=0x1<XSAVEOPT>
Mar 13 21:47:07 router kernel:   SVM: NP,NRIP,AFlush,DAssist,NAsids=8
Mar 13 21:47:07 router kernel:   TSC: P-state invariant, performance statistics
Mar 13 21:47:07 router kernel: real memory  = 4815060992 (4592 MB)
Mar 13 21:47:07 router kernel: avail memory = 4071268352 (3882 MB)
Mar 13 21:47:07 router kernel: Event timer "LAPIC" quality 100
Mar 13 21:47:07 router kernel: ACPI APIC Table: <CORE   COREBOOT>
Mar 13 21:47:07 router kernel: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
Mar 13 21:47:07 router kernel: FreeBSD/SMP: 1 package(s) x 4 core(s)
Mar 13 21:47:07 router kernel: random: unblocking device.
Mar 13 21:47:07 router kernel: ioapic1: MADT APIC ID 5 != hw id 0
Mar 13 21:47:07 router kernel: ioapic0 <Version 2.1> irqs 0-23
Mar 13 21:47:07 router kernel: ioapic1 <Version 2.1> irqs 24-55
Mar 13 21:47:07 router kernel: Launching APs: 2 1 3
Mar 13 21:47:07 router kernel: TCP_ratelimit: Is now initialized
Mar 13 21:47:07 router kernel: random: entropy device external interface
Mar 13 21:47:07 router kernel: wlan: mac acl policy registered
Mar 13 21:47:07 router kernel: ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
Mar 13 21:47:07 router kernel: ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
Mar 13 21:47:07 router kernel: module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff807475a0, 0) error 1
Mar 13 21:47:07 router kernel: ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
Mar 13 21:47:07 router kernel: ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
Mar 13 21:47:07 router kernel: module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff80747650, 0) error 1
Mar 13 21:47:07 router kernel: ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
Mar 13 21:47:07 router kernel: ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
Mar 13 21:47:07 router kernel: module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80747700, 0) error 1
Mar 13 21:47:07 router kernel: iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
Mar 13 21:47:07 router kernel: iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
Mar 13 21:47:07 router kernel: module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80765180, 0) error 1
Mar 13 21:47:07 router kernel: iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
Mar 13 21:47:07 router kernel: iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
Mar 13 21:47:07 router kernel: module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff80765230, 0) error 1
Mar 13 21:47:07 router kernel: iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
Mar 13 21:47:07 router kernel: iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
Mar 13 21:47:07 router kernel: module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff807652e0, 0) error 1
Mar 13 21:47:07 router kernel: kbd0 at kbdmux0
Mar 13 21:47:07 router kernel: WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 14.0.
Mar 13 21:47:07 router kernel: netgate0: <Netgate APU>

          System log before reboot has a lot of:

          Mar 13 20:55:54 router kernel: arpresolve: can't allocate llinfo for 192.0.0.1 on igb1
Mar 13 20:56:04 router kernel: arpresolve: can't allocate llinfo for 192.0.0.1 on igb1

          192.0.0.1 is the IPv4 IP from T-Mobile cellular I use for backup. For a deeper dive, see https://wirelessjoint.com/viewtopic.php?t=4191

          Then, in resolver log:

          Mar 13 20:25:28 router unbound[15698]: [15698:2] notice: ssl handshake failed 10.111.11.118 port 54946
Mar 13 20:26:50 router unbound[15698]: [15698:3] error: ssl handshake failed crypto error:0A000416:SSL routines::sslv3 alert certificate unknown
Mar 13 20:26:50 router unbound[15698]: [15698:3] notice: ssl handshake failed 10.111.11.115 port 61484
Mar 13 20:52:15 router unbound[15698]: [15698:0] info: service stopped (unbound 1.18.0).
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: server stats for thread 0: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: server stats for thread 1: 4 queries, 4 answers from cache, 0 recursions, 3 prefetch, 0 rejected by ip ratelimiting
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: server stats for thread 1: requestlist max 0 avg 0 exceeded 0 jostled 0
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: server stats for thread 2: 0 queries, 0 answers from cache, 0 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: server stats for thread 2: requestlist max 0 avg 0 exceeded 0 jostled 0
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: server stats for thread 3: 3 queries, 2 answers from cache, 1 recursions, 1 prefetch, 0 rejected by ip ratelimiting
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: server stats for thread 3: requestlist max 1 avg 0.5 exceeded 0 jostled 0
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: average recursion processing time 0.042078 sec
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: histogram of recursion processing times
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: [25%]=0 median[50%]=0 [75%]=0
Mar 13 20:52:19 router unbound[15698]: [15698:0] info: lower(secs) upper(secs) recursions
Mar 13 20:52:19 router unbound[15698]: [15698:0] info:    0.032768    0.065536 1
Mar 13 20:53:04 router unbound[61026]: [61026:0] notice: init module 0: iterator
Mar 13 20:53:04 router unbound[61026]: [61026:0] info: start of service (unbound 1.18.0).
Mar 13 20:53:32 router unbound[61026]: [61026:0] info: service stopped (unbound 1.18.0).
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: server stats for thread 0: 7 queries, 1 answers from cache, 6 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: server stats for thread 0: requestlist max 1 avg 0.333333 exceeded 0 jostled 0
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: average recursion processing time 0.153774 sec
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: histogram of recursion processing times
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: [25%]=0.028672 median[50%]=0.174763 [75%]=0.240299
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: lower(secs) upper(secs) recursions
Mar 13 20:53:36 router unbound[61026]: [61026:0] info:    0.016384    0.032768 2
Mar 13 20:53:36 router unbound[61026]: [61026:0] info:    0.131072    0.262144 3
Mar 13 20:53:36 router unbound[61026]: [61026:0] info:    0.262144    0.524288 1
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: server stats for thread 1: 35 queries, 12 answers from cache, 23 recursions, 0 prefetch, 0 rejected by ip ratelimiting
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: server stats for thread 1: requestlist max 5 avg 1.13043 exceeded 0 jostled 0
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: average recursion processing time 0.232330 sec
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: histogram of recursion processing times
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: [25%]=0.0251611 median[50%]=0.114688 [75%]=0.199885
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: lower(secs) upper(secs) recursions
Mar 13 20:53:36 router unbound[61026]: [61026:0] info:    0.008192    0.016384 2
Mar 13 20:53:36 router unbound[61026]: [61026:0] info:    0.016384    0.032768 7
Mar 13 20:53:36 router unbound[61026]: [61026:0] info:    0.032768    0.065536 1
Mar 13 20:53:36 router unbound[61026]: [61026:0] info:    0.065536    0.131072 2
Mar 13 20:53:36 router unbound[61026]: [61026:0] info:    0.131072    0.262144 10
Mar 13 20:53:36 router unbound[61026]: [61026:0] info:    2.000000    4.000000 1
Mar 13 20:53:36 router unbound[61026]: [61026:0] info: server stats for thread 2: 16 queries, 8 answers from cache, 8 recursions, 0 prefetch, 0 rejected by ip ratelimiting
          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Did you decide what 192.0.0.1 is?

            One interesting thing is that it's detected as an APU1 but that's an APU2. What does the output of kenv | grep smbios show?

            X 2 Replies Last reply Reply Quote 0
            • X
              Ximulate @stephenw10
              last edited by

              @stephenw10
              RE 192.0..0.1
              https://wirelessjoint.com/viewtopic.php?t=4191

              My gut here is that the gateway pinger under pfsense is wreaking havoc if/when your IG is going between 192.168.225.x and 192.0.0.x. due to its default IPPT configuration. There are changes you can make to improve the situation with IPPT or simply disable it if you don't mind the additional NAT layer. All of this explained in detail if you read on :)
              In addressing some of your issues it will first be good to shed some light on why you are seeing a 192.0.0.x address on your WAN interface even though the default UI access IP is 192.168.225.1. The InvisaGig provides IP Passthrough (IPPT) mode to the device connected to its Ethernet port by default and it uses ARP to find the connected device's MAC address which is to be assigned the IPPT address. This means that when the modem successfully connects to the carrier it bridges the IPv4 assigned by the carrier to the Ethernet interface of the InvisaGig.
              If your cellular carrier network carrier uses IPv6 for the connection and you do not have a plan which specifically provides a statically assigned IPv4 address, the modem will connect to the carrier over IPv6 but perform IPv4 translation. This translation is referred to as dual stack lite, customer-side translation (CLAT) and will utilize an IPv4 address in the range defined by RFC 6333 which is typically 192.0.0.1 at the modem gateway and 192.0.0.2 assigned to the IG-connected IPPT device. In the USA, this scenario is most typical of T-Mobile consumer cellular plans and/or business plans without the static IP add-on so I am guessing this is your carrier. This is especially the case when connecting over 5G Standalone (5G SA) as that is IPv6 only. Under 5G Non-standalone (5G NSA) and LTE it is possible you may still receive a native, legacy IPv4 address from the carrier grade NAT (CGNAT) DHCP pool instead of a 192.0.0.x address.
              Most devices and/or routers connected to the InvisaGig with the default IPPT functionality enabled will not have an issue with 192.0.0.x addressing on their WAN but from time to time we do see some enterprise equipment which has an issue with this address space and/or issues with IPPT bridge creation / MAC autodetection of its WAN port by the IG when reconnecting to the carrier. Depending on the root cause in the case of your PFSense router/firewall there may be a couple of settings which need changing. The first I would suggest would be to set the IPPT MAC address of the PFSense WAN port manually.

              1 Reply Last reply Reply Quote 0
              • X
                Ximulate @stephenw10
                last edited by

                @stephenw10
                RE kenv | grep smbios

                smbios.bios.reldate="02/28/2017"
                smbios.bios.revision="4.0"
                smbios.bios.vendor="coreboot"
                smbios.bios.version="4.0.7"
                smbios.chassis.maker="PC Engines"
                smbios.chassis.type="Desktop"
                smbios.planar.location=""
                smbios.planar.maker="PC Engines"
                smbios.planar.product="APU2"
                smbios.planar.serial="1131876"
                smbios.planar.tag=""
                smbios.planar.version="1.0"
                smbios.system.maker="PC Engines"
                smbios.system.product="APU2"
                smbios.system.serial="1131876"
                smbios.system.sku="4 GB"
                smbios.system.version="1.0"
                smbios.version="2.7"

                1 Reply Last reply Reply Quote 0
                • JonathanLeeJ
                  JonathanLee
                  last edited by JonathanLee

                  Is it overheating? I had a pcie mini card reboot randomly until I added heatsinks. My logs however just showed <Boot> nothing else before it. Weird one

                  Make sure to upvote

                  X 1 Reply Last reply Reply Quote 0
                  • X
                    Ximulate @JonathanLee
                    last edited by Ximulate

                    @JonathanLee

                    I've wondered about that, but I don't see any evidince of it. The box get good air flow, its in a cool basement. I've even placed old heat sinks on the case (not attached) to help a little. Not sure that the screenshot below is going to post, but it shows fairly steady temps of about 55°C +/- 1° (the dip is when I power cycled)

                    Screenshot of Monitoring Graph

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Ah yes DS-Lite. I see that so infrequently I always forget about it. So 192.0.0.2 is a legitimate IP address in that case.

                      I agree the switch from local DHCP to bridged is what I always suspected is the issue. The problem happens is the modem reboots or it;s link is lost and it starts handing out IPs in it's own subnet. When the link is re-established pfSense will not necessarily try to pull a new lease so fails until it renews.

                      This is quite common in cable modems and the usu workaround is to deny leases from the cable modem IP so it just keeps trying until the upstream link comes back up.

                      If you are seeing that it will be logged. The dhclient logs in the dhcp log will show the local modem address that's serving leases.

                      X 1 Reply Last reply Reply Quote 0
                      • X
                        Ximulate @stephenw10
                        last edited by

                        @stephenw10

                        Mar 13 19:48:49 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:48:51 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:48:54 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:48:59 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:04 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:10 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:22 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:31 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:39 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:50 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:50:13 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:50:29 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:50:59 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:51:47 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:52:31 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:52:54 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:04 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:17 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:24 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:31 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:45 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:54:17 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:54:29 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:54:39 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:54:51 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:55:13 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:55:38 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:55:45 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:55:56 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:56:11 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:56:22 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:56:32 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:56:47 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:57:17 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:57:53 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:58:34 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:58:56 router dhcpd[74227]: DHCPREQUEST for 10.11.11.64 from 00:15:88:6b:bb:4d via igb2.11
Mar 13 19:58:56 router dhcpd[74227]: DHCPACK on 10.11.11.64 to 00:15:88:6b:bb:4d via igb2.11
Mar 13 19:59:28 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 20:00:36 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 20:01:32 router dhcpd[74227]: DHCPREQUEST for 10.111.11.138 from 82:58:13:8d:90:9a (Pixel-8) via igb2.111
Mar 13 20:01:32 router dhcpd[74227]: DHCPACK on 10.111.11.138 to 82:58:13:8d:90:9a (Pixel-8) via igb2.111
Mar 13 20:01:51 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 20:03:59 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 20:05:14 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 20:06:08 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 20:08:35 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 20:08:44 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:08:54 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:09:04 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:09:18 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:09:43 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:10:05 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:10:28 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:10:41 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:10:55 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:11:30 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:13:01 router dhcpd[74227]: DHCPREQUEST for 10.111.11.134 from 00:15:cd:5a:43:44 (0005CD5A4344) via igb2.111
Mar 13 20:13:01 router dhcpd[74227]: DHCPACK on 10.111.11.134 to 00:15:cd:5a:43:44 (0005CD5A4344) via igb2.111
Mar 13 20:13:05 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:13:53 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:14:45 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:16:01 router dhcpd[74227]: DHCPREQUEST for 10.251.11.126 from dc:h6:32:9a:88:95 (svrRPI364b) via igb2.251
Mar 13 20:16:01 router dhcpd[74227]: Wrote 0 class decls to leases file.
Mar 13 20:16:01 router dhcpd[74227]: Wrote 0 deleted host decls to leases file.
Mar 13 20:16:01 router dhcpd[74227]: Wrote 0 new dynamic host decls to leases file.
Mar 13 20:16:01 router dhcpd[74227]: Wrote 129 leases to leases file.
Mar 13 20:16:01 router dhcpd[74227]: DHCPACK on 10.251.11.126 to dc:h6:32:9a:88:95 (svrRPI364b) via igb2.251
Mar 13 20:16:14 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:19:42 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:21:45 router dhcpd[74227]: DHCPREQUEST for 10.111.11.130 from e6:34:b1:e6:73:8a (Pixel-7a) via igb2.111
Mar 13 20:21:45 router dhcpd[74227]: DHCPACK on 10.111.11.130 to e6:34:b1:e6:73:8a (Pixel-7a) via igb2.111
Mar 13 20:21:48 router dhclient[84101]: DHCPREQUEST on igb1 to 255.255.255.255 port 67
Mar 13 20:22:20 router dhclient[6155]: EXPIRE
Mar 13 20:22:20 router dhclient[7266]: Deleting old routes
Mar 13 20:22:20 router dhclient[8836]: PREINIT

                        Mar 13 20:54:36 router dhcpd[50201]: Server starting service.
Mar 13 20:54:36 router dhclient[51958]: PREINIT
Mar 13 20:54:36 router dhclient[53206]: EXPIRE
Mar 13 20:54:36 router dhclient[54111]: Deleting old routes
Mar 13 20:54:36 router dhclient[55441]: PREINIT
Mar 13 20:54:37 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 20:54:39 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 3
Mar 13 20:54:42 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 5
Mar 13 20:54:47 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 11
Mar 13 20:54:58 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 12
Mar 13 20:55:10 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 15
Mar 13 20:55:25 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 12
Mar 13 20:55:37 router dhclient[50960]: No DHCPOFFERS received.
Mar 13 20:55:37 router dhclient[50960]: No working leases in persistent database - sleeping.
Mar 13 20:55:37 router dhclient[92864]: FAIL
Mar 13 20:55:51 router dhcpd[82618]: Internet Systems Consortium DHCP Server 4.4.3-P1
Mar 13 20:55:51 router dhcpd[82618]: Copyright 2004-2022 Internet Systems Consortium.
Mar 13 20:55:51 router dhcpd[82618]: All rights reserved.
Mar 13 20:55:51 router dhcpd[82618]: For info, please visit https://www.isc.org/software/dhcp/
Mar 13 20:55:51 router dhcpd[82618]: Config file: /etc/dhcpd.conf
Mar 13 20:55:51 router dhcpd[82618]: Database file: /var/db/dhcpd.leases
Mar 13 20:55:51 router dhcpd[82618]: PID file: /var/run/dhcpd.pid
Mar 13 20:55:51 router dhcpd[82618]: Internet Systems Consortium DHCP Server 4.4.3-P1
Mar 13 20:55:51 router dhcpd[82618]: Copyright 2004-2022 Internet Systems Consortium.
Mar 13 20:55:51 router dhcpd[82618]: All rights reserved.
Mar 13 20:55:51 router dhcpd[82618]: For info, please visit https://www.isc.org/software/dhcp/
Mar 13 20:55:51 router dhcpd[82618]: Wrote 0 class decls to leases file.
Mar 13 20:55:51 router dhcpd[82618]: Wrote 0 deleted host decls to leases file.
Mar 13 20:55:51 router dhcpd[82618]: Wrote 0 new dynamic host decls to leases file.
Mar 13 20:55:51 router dhcpd[82618]: Wrote 129 leases to leases file.
Mar 13 20:55:51 router dhcpd[82618]: Listening on BPF/igb2.101/00:0d:b9:a9:16:92/10.101.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Sending on   BPF/igb2.101/00:0d:b9:a9:16:92/10.101.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Listening on BPF/igb2.251/00:0d:b9:a9:16:92/10.251.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Sending on   BPF/igb2.251/00:0d:b9:a9:16:92/10.251.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Listening on BPF/igb2.111/00:0d:b9:a9:16:92/10.111.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Sending on   BPF/igb2.111/00:0d:b9:a9:16:92/10.111.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Listening on BPF/igb2.31/00:0d:b9:a9:16:92/10.31.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Sending on   BPF/igb2.31/00:0d:b9:a9:16:92/10.31.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Listening on BPF/igb2.11/00:0d:b9:a9:16:92/10.11.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Sending on   BPF/igb2.11/00:0d:b9:a9:16:92/10.11.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Listening on BPF/igb2/00:0d:b9:a9:16:92/10.0.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Sending on   BPF/igb2/00:0d:b9:a9:16:92/10.0.11.0/24
Mar 13 20:55:51 router dhcpd[82618]: Sending on   Socket/fallback/fallback-net
Mar 13 20:55:51 router dhcpd[82618]: Server starting service.
Mar 13 20:55:52 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 20:55:54 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 4
Mar 13 20:55:58 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 6
Mar 13 20:56:04 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 10
Mar 13 20:56:14 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 17
Mar 13 20:56:31 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 20
Mar 13 20:56:51 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 20:56:53 router dhclient[50960]: No DHCPOFFERS received.
Mar 13 20:56:53 router dhclient[50960]: No working leases in persistent database - sleeping.
Mar 13 20:56:53 router dhclient[34633]: FAIL
                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          Hmm, nothing shown there at all. No responses from any DHCP server. That was during some outage I assume?

                          X 1 Reply Last reply Reply Quote 0
                          • X
                            Ximulate @stephenw10
                            last edited by

                            @stephenw10
                            Right, that was about the time the network failure occured yesterday. I don't know exactly when it started failing, but I was troubleshooting prior to having to power cycle at 21:47 (see log in yesterday's post).

                            Mar 13 19:44:30 router dhcpd[74227]: DHCPDISCOVER from 30:e9:50:8e:e2:91 (Sweeper_p100) via igb2.31
Mar 13 19:44:30 router dhcpd[74227]: DHCPOFFER on 10.31.11.235 to 30:e9:50:8e:e2:91 (Sweeper_p100) via igb2.31
Mar 13 19:44:31 router dhcpd[74227]: reuse_lease: lease age 1510 (secs) under 25% threshold, reply with unaltered, existing lease for 10.31.11.235
Mar 13 19:44:31 router dhcpd[74227]: DHCPREQUEST for 10.31.11.235 (10.31.11.1) from 30:e9:50:8e:e2:91 (Sweeper_p100) via igb2.31
Mar 13 19:48:49 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:48:51 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:48:54 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:48:59 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:04 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:10 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:22 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:31 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:39 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:49:50 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:50:13 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:50:29 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:50:59 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:51:47 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:52:31 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:52:54 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:04 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:17 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:24 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:31 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:53:45 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:54:17 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:54:29 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:54:39 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:54:51 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:55:13 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:55:38 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:55:45 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:55:56 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:56:11 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:56:22 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:56:32 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:56:47 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:57:17 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:57:53 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:58:34 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 19:58:56 router dhcpd[74227]: DHCPREQUEST for 10.11.11.64 from 00:15:88:6b:bb:4d via igb2.11
Mar 13 19:58:56 router dhcpd[74227]: DHCPACK on 10.11.11.64 to 00:15:88:6b:bb:4d via igb2.11
Mar 13 19:59:28 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 20:00:36 router dhclient[84101]: DHCPREQUEST on igb1 to 192.0.0.1 port 67
Mar 13 20:01:32 router dhcpd[74227]: DHCPREQUEST for 10.111.11.138 from 82:58:13:8d:90:9a (Pixel-8) via igb2.111
Mar 13 20:01:32 router dhcpd[74227]: DHCPACK on 10.111.11.138 to 82:58:13:8d:90:9a (Pixel-8) via igb2.111

                            Then is a lot of the following DHCP logs just prior to power cycle:

                            Mar 13 21:29:49 router dhclient[9249]: FAIL
Mar 13 21:30:04 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 1
Mar 13 21:30:05 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 21:30:07 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 5
Mar 13 21:30:12 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 6
Mar 13 21:30:18 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 15
Mar 13 21:30:33 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 18
Mar 13 21:30:51 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 12
Mar 13 21:31:03 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 21:31:05 router dhclient[50960]: No DHCPOFFERS received.
Mar 13 21:31:05 router dhclient[50960]: No working leases in persistent database - sleeping.
Mar 13 21:31:05 router dhclient[53331]: FAIL
Mar 13 21:31:20 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 21:31:22 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 4
Mar 13 21:31:26 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 4
Mar 13 21:31:30 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 10
Mar 13 21:31:40 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 13
Mar 13 21:31:53 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 16
Mar 13 21:32:06 router dhcpd[82618]: uid lease 10.111.11.126 for client 48:a9:44:91:ec:4f is duplicate on 10.111.11.0/24
Mar 13 21:32:06 router dhcpd[82618]: DHCPREQUEST for 10.111.11.51 from 48:e9:44:91:ec:4f via igb2.111
Mar 13 21:32:06 router dhcpd[82618]: DHCPACK on 10.111.11.51 to 48:a9:44:91:ec:4f via igb2.111
Mar 13 21:32:09 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 12
Mar 13 21:32:13 router dhcpd[82618]: DHCPREQUEST for 10.0.11.121 from 78:a9:58:dd:69:cf (MstrCloset) via igb2
Mar 13 21:32:13 router dhcpd[82618]: DHCPACK on 10.0.11.121 to 78:a9:58:dd:69:cf (MstrCloset) via igb2
Mar 13 21:32:18 router dhcpd[82618]: DHCPREQUEST for 10.11.11.62 from dc:a6:32:9a:88:8c via igb2.11
Mar 13 21:32:18 router dhcpd[82618]: DHCPACK on 10.11.11.62 to dc:a6:32:9a:88:8c via igb2.11
Mar 13 21:32:21 router dhclient[50960]: No DHCPOFFERS received.
Mar 13 21:32:21 router dhclient[50960]: No working leases in persistent database - sleeping.
Mar 13 21:32:21 router dhclient[22]: FAIL
Mar 13 21:32:36 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 1
Mar 13 21:32:37 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 21:32:39 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 21:32:41 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 5
Mar 13 21:32:44 router dhcpd[82618]: DHCPREQUEST for 10.11.11.91 from 00:e9:9d:db:a6:54 via igb2.11
Mar 13 21:32:44 router dhcpd[82618]: DHCPACK on 10.11.11.91 to 00:e9:9d:db:a6:54 via igb2.11
Mar 13 21:32:46 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 9
Mar 13 21:32:55 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 16
Mar 13 21:33:11 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 7
Mar 13 21:33:18 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 11
Mar 13 21:33:29 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 8
Mar 13 21:33:37 router dhclient[50960]: No DHCPOFFERS received.
Mar 13 21:33:37 router dhclient[50960]: No working leases in persistent database - sleeping.
Mar 13 21:33:37 router dhclient[79182]: FAIL
Mar 13 21:33:52 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 21:33:54 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 4
Mar 13 21:33:58 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 11
Mar 13 21:34:09 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 11
Mar 13 21:34:20 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 12
Mar 13 21:34:32 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 15
Mar 13 21:34:47 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 6
Mar 13 21:34:53 router dhclient[50960]: No DHCPOFFERS received.
Mar 13 21:34:53 router dhclient[50960]: No working leases in persistent database - sleeping.
Mar 13 21:34:53 router dhclient[36776]: FAIL
Mar 13 21:35:08 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 1
Mar 13 21:35:09 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 2
Mar 13 21:35:11 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 5
Mar 13 21:35:16 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 10
Mar 13 21:35:18 router dhcpd[82618]: DHCPREQUEST for 10.0.11.33 from 78:a9:58:46:f9:44 via igb2
Mar 13 21:35:18 router dhcpd[82618]: DHCPACK on 10.0.11.33 to 78:a9:58:46:f9:44 via igb2
Mar 13 21:35:26 router dhclient[50960]: DHCPDISCOVER on igb1 to 255.255.255.255 port 67 interval 13
Mar 13 21:47:13 router dhclient[30088]: PREINIT

                            I just noticed this in above, which is our printer. However, there is no duplicate in the currect DHCP leases.

                            Mar 13 21:32:06 router dhcpd[82618]: uid lease 10.111.11.126 for client 48:a9:44:91:ec:4f is duplicate on 10.111.11.0/24
Mar 13 21:32:06 router dhcpd[82618]: DHCPREQUEST for 10.111.11.51 from 48:e9:44:91:ec:4f via igb2.111
Mar 13 21:32:06 router dhcpd[82618]: DHCPACK on 10.111.11.51 to 48:a9:44:91:ec:4f via igb2.111
                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Hmm, so it appears that when this goes down there are no responding DHCP servers on the WAN. So either the modem stops both bridging the ISP DS-Lite connection or handing out it's own leases. Or the NIC in pfSense is not actually sending the DHCP leases (or not seeing the responses).

                              Are you able to try to capture traffic on the WAN side?

                              Since you're already behind double NAT you might just try disabling the IP pass through in the modem. That would avoid the IP/subnet change. It might also prevent the modem failing if that's what is happening.

                              X 1 Reply Last reply Reply Quote 0
                              • X
                                Ximulate @stephenw10
                                last edited by

                                @stephenw10
                                It would take some time for me to set-up, but I could probably set-up wireshark on a RasPi & a small switch to log packets (I think thats what your suggesting.)

                                There is a way to disable IP pass though in the modem, so I'll try that next.

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Yep an ideal test there would be capturing packets from a switch mirror port but that's clearly quite involved to get in place.

                                  That would prove it's not an issue with the NIC though.

                                  X 1 Reply Last reply Reply Quote 0
                                  • X
                                    Ximulate @stephenw10
                                    last edited by Ximulate

                                    @stephenw10 So I'm back this 😧
                                    I had all but decided its time to buy a new router, but had to go out of town so I threw together the script descibed below executed by cron ever 12 minutes. That was about 3 weeks ago. Router hasn't had any issue until today.

                                    The only reason it an issue was that I was testing the back-up connection by killing power the primary wan modem. Failover was fine, but when I brought the primary WAN back online the LAN comms would die. Looking at the log generated by my script, it appears the script has cycled the wan several times over the last few weeks which makes me think the script mostly works but I'm not sure why or what problem its resolving by cycling the wan.

                                    gwstat=$(pfSsh.php playback gatewaystatus)
WAN_STATE=$(echo "$gwstat" | awk '/'$GW_ID'/ { print $7}')
if [ $WAN_STATE = "online" ]; then exit; fi
echo "WAN Cycling on $varDate" >> $log
#turn off modem using Hue Appliance Plug (doubtful this works, because lan comms are down so the Hue Hub is prob unreachable. Just now modifed script to log ouput)
/usr/local/bin/curl -X PUT -H "Content-Type: application/json" -d '{"on": false}' $url >> $log
ifconfig igb0 down
# Sleep, then bring up modem
sleep 30
/usr/local/bin/curl -X PUT -H "Content-Type: application/json" -d '{"on": true}' $url >> $log
ifconfig igb0 up
                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      Hmm, well if you kill the WAN connection deliberately that script is going to continually try to cycle the WAN modem. Though I'm not sure why that would prevent it coming back up or otherwise kill the LAN side connectivity.

                                      X 1 Reply Last reply Reply Quote 0
                                      • VioletDragonV
                                        VioletDragon
                                        last edited by

                                        One thing that was neglected to mention here, is Memtest. I’d suggest running memtest to see if there are any memory errors.

                                        Second. Test another Power Supply, recently I had a client that had the same problem due to a faulty 12v rail.

                                        Regards

                                        1 Reply Last reply Reply Quote 1
                                        • X
                                          Ximulate @stephenw10
                                          last edited by

                                          @stephenw10

                                          Network just went down again, kind of, and through no action of mine. This time, I was able to load internet webpages through my fail-over connection (so that was working). I could ping the router over the LAN. I really don't think the primary WAN was down, as once I power cycled it was online. I also confirmed that the power plug was on (so everything was getting power and should be online.)

                                          However, the router GUI was reporting a 504 error (white page with black text served by ngnix.) I was able SSH into the router, so I tried to restart PHP-FAM. This allowed me to load the GUI login, but after entering credentials the browser waited, waited, waited and then back to 504 error. Ended up power cycling.

                                          Per @VioletDragon suggestion, I just replaced the power supply. Earlier in the troubleshooting process, I think I already tried another new power supply (maybe 3rd's the charm) and I had already run memtest overnight, no errors reported.

                                          VioletDragonV 1 Reply Last reply Reply Quote 0
                                          • VioletDragonV
                                            VioletDragon @Ximulate
                                            last edited by

                                            @Ximulate Just out of interest. Can you share System -> Advanced -> Network Interfaces.

                                            Are you routing Layer 2 or 3 ? Is Jumbo Frames been left at default 1500 ?

                                            Have you tested a fresh install ?

                                            Regards.

                                            X 2 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.