WAN link going down
-
This:
kernel: sk0: link state changed to DOWN
Indicates that the link is actually going down not just being marked down by apinger. Are you using the modified sk(4) driver? There have been a number of reports of them not working well under various conditions. Are you seeing errors in the boot log like this?:
https://forum.pfsense.org/index.php?topic=20095.msg485091#msg485091Steve
-
I'll check this when i get home, i also see inthe gateway logs that apinger couldn't get the gateway.
But that is probably cause and effect, if the link is down then the gateway can't be located.Thank in advance steve !
-
Nope, i can't see any problems in the boot log.
I'm also not using the modified sk(4) drivers, it's just a default installationCopyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.1-RELEASE-p4 #0 36d7dec(releng/10.1)-dirty: Thu Jan 22 15:12:38 CST 2015 root@pfsense-22-i386-builder:/usr/obj.i386/usr/pfSensesrc/src/sys/pfSense_SMP.10 i386 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Intel(R) Pentium(R) M processor 2.00GHz (2000.11-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6d8 Family = 0x6 Model = 0xd Stepping = 8 Features=0xafe9fbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,tm,pbe>Features2=0x180 <est,tm2>AMD Features=0x100000 <nx>real memory = 2139029504 (2039 MB) avail memory = 2064244736 (1968 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: <intelr awrdacpi="">ioapic0 <version 2.0="">irqs 0-23 on motherboard wlan: mac acl policy registered kbd1 at kbdmux0 random: <software, yarrow="">initialized cryptosoft0: <software crypto="">on motherboard padlock0: No ACE support. acpi0: <intelr awrdacpi="">on motherboard acpi0: Power Button (fixed) acpi0: reservation of a2, 1e (4) failed acpi0: reservation of e0, 10 (4) failed acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 7f6f0000 (3) failed cpu0: <acpi cpu="">on acpi0 attimer0: <at timer="">port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: <at realtime="" clock="">port 0x70-0x73 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 acpi_button0: <power button="">on acpi0 pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0 pci0: <acpi pci="" bus="">on pcib0 vgapci0: <vga-compatible display="">port 0xea00-0xea07 mem 0xd0500000-0xd057ffff,0xc0000000-0xcfffffff,0xd0580000-0xd05bffff irq 16 at device 2.0 on pci0 agp0: <intel 82915gm="" (915gm="" gmch)="" svga="" controller="">on vgapci0 agp0: aperture size is 256M, detected 7932k stolen memory vgapci0: Boot video device pcib1: <acpi pci-pci="" bridge="">irq 16 at device 28.0 on pci0 pci1: <acpi pci="" bus="">on pcib1 mskc0: <marvell yukon="" 88e8053="" gigabit="" ethernet="">port 0x8000-0x80ff mem 0xd0020000-0xd0023fff irq 16 at device 0.0 on pci1 msk0: <marvell technology="" group="" ltd.="" yukon="" ec="" id="" 0xb6="" rev="" 0x02="">on mskc0 miibus0: <mii bus="">on msk0 e1000phy0: <marvell 88e1111="" gigabit="" phy="">PHY 0 on miibus0 e1000phy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow pcib2: <acpi pci-pci="" bridge="">irq 17 at device 28.1 on pci0 pci2: <acpi pci="" bus="">on pcib2 mskc1: <marvell yukon="" 88e8053="" gigabit="" ethernet="">port 0x9000-0x90ff mem 0xd0120000-0xd0123fff irq 17 at device 0.0 on pci2 msk1: <marvell technology="" group="" ltd.="" yukon="" ec="" id="" 0xb6="" rev="" 0x02="">on mskc1 miibus1: <mii bus="">on msk1 e1000phy1: <marvell 88e1111="" gigabit="" phy="">PHY 0 on miibus1 e1000phy1: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow pcib3: <acpi pci-pci="" bridge="">irq 18 at device 28.2 on pci0 pci3: <acpi pci="" bus="">on pcib3 mskc2: <marvell yukon="" 88e8053="" gigabit="" ethernet="">port 0xa000-0xa0ff mem 0xd0220000-0xd0223fff irq 18 at device 0.0 on pci3 msk2: <marvell technology="" group="" ltd.="" yukon="" ec="" id="" 0xb6="" rev="" 0x02="">on mskc2 miibus2: <mii bus="">on msk2 e1000phy2: <marvell 88e1111="" gigabit="" phy="">PHY 0 on miibus2 e1000phy2: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow pcib4: <acpi pci-pci="" bridge="">irq 19 at device 28.3 on pci0 pci4: <acpi pci="" bus="">on pcib4 mskc3: <marvell yukon="" 88e8053="" gigabit="" ethernet="">port 0xb000-0xb0ff mem 0xd0320000-0xd0323fff irq 19 at device 0.0 on pci4 msk3: <marvell technology="" group="" ltd.="" yukon="" ec="" id="" 0xb6="" rev="" 0x02="">on mskc3 miibus3: <mii bus="">on msk3 e1000phy3: <marvell 88e1111="" gigabit="" phy="">PHY 0 on miibus3 e1000phy3: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow uhci0: <intel 82801fb="" fr="" fw="" frw="" (ich6)="" usb="" controller="" usb-a="">port 0xeb00-0xeb1f irq 23 at device 29.0 on pci0 usbus0 on uhci0 uhci1: <intel 82801fb="" fr="" fw="" frw="" (ich6)="" usb="" controller="" usb-b="">port 0xed00-0xed1f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: <intel 82801fb="" fr="" fw="" frw="" (ich6)="" usb="" controller="" usb-c="">port 0xe800-0xe81f irq 18 at device 29.2 on pci0 usbus2 on uhci2 uhci3: <intel 82801fb="" fr="" fw="" frw="" (ich6)="" usb="" controller="" usb-d="">port 0xe900-0xe91f irq 16 at device 29.3 on pci0 usbus3 on uhci3 ehci0: <intel 82801fb="" (ich6)="" usb="" 2.0="" controller="">mem 0xd05c0000-0xd05c03ff irq 23 at device 29.7 on pci0 usbus4: EHCI version 1.0 usbus4 on ehci0 pcib5: <acpi pci-pci="" bridge="">at device 30.0 on pci0 pci5: <acpi pci="" bus="">on pcib5 skc0: <marvell gigabit="" ethernet="">port 0xc000-0xc0ff mem 0xd042c000-0xd042ffff irq 16 at device 0.0 on pci5 skc0: Marvell Yukon Lite Gigabit Ethernet rev. (0x9) sk0: <marvell semiconductor,="" inc.="" yukon="">on skc0 miibus4: <mii bus="">on sk0 e1000phy4: <marvell 88e1011="" gigabit="" phy="">PHY 0 on miibus4 e1000phy4: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto skc1: <marvell gigabit="" ethernet="">port 0xc400-0xc4ff mem 0xd0420000-0xd0423fff irq 17 at device 1.0 on pci5 skc1: Marvell Yukon Lite Gigabit Ethernet rev. (0x9) sk1: <marvell semiconductor,="" inc.="" yukon="">on skc1 miibus5: <mii bus="">on sk1 e1000phy5: <marvell 88e1011="" gigabit="" phy="">PHY 0 on miibus5 e1000phy5: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto skc2: <marvell gigabit="" ethernet="">port 0xc800-0xc8ff mem 0xd0424000-0xd0427fff irq 18 at device 2.0 on pci5 skc2: Marvell Yukon Lite Gigabit Ethernet rev. (0x9) sk2: <marvell semiconductor,="" inc.="" yukon="">on skc2 miibus6: <mii bus="">on sk2 e1000phy6: <marvell 88e1011="" gigabit="" phy="">PHY 0 on miibus6 e1000phy6: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto skc3: <marvell gigabit="" ethernet="">port 0xcc00-0xccff mem 0xd0428000-0xd042bfff irq 19 at device 3.0 on pci5 skc3: Marvell Yukon Lite Gigabit Ethernet rev. (0x9) sk3: <marvell semiconductor,="" inc.="" yukon="">on skc3 miibus7: <mii bus="">on sk3 e1000phy7: <marvell 88e1011="" gigabit="" phy="">PHY 0 on miibus7 e1000phy7: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto pci5: <encrypt decrypt,="" network="" computer="" crypto="">at device 4.0 (no driver attached) isab0: <pci-isa bridge="">at device 31.0 on pci0 isa0: <isa bus="">on isab0 atapci0: <intel ich6="" udma100="" controller="">port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 31.1 on pci0 ata0: <ata channel="">at channel 0 on atapci0 ata1: <ata channel="">at channel 1 on atapci0 acpi_tz0: <thermal zone="">on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 ppc0: <parallel port="">port 0x378-0x37f,0x778-0x77b irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: <parallel port="" bus="">on ppc0 lpt0: <printer>on ppbus0 lpt0: Interrupt-driven port ppi0: <parallel i="" o="">on ppbus0 pmtimer0 on isa0 orm0: <isa option="" rom="">at iomem 0xc0000-0xce7ff pnpid ORM0000 on isa0 sc0: <system console="">at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <generic isa="" vga="">at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: <keyboard controller="" (i8042)="">at port 0x60,0x64 on isa0 atkbd0: <at keyboard="">irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] est0: <enhanced speedstep="" frequency="" control="">on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 6120f2906000f29 device_attach: est0 attach returned 6 p4tcc0: <cpu frequency="" thermal="" control="">on cpu0 Timecounters tick every 1.000 msec IPsec: Initialized Security Association Processing. random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 12Mbps Full Speed USB v1.0 ugen0.1: <intel>at usbus0 uhub0: <intel 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usbus0 ugen1.1: <intel>at usbus1 uhub1: <intel 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usbus1 ugen2.1: <intel>at usbus2 uhub2: <intel 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usbus2 ugen3.1: <intel>at usbus3 uhub3: <intel 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usbus3 usbus4: 480Mbps High Speed USB v2.0 ugen4.1: <intel>at usbus4 uhub4: <intel 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usbus4 ata0: DMA limited to UDMA33, controller found non-ATA66 cable ada0 at ata0 bus 0 scbus0 target 0 lun 0 ada0: <wdc wd800beve-00uyt0="" 01.04a01="">ATA-7 device ada0: Serial Number WD-WXEX08TJ6328 ada0: 33.300MB/s transfers (UDMA2, PIO 8192bytes) ada0: 76319MB (156301488 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad0 Timecounter "TSC" frequency 2000113799 Hz quality 800 Root mount waiting for: usbus4 usbus3 usbus2 usbus1 usbus0 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 uhub4: 8 ports with 8 removable, self powered Trying to mount root from ufs:/dev/ad0s1a [rw]...</wdc></intel></intel></intel></intel></intel></intel></intel></intel></intel></intel></cpu></enhanced></at></keyboard></generic></system></isa></parallel></printer></parallel></parallel></thermal></ata></ata></intel></isa></pci-isa></encrypt></marvell></mii></marvell></marvell></marvell></mii></marvell></marvell></marvell></mii></marvell></marvell></marvell></mii></marvell></marvell></acpi></acpi></intel></intel></intel></intel></intel></marvell></mii></marvell></marvell></acpi></acpi></marvell></mii></marvell></marvell></acpi></acpi></marvell></mii></marvell></marvell></acpi></acpi></marvell></mii></marvell></marvell></acpi></acpi></intel></vga-compatible></acpi></acpi></power></at></at></acpi></intelr></software></software,></version></intelr></nx></est,tm2></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,tm,pbe>
-
That's good because I'm not sure there's much we can do with the drivers! ;)
Try disabling apinger for the gateway in System > Routing, edit gateway.
In 2.2 there is an option to get more info from apinger by enabling 'gateway monitoring debug logging' in System > Advanced, Miscellaneous tab. Try that if disabling apinger makes any difference.
Steve
-
That's good because I'm not sure there's much we can do with the drivers! ;)
Try disabling apinger for the gateway in System > Routing, edit gateway.
In 2.2 there is an option to get more info from apinger by enabling 'gateway monitoring debug logging' in System > Advanced, Miscellaneous tab. Try that if disabling apinger makes any difference.
Steve
Ok ;D
I'll try disabling apinger first and see how it goes the next days.
Its happens always after the Firewall is online for 2 days.Greetz
Tim -
Ow yeah, I'm also getting these messages…
On a disabled interface and no cable plugged in.They are on the same interface card though (sk..)
I only use sk0 and sk1 for now.Feb 3 18:02:08 php-fpm[61051]: /rc.linkup: Linkup detected on disabled interface...Ignoring Feb 3 18:02:08 php-fpm[61051]: /rc.linkup: Linkup detected on disabled interface...Ignoring Feb 3 18:02:07 check_reload_status: Linkup starting sk3 Feb 3 18:02:07 kernel: sk3: link state changed to DOWN Feb 3 18:02:07 kernel: sk2: link state changed to DOWN Feb 3 18:02:07 check_reload_status: Linkup starting sk2
-
Is that at any time or just at boot?
-
Is that at any time or just at boot?
At the end of the boot cycle :)
I also see that WOL isn't working anymore.. strange -
WOL is a known x86 issue with a patch on Redmine, completely unrelated.
-
I also see that WOL isn't working anymore.. strange
I guess you are using 32-bit pfSense 2.2 - that is effected by the gen_subnetv4_max issue, fixed by:
https://github.com/pfsense/pfsense/commit/7094c303b7d46c9f7b24c3f1bd4432187832e85cThings that try to calculate the max IP address of a subnet are effected - WOL is one of them.
You can apply that small fix and WOL should work again.
-
I also see that WOL isn't working anymore.. strange
I guess you are using 32-bit pfSense 2.2 - that is effected by the gen_subnetv4_max issue, fixed by:
https://github.com/pfsense/pfsense/commit/7094c303b7d46c9f7b24c3f1bd4432187832e85cThings that try to calculate the max IP address of a subnet are effected - WOL is one of them.
You can apply that small fix and WOL should work again.
Thank you !
I was guessing in that direction, because when using WOL in the terminal it worked :) -
Spoke too soon.
The WAN link is still losing its connection.
Feb 4 13:37:17 kernel: arpresolve: can't allocate llinfo for 84.192.192.1 on sk0 Feb 4 13:37:17 kernel: arpresolve: can't allocate llinfo for 84.192.192.1 on sk0 Feb 4 13:37:13 kernel: arpresolve: can't allocate llinfo for 84.192.192.1 on sk0 Feb 4 13:37:06 php-fpm[5814]: /rc.linkup: DEVD Ethernet detached event for wan Feb 4 13:37:05 kernel: sk0: link state changed to DOWN Feb 4 13:37:05 check_reload_status: Linkup starting sk0 Feb 4 13:36:10 php-fpm[26356]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (wan). Feb 4 13:36:08 check_reload_status: Reloading filter Feb 4 13:36:08 php-fpm[26356]: /rc.newwanip: rc.newwanip: on (IP address: 192.168.1.254) (interface: LAN[lan]) (real interface: sk1). Feb 4 13:36:08 php-fpm[26356]: /rc.newwanip: rc.newwanip: Info: starting on sk1. Feb 4 13:36:07 check_reload_status: rc.newwanip starting sk1 Feb 4 13:36:07 php-fpm[26356]: /rc.linkup: Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.1.254 ) Feb 4 13:36:06 kernel: sk1: link state changed to UP Feb 4 13:36:06 check_reload_status: Linkup starting sk1 Feb 4 13:36:01 php-fpm[26356]: /rc.linkup: Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (192.168.1.254 ) Feb 4 13:36:00 kernel: sk1: link state changed to DOWN
Firewall Logs
block/1000000104 Feb 4 13:40:18 Direction=OUT lo0 Default deny rule IPv4 (1000000104) Icon Reverse Resolve with DNS Icon Easy Rule: Add to Block List 74.125.24.139:80 Icon Reverse Resolve with DNS Icon Easy Rule: Pass this traffic 84.192.215.xxx:46001 TCP:FA block/1000000104 Feb 4 13:40:18 Direction=OUT lo0 Default deny rule IPv4 (1000000104) Icon Reverse Resolve with DNS Icon Easy Rule: Add to Block List 74.125.24.139:80 Icon Reverse Resolve with DNS Icon Easy Rule: Pass this traffic 84.192.215.xxx:46001 TCP:FA block/1000000104 Feb 4 13:40:17 Direction=OUT lo0 Default deny rule IPv4 (1000000104) Icon Reverse Resolve with DNS Icon Easy Rule: Add to Block List 74.125.24.139:80 Icon Reverse Resolve with DNS Icon Easy Rule: Pass this traffic 84.192.215.xxx:46001 TCP:FA pass/1422181121 Feb 4 13:24:22 ovpns1 USER_RULE OpenVPN OpenVPN Server wizard (1422181121) Icon Reverse Resolve with DNS Icon Easy Rule: Add to Block List 192.168.10.10 Icon Reverse Resolve with DNS Icon Easy Rule: Pass this traffic 192.168.1.120 ICMP pass/1422181121 Feb 4 13:24:11 ovpns1 USER_RULE OpenVPN OpenVPN Server wizard (1422181121) Icon Reverse Resolve with DNS Icon Easy Rule: Add to Block List 192.168.10.10:55720 Icon Reverse Resolve with DNS Icon Easy Rule: Pass this traffic 192.168.1.254:443 TCP:S
-
I don't know if there is a relation , but tonight i noticed something strange in the logs.
My WAN ip changed to a private range… (twice)
I'm also seeing openvpn and LCDProc errors...wtf !
Feb 11 01:10:30 fw1 snort[39066]: invalid appid in appStatRecord (186) Feb 11 01:28:21 fw1 check_reload_status: updating dyndns WAN_DHCP Feb 11 01:28:21 fw1 check_reload_status: Restarting ipsec tunnels Feb 11 01:28:21 fw1 check_reload_status: Restarting OpenVPN tunnels/interfaces Feb 11 01:28:21 fw1 check_reload_status: Reloading filter Feb 11 01:28:22 fw1 php-fpm[89141]: /rc.dyndns.update: phpDynDNS (***-ip.biz): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Feb 11 01:28:22 fw1 php-fpm[76014]: /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP. Feb 11 01:28:22 fw1 php-fpm[76014]: /rc.openvpn: OpenVPN: Resync server1 OpenVPN Server Feb 11 01:28:22 fw1 kernel: ovpns1: link state changed to DOWN Feb 11 01:28:22 fw1 check_reload_status: Reloading filter Feb 11 01:28:23 fw1 kernel: ovpns1: link state changed to UP Feb 11 01:28:23 fw1 check_reload_status: rc.newwanip starting ovpns1 Feb 11 01:28:24 fw1 php-fpm[89141]: /rc.newwanip: rc.newwanip: Info: starting on ovpns1. Feb 11 01:28:24 fw1 php-fpm[89141]: /rc.newwanip: rc.newwanip: on (IP address: 192.168.10.1) (interface: []) (real interface: ovpns1). Feb 11 01:28:24 fw1 check_reload_status: Reloading filter Feb 11 01:28:24 fw1 php-fpm[89141]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 192.168.10.1 - Restarting packages. Feb 11 01:28:24 fw1 check_reload_status: Starting packages Feb 11 01:28:24 fw1 php-fpm[76014]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (wan). Feb 11 01:28:25 fw1 php-fpm[89141]: /rc.start_packages: Restarting/Starting all packages. Feb 11 01:28:25 fw1 php-fpm[89141]: /rc.start_packages: Reloading Squid for configuration sync Feb 11 01:28:26 fw1 check_reload_status: Syncing firewall Feb 11 01:28:26 fw1 check_reload_status: Syncing firewall Feb 11 01:28:27 fw1 php-fpm[79907]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (wan). Feb 11 01:28:28 fw1 php: lcdproc: Start client procedure. Error counter: (0) Feb 11 01:28:39 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 01:28:39 fw1 php: lcdproc: Start client procedure. Error counter: (1) Feb 11 01:28:50 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 01:28:50 fw1 php: lcdproc: Start client procedure. Error counter: (2) Feb 11 01:29:01 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 01:29:01 fw1 php: lcdproc: Start client procedure. Error counter: (3) Feb 11 01:29:12 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 01:29:12 fw1 php: lcdproc: Too many errors, the client ends. Feb 11 01:33:20 fw1 kernel: pid 71053 (ntpd), uid 0: exited on signal 11 (core dumped) Feb 11 01:55:12 fw1 snort[39066]: invalid appid in appStatRecord (498) Feb 11 02:00:31 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30622 milliseconds Feb 11 02:00:31 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:01:17 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30630 milliseconds Feb 11 02:01:17 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:02:03 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30797 milliseconds Feb 11 02:02:03 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:02:49 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30628 milliseconds Feb 11 02:02:49 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:03:04 fw1 php: snort_check_for_rule_updates.php: File 'snortrules-snapshot-2970.tar.gz.md5' download attempts: 4 ... Feb 11 02:03:04 fw1 php: snort_check_for_rule_updates.php: [Snort] Snort VRT rules md5 download failed... Feb 11 02:03:04 fw1 php: snort_check_for_rule_updates.php: [Snort] Server returned error code 0... Feb 11 02:03:34 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30759 milliseconds Feb 11 02:03:34 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:04:20 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30773 milliseconds Feb 11 02:04:20 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:05:06 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30740 milliseconds Feb 11 02:05:06 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:05:52 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30696 milliseconds Feb 11 02:05:52 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:06:07 fw1 php: snort_check_for_rule_updates.php: File 'snort-openappid.tar.gz.md5' download attempts: 4 ... Feb 11 02:06:07 fw1 php: snort_check_for_rule_updates.php: [Snort] Snort OpenAppID detectors md5 download failed... Feb 11 02:06:07 fw1 php: snort_check_for_rule_updates.php: [Snort] Server returned error code 0... Feb 11 02:06:37 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30597 milliseconds Feb 11 02:06:37 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:07:23 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30771 milliseconds Feb 11 02:07:23 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:08:09 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30889 milliseconds Feb 11 02:08:09 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:08:54 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30633 milliseconds Feb 11 02:08:54 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:09:09 fw1 php: snort_check_for_rule_updates.php: File 'community-rules.tar.gz.md5' download attempts: 4 ... Feb 11 02:09:09 fw1 php: snort_check_for_rule_updates.php: [Snort] Snort GPLv2 Community Rules md5 download failed... Feb 11 02:09:09 fw1 php: snort_check_for_rule_updates.php: [Snort] Server returned error code 0... Feb 11 02:09:40 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30721 milliseconds Feb 11 02:09:40 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:10:26 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30805 milliseconds Feb 11 02:10:26 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:11:12 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30806 milliseconds Feb 11 02:11:12 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:11:58 fw1 php: snort_check_for_rule_updates.php: [Snort] Rules download error: Resolving timed out after 30717 milliseconds Feb 11 02:11:58 fw1 php: snort_check_for_rule_updates.php: [Snort] Will retry in 15 seconds... Feb 11 02:12:13 fw1 php: snort_check_for_rule_updates.php: File 'emerging.rules.tar.gz.md5' download attempts: 4 ... Feb 11 02:12:13 fw1 php: snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules md5 download failed... Feb 11 02:12:13 fw1 php: snort_check_for_rule_updates.php: [Snort] Server returned error code 0... Feb 11 02:12:13 fw1 php: snort_check_for_rule_updates.php: [Snort] The Rules update has finished. Feb 11 02:12:13 fw1 check_reload_status: Syncing firewall Feb 11 02:12:14 fw1 kernel: pid 65763 (php), uid 0: exited on signal 11 (core dumped) Feb 11 02:13:09 fw1 check_reload_status: rc.newwanip starting sk0 Feb 11 02:13:10 fw1 php-fpm[87333]: /rc.newwanip: rc.newwanip: Info: starting on sk0. Feb 11 02:13:10 fw1 php-fpm[87333]: /rc.newwanip: rc.newwanip: on (IP address: 192.168.100.10) (interface: WAN[wan]) (real interface: sk0). Feb 11 02:13:10 fw1 php-fpm[87333]: /rc.newwanip: IP has changed, killing states on former IP 84.192.***.***. Feb 11 02:13:11 fw1 php-fpm[87333]: /rc.newwanip: Could not find IPv6 gateway for interface (wan). Feb 11 02:13:12 fw1 php-fpm[87333]: /rc.newwanip: ROUTING: setting default route to 192.168.100.1 Feb 11 02:13:12 fw1 php-fpm[87333]: /rc.newwanip: Removing static route for monitor 84.192.***.***and adding a new route through 192.168.100.1 Feb 11 02:13:32 fw1 check_reload_status: updating dyndns WAN_DHCP Feb 11 02:13:32 fw1 check_reload_status: Restarting ipsec tunnels Feb 11 02:13:32 fw1 check_reload_status: Restarting OpenVPN tunnels/interfaces Feb 11 02:13:32 fw1 check_reload_status: Reloading filter Feb 11 02:13:33 fw1 php-fpm[15216]: /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP. Feb 11 02:13:33 fw1 php-fpm[15216]: /rc.openvpn: OpenVPN: Resync server1 OpenVPN Server Feb 11 02:13:33 fw1 kernel: ovpns1: link state changed to DOWN Feb 11 02:13:33 fw1 check_reload_status: Reloading filter Feb 11 02:13:34 fw1 kernel: ovpns1: link state changed to UP Feb 11 02:13:34 fw1 check_reload_status: rc.newwanip starting ovpns1 Feb 11 02:13:35 fw1 php-fpm[15216]: /rc.filter_configure_sync: Could not find IPv6 Feb 11 02:13:39 fw1 check_reload_status: Syncing firewall Feb 11 02:13:39 fw1 php-fpm[19772]: /rc.newwanip: rc.newwanip: on (IP address: 192.168.10.1) (interface: []) (real interface: ovpns1). Feb 11 02:13:39 fw1 check_reload_status: Reloading filter Feb 11 02:13:39 fw1 php-fpm[19772]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 192.168.10.1 - Restarting packages. Feb 11 02:13:39 fw1 check_reload_status: Starting packages Feb 11 02:13:39 fw1 check_reload_status: Syncing firewall Feb 11 02:13:39 fw1 php-fpm[87333]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 84.192.***.***-> 192.168.100.10 - Restarting packages. Feb 11 02:13:40 fw1 php-fpm[70546]: /rc.start_packages: Restarting/Starting all packages. Feb 11 02:13:40 fw1 php-fpm[70546]: /rc.start_packages: Reloading Squid for configuration sync Feb 11 02:13:41 fw1 check_reload_status: Reloading filter Feb 11 02:13:41 fw1 php: lcdproc: Start client procedure. Error counter: (0) Feb 11 02:13:41 fw1 check_reload_status: Syncing firewall Feb 11 02:13:42 fw1 php-fpm[19772]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (wan). Feb 11 02:13:44 fw1 php-fpm[87333]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (wan). Feb 11 02:13:44 fw1 php: lcdproc: Start client procedure. Error counter: (0) Feb 11 02:13:55 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:13:55 fw1 php: lcdproc: Start client procedure. Error counter: (1) Feb 11 02:14:06 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:14:06 fw1 php: lcdproc: Start client procedure. Error counter: (2) Feb 11 02:14:11 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:14:12 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:14:13 fw1 check_reload_status: rc.newwanip starting sk0 Feb 11 02:14:14 fw1 php-fpm[80275]: /rc.newwanip: rc.newwanip: Info: starting on sk0. Feb 11 02:14:14 fw1 php-fpm[80275]: /rc.newwanip: rc.newwanip: on (IP address: 192.168.100.10) (interface: WAN[wan]) (real interface: sk0). Feb 11 02:14:15 fw1 php-fpm[80275]: /rc.newwanip: Could not find IPv6 gateway for interface (wan). Feb 11 02:14:16 fw1 php-fpm[80275]: /rc.newwanip: ROUTING: setting default route to 192.168.100.1 Feb 11 02:14:16 fw1 php-fpm[80275]: /rc.newwanip: Removing static route for monitor 84.192.***.*** and adding a new route through 192.168.100.1 Feb 11 02:14:17 fw1 php-fpm[80275]: /rc.newwanip: Dyndns debug information (tiet.no-ip.biz): Could not resolve checkip.dyndns.org to IP using interface IP 192.168.100.10. Feb 11 02:14:17 fw1 php-fpm[80275]: /rc.newwanip: DynDNS (*****-ip.biz) There was an error trying to determine the public IP for interface - wan(sk0). Probably interface is not a WAN interface. Feb 11 02:14:17 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:14:17 fw1 php: lcdproc: Start client procedure. Error counter: (3) Feb 11 02:14:18 fw1 php-fpm[80275]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN. Feb 11 02:14:18 fw1 kernel: ovpns1: link state changed to DOWN Feb 11 02:14:18 fw1 check_reload_status: Reloading filter Feb 11 02:14:18 fw1 check_reload_status: Reloading filter Feb 11 02:14:18 fw1 php-fpm[80275]: /rc.newwanip: Creating rrd update script Feb 11 02:14:18 fw1 kernel: ovpns1: link state changed to UP Feb 11 02:14:18 fw1 check_reload_status: rc.newwanip starting ovpns1 Feb 11 02:14:20 fw1 php-fpm[35317]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (wan). Feb 11 02:14:20 fw1 php-fpm[80275]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 192.168.100.10 -> 192.168.100.10 - Restarting packages. Feb 11 02:14:20 fw1 check_reload_status: Starting packages Feb 11 02:14:20 fw1 php-fpm[80275]: /rc.newwanip: rc.newwanip: Info: starting on ovpns1. Feb 11 02:14:20 fw1 php-fpm[80275]: /rc.newwanip: rc.newwanip: on (IP address: 192.168.10.1) (interface: []) (real interface: ovpns1). Feb 11 02:14:20 fw1 check_reload_status: Reloading filter Feb 11 02:14:20 fw1 php-fpm[80275]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 192.168.10.1 - Restarting packages. Feb 11 02:14:20 fw1 check_reload_status: Starting packages Feb 11 02:14:22 fw1 php-fpm[80275]: /rc.start_packages: Restarting/Starting all packages. Feb 11 02:14:22 fw1 php-fpm[35317]: /rc.start_packages: Skipping STARTing packages process because previous/another instance is already running Feb 11 02:14:22 fw1 php-fpm[80275]: /rc.start_packages: Reloading Squid for configuration sync Feb 11 02:14:23 fw1 php-fpm[67206]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (wan). Feb 11 02:14:23 fw1 check_reload_status: Syncing firewall Feb 11 02:14:23 fw1 check_reload_status: Syncing firewall Feb 11 02:14:25 fw1 php: lcdproc: Start client procedure. Error counter: (0) Feb 11 02:14:36 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:14:36 fw1 php: lcdproc: Start client procedure. Error counter: (1) Feb 11 02:14:47 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:14:47 fw1 php: lcdproc: Start client procedure. Error counter: (2) Feb 11 02:14:58 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:14:58 fw1 php: lcdproc: Start client procedure. Error counter: (3) Feb 11 02:15:09 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:15:09 fw1 php: lcdproc: Too many errors, the client ends. Feb 11 02:15:14 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:15:15 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:15:16 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:15:17 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:15:18 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:15:19 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:15:20 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:15:20 fw1 check_reload_status: rc.newwanip starting sk0 Feb 11 02:15:21 fw1 kernel: arpresolve: can't allocate llinfo for 192.168.100.1 on sk0 Feb 11 02:15:21 fw1 php-fpm[67206]: /rc.newwanip: rc.newwanip: Info: starting on sk0. Feb 11 02:15:21 fw1 php-fpm[67206]: /rc.newwanip: rc.newwanip: on (IP address: 84.192.***.***) (interface: WAN[wan]) (real interface: sk0). Feb 11 02:15:21 fw1 php-fpm[67206]: /rc.newwanip: IP has changed, killing states on former IP 192.168.100.10. Feb 11 02:15:23 fw1 php-fpm[67206]: /rc.newwanip: Could not find IPv6 gateway for interface (wan). Feb 11 02:15:23 fw1 php-fpm[67206]: /rc.newwanip: ROUTING: setting default route to 84.192.***.*** Feb 11 02:15:25 fw1 php-fpm[67206]: /rc.newwanip: phpDynDNS (*****-ip.biz): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Feb 11 02:15:26 fw1 php-fpm[67206]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN. Feb 11 02:15:26 fw1 kernel: ovpns1: link state changed to DOWN Feb 11 02:15:26 fw1 check_reload_status: Reloading filter Feb 11 02:15:26 fw1 check_reload_status: Reloading filter Feb 11 02:15:26 fw1 php-fpm[67206]: /rc.newwanip: Creating rrd update script Feb 11 02:15:26 fw1 kernel: ovpns1: link state changed to UP Feb 11 02:15:26 fw1 check_reload_status: rc.newwanip starting ovpns1 Feb 11 02:15:28 fw1 php-fpm[56235]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (wan). Feb 11 02:15:28 fw1 php-fpm[67206]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 192.168.100.10 -> 84.192.***.*** - Restarting packages. Feb 11 02:15:28 fw1 check_reload_status: Starting packages Feb 11 02:15:28 fw1 php-fpm[67206]: /rc.newwanip: rc.newwanip: Info: starting on ovpns1. Feb 11 02:15:28 fw1 php-fpm[67206]: /rc.newwanip: rc.newwanip: on (IP address: 192.168.10.1) (interface: []) (real interface: ovpns1). Feb 11 02:15:28 fw1 check_reload_status: Reloading filter Feb 11 02:15:28 fw1 php-fpm[67206]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 192.168.10.1 - Restarting packages. Feb 11 02:15:28 fw1 check_reload_status: Starting packages Feb 11 02:15:29 fw1 php-fpm[67206]: /rc.start_packages: Restarting/Starting all packages. Feb 11 02:15:29 fw1 php-fpm[56235]: /rc.start_packages: Skipping STARTing packages process because previous/another instance is already running Feb 11 02:15:30 fw1 php-fpm[67206]: /rc.start_packages: Reloading Squid for configuration sync Feb 11 02:15:30 fw1 php-fpm[86166]: /rc.filter_configure_sync: Could not find IPv6 gateway for interface (wan). Feb 11 02:15:31 fw1 check_reload_status: Syncing firewall Feb 11 02:15:31 fw1 check_reload_status: Syncing firewall Feb 11 02:15:32 fw1 php: lcdproc: Start client procedure. Error counter: (0) Feb 11 02:15:43 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:15:43 fw1 php: lcdproc: Start client procedure. Error counter: (1) Feb 11 02:15:54 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:15:54 fw1 php: lcdproc: Start client procedure. Error counter: (2) Feb 11 02:16:05 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:16:05 fw1 php: lcdproc: Start client procedure. Error counter: (3) Feb 11 02:16:16 fw1 php: lcdproc: Failed to connect to LCDd process Operation timed out (60) Feb 11 02:16:16 fw1 php: lcdproc: Too many errors, the client ends. Feb 11 07:04:06 fw1 php-fpm[73586]: /index.php: Session timed out for user 'admin'
-
A private IP on your WAN is usually caused by a cable modem that's lost its upstream connection. Could that be happening here. Is the private address being assigned in any of your internal subnets?
Steve
-
Is your modem rebooting? That might explain both symptoms.
-
Don't know , but that makes sense in a way… :)
On the other hand it's giving me 2 different IP's. -
You mean 192.168.10.1 and 192.168.100.10?
You aren't using either of those subnets (/24) internally?Steve
-
Nope :-)
The 192.168.10.0/24 range is used by openvpn for vpn access
-
Ok, so you could have something handing out address in that subnet. The 100.1 address is totally spurious, not used as a tunnel address etc?
Steve
-
Nope not used.
But the WAN link seems stable for now.
I made some changes in the System general settings and they seem to work. -
What did you change?
Steve
-
I have the same issue and posted some logs in another thread yesterday. I've seen a few posts like this recently do possibly something needs tweaking. I'll add a link to my logs tomorrow when I get back to my desk. I've tried setting static address, disabling monitoring but every 3-7 days my line drops and needs s reboot of wan or box to resolve.
Edit: here's the link to my logs… https://forum.pfsense.org/index.php?topic=88236.msg491251#msg491251
-
In System -> general settings i added 127.0.0.1 as first dns server without any gateway
then on the second line:
8.8.8.8 with the gateway from my provider 84.192.192.1
on the third:
8.8.4.4 with 84.192.192.1the option to automatically find the gateway seems to f*ck the wan link sometimes…
I didn't make any changes otherwise to the network config.
Only updated LCDProc and used your guide to set it up because it kept crashing badly.
-
Interesting, thanks.
-
Hi,
Are you sure this is resolved? I find it strange that changing dns settings solves this.
Looking at your IP gateway, we're using the same internet provider and I'm having the same issue. (only updated yesterday, noticed the link going down today)
I had a similar issue a few years back: https://forum.pfsense.org/index.php?topic=51420.0
It was solved by using the 2.1 release. But now the problem returned after updating to 2.2.
I'll keep an eye on it (it only happened once now) and will collect logs in case it returns.
Regards,
Kristof. -
Okay, just had this happen again, here's the logs from that time.
All VPNs going down..
Feb 25 13:53:01 pfsense kernel: ovpns23: link state changed to DOWN Feb 25 13:53:01 pfsense php-fpm[89891]: /rc.newwanip: rc.newwanip: Info: starting on ovpns19. Feb 25 13:53:01 pfsense php-fpm[89891]: /rc.newwanip: rc.newwanip: on (IP address: 192.168.250.17) (interface: []) (real interface: ovpns19). Feb 25 13:53:01 pfsense php-fpm[89891]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 192.168.250.17 - Restarting packages. Feb 25 13:53:01 pfsense check_reload_status: Starting packages Feb 25 13:53:02 pfsense php-fpm[21603]: /rc.openvpn: OpenVPN: Resync server24 [CUST26] S2S Feb 25 13:53:02 pfsense kernel: ovpns23: link state changed to UP Feb 25 13:53:02 pfsense check_reload_status: rc.newwanip starting ovpns23 Feb 25 13:53:02 pfsense kernel: ovpns24: link state changed to DOWN Feb 25 13:53:02 pfsense check_reload_status: Reloading filter
Our WAN interface.. (em1)
Feb 25 13:53:02 pfsense kernel: em1: Watchdog timeout -- resetting Feb 25 13:53:02 pfsense kernel: em1: Queue(0) tdh = 801, hw tdt = 770 Feb 25 13:53:02 pfsense kernel: em1: TX(0) desc avail = 31,Next TX to Clean = 801
The rest of the VPNs going down. (we've got 30 or so)
Feb 25 13:53:02 pfsense php-fpm[21603]: /rc.openvpn: OpenVPN: Resync server25 [CUST25] S2S Feb 25 13:53:02 pfsense kernel: ovpns24: link state changed to UP Feb 25 13:53:02 pfsense check_reload_status: rc.newwanip starting ovpns24 Feb 25 13:53:02 pfsense kernel: ovpns25: link state changed to DOWN Feb 25 13:53:02 pfsense kernel: em1: link state changed to DOWN Feb 25 13:53:02 pfsense check_reload_status: Linkup starting em1 Feb 25 13:53:02 pfsense php-fpm[21603]: /rc.openvpn: OpenVPN: Resync server26 [CUST26] S2S Feb 25 13:53:02 pfsense kernel: ovpns25: link state changed to UP Feb 25 13:53:02 pfsense check_reload_status: rc.newwanip starting ovpns25 Feb 25 13:53:02 pfsense kernel: ovpns26: link state changed to DOWN Feb 25 13:53:03 pfsense php-fpm[21603]: /rc.linkup: DEVD Ethernet detached event for wan Feb 25 13:53:04 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 25 13:53:04 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 25 13:53:04 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 25 13:53:04 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1
And then it continues with the can't allocate llinfo.
I tried disconnecting and re-connecting the network cable: no result.
I tried rebooting the cable modem: no result.It did try getting a new DHCP address:
$ tcpdump -i em1 -n 14:04:25.808903 IP 81.82.209.44.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:0d:88:cc:fa:2f, length 300 14:04:27.003202 IP 81.82.209.44.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:0d:88:cc:fa:2f, length 300 14:04:31.003925 IP 81.82.209.44.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:0d:88:cc:fa:2f, length 300 14:04:40.009358 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:0d:88:cc:fa:2f, length 300 14:04:41.009469 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:0d:88:cc:fa:2f, length 300 14:04:42.000206 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:0d:88:cc:fa:2f, length 300 14:04:44.001156 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:0d:88:cc:fa:2f, length 300
But that's it, never got any.
Final step we took (because it was taking us +5 minutes already) was reboot the pfsense. After the reboot connection was good..
I will probably take a 2.1.5 to temporarily switch this one out (luckily it's in our building) but will leave the 2.2 in place so I can switch easily.
So I there's anything I can/should try, let me know, no problem.
Regards,
Kristof -
Id love to share with you a solution but Im seeing exactly the same thing here and can confirm its not fixed for me by manipulating DNS as TieT suggested fixed his. My issue is triggered by my ISP WAN dhcp address expiring every seven days. Rebooting the firewall is the easiest way to fix it, I spent a couple of hours last time it went down capturing packets and what not and it appears in my situation to be related to DHCP queries not syncing. It could be specific to my ISP and I'll keep capturing and investigating if/when it continues. Thanks for sharing your logs.
-
I've put my 2.1.5 back in place. let's see how this goes. (I'm pretty sure this will stay up)
I also suspect it has something to do with the dhcp renewal.
2.1 solved it in 2012 for me :-) - https://forum.pfsense.org/index.php?topic=51420.0
-
Feb 25 13:53:02 pfsense kernel: em1: Watchdog timeout -- resetting
That is a pretty bad sign.
You might try looking at sysctl dev.em.1
There may be some error couters there that show something nasty happened.Steve
-
Ok, put a switch in between modem<->pfsense , recorded the current state of sysctl dev.em.1 (attached)
Currently it's good :o (lol) but if the problem returns I'll redo the sysctl dev.em.1 and won't reboot. (I'll just use a 2.1.5 device so can grab any logs needed of the 2.2)
Thanks, I'll keep you posted.
Kristof
-
I'm getting the same issue.
Seems my external IPv4, which is set by DHCP, is randomly disconnecting while the IPv6 address is fine.
No clue what's the cause, but I'm running the latest version on a Nano USB.
-
Ok, not resolved yet :(
My current configuration is Modem <-> Switch <-> pfsense.
I did see some bad things in the sysctl.em.1 :
dev.em.1.mac_stats.collision_count: 0 dev.em.1.mac_stats.symbol_errors: 4294967295 dev.em.1.mac_stats.sequence_errors: 0 dev.em.1.mac_stats.missed_packets: 28403
My log was already full (I will make it bigger or use remote syslog) but I did see this:
.. first a ton of these messages: Feb 28 11:21:04 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 28 11:21:04 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 28 11:21:04 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 28 11:21:05 pfsense kernel: em1: Watchdog timeout -- resetting Feb 28 11:21:05 pfsense kernel: em1: Queue(0) tdh = 0, hw tdt = 993 Feb 28 11:21:05 pfsense kernel: em1: TX(0) desc avail = 31,Next TX to Clean = 0 Feb 28 11:21:05 pfsense kernel: em1: link state changed to DOWN Feb 28 11:21:05 pfsense check_reload_status: Linkup starting em1 Feb 28 11:21:06 pfsense php-fpm[43360]: /rc.linkup: DEVD Ethernet detached event for wan Feb 28 11:21:06 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 28 11:21:06 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 28 11:21:09 pfsense check_reload_status: Linkup starting em1 Feb 28 11:21:09 pfsense kernel: em1: link state changed to UP Feb 28 11:21:10 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 28 11:21:10 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 28 11:21:11 pfsense php-fpm[43360]: /rc.linkup: Shutting down Router Advertisment daemon cleanly Feb 28 11:21:11 pfsense php-fpm[54180]: /rc.linkup: DEVD Ethernet attached event for wan Feb 28 11:21:11 pfsense php-fpm[54180]: /rc.linkup: HOTPLUG: Configuring interface wan Feb 28 11:21:13 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 28 11:21:13 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 Feb 28 11:21:14 pfsense kernel: arpresolve: can't allocate llinfo for 81.82.192.1 on em1 .. and then more of these
This only happens on 2.2, the same configuration with 2.1.5, no problem. It took about 30 hours before it occured the first time.
Anything else I might try to resolve/further pinpoint this?
Regards,
Kristof. -
New hardware?
-
-
@man:
arpresolve: can't allocate llinfo for %d.%d.%d.%d The route for the ref-
erenced host points to a device upon which ARP is required, but ARP was
unable to allocate a routing table entry in which to store the host's MAC
address. This usually points to a misconfigured routing table. It can
also occur if the kernel cannot allocate memory.What's in netstat -rn when it's failing?
-
Just resolved my issue by disabling the local DNS resolution. Changed it to just use Google's DNS servers instead (8.8.8.8, 8.8.4.4) and it's working fine.
Not sure if it's the exact same issue as what everyone else has had, but that's what did it for me.
-
Test it for a few days, it worked for me for a few days, but then the gateway was lost again.
It seems when I create a vpn to my fw the issue begins…Just resolved my issue by disabling the local DNS resolution. Changed it to just use Google's DNS servers instead (8.8.8.8, 8.8.4.4) and it's working fine.
Not sure if it's the exact same issue as what everyone else has had, but that's what did it for me.
-
@man:
arpresolve: can't allocate llinfo for %d.%d.%d.%d The route for the ref-
erenced host points to a device upon which ARP is required, but ARP was
unable to allocate a routing table entry in which to store the host's MAC
address. This usually points to a misconfigured routing table. It can
also occur if the kernel cannot allocate memory.What's in netstat -rn when it's failing?
Tnx for the suggestion, will try this and the sysctl dev.em.1 on next problem.
I'm waiting for it to occur again (it can take several hours but it might as well take 48 hours) and report back.
Kristof
-
I find it odd that in both cases here the actual physical link is going down. If that was a modem issue I would expect the switch to as least show a different result. I guess in a watchdog time-out scenario the link is probably brought down when the NIC is reset. Hmmm.
Steve
-
I had my line go down this morning at 6:50am. I spent a couple of hours over breakfast debugging but this is beyond my level of knowledge to understand fully whats happening. I captured a ton of log material and tcpdump of my WAN interface during the initial outage and also during various reboots of pfsense and later my modem. I'll try and summarise this here but if needed I can zip this lot up and pass eton to a dev or something if that helps? Excuse me if I break this into a few posts to make it easy to write.
Notes
I'm using virgin media in the UK with their Superhub in modem mode
igb4 is my WAN port and I've set its MAC address to 00:22:68:1f:74:81.this is what i recorded when I got to my desk and found the WAN had dropped.
[2.2-RELEASE][admin@pfsense.irj972.local.lan]/var/log: netstat -rn Routing tables Internet: Destination Gateway Flags Netif Expire 0.0.0.0/8 link#7 U igb4 127.0.0.1 link#13 UH lo0 192.168.10.0/24 link#1 U igb0 192.168.10.1 link#1 UHS lo0 192.168.50.0/24 link#8 U igb5 192.168.50.1 link#8 UHS lo0 192.168.60.0/24 link#9 U igb6 192.168.60.1 link#9 UHS lo0 192.168.70.0/24 link#10 U igb7 192.168.70.1 link#10 UHS lo0 192.168.101.0/24 link#5 U ix0 192.168.101.1 link#5 UHS lo0 192.168.110.0/24 link#6 U ix1 192.168.110.1 link#6 UHS lo0 192.168.210.0/24 192.168.210.2 UGS ovpns2 192.168.210.1 link#15 UHS lo0 192.168.210.2 link#15 UH ovpns2 Internet6: Destination Gateway Flags Netif Expire ::1 link#13 UH lo0 fe80::%igb0/64 link#1 U igb0 fe80::a236:9fff:fe2d:eba8%igb0 link#1 UHS lo0 fe80::%ix0/64 link#5 U ix0 fe80::92e2:baff:fe37:fba4%ix0 link#5 UHS lo0 fe80::%ix1/64 link#6 U ix1 fe80::92e2:baff:fe37:fba5%ix1 link#6 UHS lo0 fe80::%igb4/64 link#7 U igb4 fe80::222:68ff:fe1f:7481%igb4 link#7 UHS lo0 fe80::%igb5/64 link#8 U igb5 fe80::ec4:7aff:fe07:d551%igb5 link#8 UHS lo0 fe80::%igb6/64 link#9 U igb6 fe80::ec4:7aff:fe07:d552%igb6 link#9 UHS lo0 fe80::%igb7/64 link#10 U igb7 fe80::ec4:7aff:fe07:d553%igb7 link#10 UHS lo0 fe80::%lo0/64 link#13 U lo0 fe80::1%lo0 link#13 UHS lo0 fe80::a236:9fff:fe2d:eba8%ovpns2 link#15 UHS lo0 ff01::%igb0/32 fe80::a236:9fff:fe2d:eba8%igb0 U igb0 ff01::%ix0/32 fe80::92e2:baff:fe37:fba4%ix0 U ix0 ff01::%ix1/32 fe80::92e2:baff:fe37:fba5%ix1 U ix1 ff01::%igb4/32 fe80::222:68ff:fe1f:7481%igb4 U igb4 ff01::%igb5/32 fe80::ec4:7aff:fe07:d551%igb5 U igb5 ff01::%igb6/32 fe80::ec4:7aff:fe07:d552%igb6 U igb6 ff01::%igb7/32 fe80::ec4:7aff:fe07:d553%igb7 U igb7 ff01::%lo0/32 ::1 U lo0 ff01::%ovpns2/32 fe80::a236:9fff:fe2d:eba8%ovpns2 U ovpns2 ff02::%igb0/32 fe80::a236:9fff:fe2d:eba8%igb0 U igb0 ff02::%ix0/32 fe80::92e2:baff:fe37:fba4%ix0 U ix0 ff02::%ix1/32 fe80::92e2:baff:fe37:fba5%ix1 U ix1 ff02::%igb4/32 fe80::222:68ff:fe1f:7481%igb4 U igb4 ff02::%igb5/32 fe80::ec4:7aff:fe07:d551%igb5 U igb5 ff02::%igb6/32 fe80::ec4:7aff:fe07:d552%igb6 U igb6 ff02::%igb7/32 fe80::ec4:7aff:fe07:d553%igb7 U igb7 ff02::%lo0/32 ::1 U lo0 ff02::%ovpns2/32 fe80::a236:9fff:fe2d:eba8%ovpns2 U ovpns2
and this is the dhcpd.log which is basically full of repeating sections of this
Same old pattern Ive been seeing for a few weeks now which I think syncs with my virgin modems 7 day DHCP expiryMar 2 08:29:25 pfsense dhclient: PREINIT Mar 2 08:29:25 pfsense dhclient: EXPIRE Mar 2 08:29:25 pfsense dhclient: Deleting old routes Mar 2 08:29:25 pfsense dhclient: PREINIT Mar 2 08:29:25 pfsense dhclient[25776]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 2 Mar 2 08:29:25 pfsense dhclient[25776]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 4 Mar 2 08:29:29 pfsense dhclient[25776]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 7 Mar 2 08:29:36 pfsense dhclient[25776]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 14 Mar 2 08:29:50 pfsense dhclient[25776]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 17 Mar 2 08:30:07 pfsense dhclient[25776]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 14 Mar 2 08:30:21 pfsense dhclient[25776]: No DHCPOFFERS received. Mar 2 08:30:21 pfsense dhclient[25776]: Trying recorded lease 86.5.163.20 Mar 2 08:30:21 pfsense dhclient: TIMEOUT Mar 2 08:30:21 pfsense dhclient: Starting add_new_address() Mar 2 08:30:21 pfsense dhclient: ifconfig igb4 inet 86.5.163.20 netmask 255.255.255.0 broadcast 255.255.255.255 Mar 2 08:30:21 pfsense dhclient: New IP Address (igb4): 86.5.163.20 Mar 2 08:30:21 pfsense dhclient: New Subnet Mask (igb4): 255.255.255.0 Mar 2 08:30:21 pfsense dhclient: New Broadcast Address (igb4): 255.255.255.255 Mar 2 08:30:21 pfsense dhclient: New Routers (igb4): 86.5.163.1 Mar 2 08:30:22 pfsense dhclient: New Routers (igb4): 86.5.163.1 Mar 2 08:30:23 pfsense dhclient: Deleting old routes Mar 2 08:30:23 pfsense dhclient[25776]: bound: renewal in 244840 seconds. Mar 2 08:30:26 pfsense dhcpd: Received signal 15, initiating shutdown. Mar 2 08:30:26 pfsense dhcpd: Disabling output on BPF/igb0/a0:36:9f:2d:eb:a8/192.168.10.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling input on BPF/igb0/a0:36:9f:2d:eb:a8/192.168.10.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling output on BPF/ix1/90:e2:ba:37:fb:a5/192.168.110.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling input on BPF/ix1/90:e2:ba:37:fb:a5/192.168.110.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling output on BPF/ix0/90:e2:ba:37:fb:a4/192.168.101.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling input on BPF/ix0/90:e2:ba:37:fb:a4/192.168.101.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling output on BPF/igb6/0c:c4:7a:07:d5:52/192.168.60.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling input on BPF/igb6/0c:c4:7a:07:d5:52/192.168.60.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling output on BPF/igb7/0c:c4:7a:07:d5:53/192.168.70.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling input on BPF/igb7/0c:c4:7a:07:d5:53/192.168.70.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling output on BPF/igb5/0c:c4:7a:07:d5:51/192.168.50.0/24 Mar 2 08:30:26 pfsense dhcpd: Disabling input on BPF/igb5/0c:c4:7a:07:d5:51/192.168.50.0/24 Mar 2 08:30:27 pfsense dhcpd: Internet Systems Consortium DHCP Server 4.2.6 Mar 2 08:30:27 pfsense dhcpd: Copyright 2004-2014 Internet Systems Consortium. Mar 2 08:30:27 pfsense dhcpd: All rights reserved. Mar 2 08:30:27 pfsense dhcpd: For info, please visit https://www.isc.org/software/dhcp/ Mar 2 08:30:27 pfsense dhcpd: Internet Systems Consortium DHCP Server 4.2.6 Mar 2 08:30:27 pfsense dhcpd: Copyright 2004-2014 Internet Systems Consortium. Mar 2 08:30:27 pfsense dhcpd: All rights reserved. Mar 2 08:30:27 pfsense dhcpd: For info, please visit https://www.isc.org/software/dhcp/ Mar 2 08:30:27 pfsense dhcpd: Wrote 0 deleted host decls to leases file. Mar 2 08:30:27 pfsense dhcpd: Wrote 0 new dynamic host decls to leases file. Mar 2 08:30:27 pfsense dhcpd: Wrote 45 leases to leases file. Mar 2 08:30:27 pfsense dhcpd: Listening on BPF/igb0/a0:36:9f:2d:eb:a8/192.168.10.0/24 Mar 2 08:30:27 pfsense dhcpd: Sending on BPF/igb0/a0:36:9f:2d:eb:a8/192.168.10.0/24 Mar 2 08:30:27 pfsense dhcpd: Listening on BPF/ix1/90:e2:ba:37:fb:a5/192.168.110.0/24 Mar 2 08:30:27 pfsense dhcpd: Sending on BPF/ix1/90:e2:ba:37:fb:a5/192.168.110.0/24 Mar 2 08:30:27 pfsense dhcpd: Listening on BPF/ix0/90:e2:ba:37:fb:a4/192.168.101.0/24 Mar 2 08:30:27 pfsense dhcpd: Sending on BPF/ix0/90:e2:ba:37:fb:a4/192.168.101.0/24 Mar 2 08:30:27 pfsense dhcpd: Listening on BPF/igb6/0c:c4:7a:07:d5:52/192.168.60.0/24 Mar 2 08:30:27 pfsense dhcpd: Sending on BPF/igb6/0c:c4:7a:07:d5:52/192.168.60.0/24 Mar 2 08:30:27 pfsense dhcpd: Listening on BPF/igb7/0c:c4:7a:07:d5:53/192.168.70.0/24 Mar 2 08:30:27 pfsense dhcpd: Sending on BPF/igb7/0c:c4:7a:07:d5:53/192.168.70.0/24 Mar 2 08:30:27 pfsense dhcpd: Listening on BPF/igb5/0c:c4:7a:07:d5:51/192.168.50.0/24 Mar 2 08:30:27 pfsense dhcpd: Sending on BPF/igb5/0c:c4:7a:07:d5:51/192.168.50.0/24 Mar 2 08:30:27 pfsense dhcpd: Sending on Socket/fallback/fallback-net Mar 2 08:30:27 pfsense dhclient[26757]: connection closed Mar 2 08:30:27 pfsense dhclient[26757]: exiting.
sysctl igb.4 related info. Nothing that jumped out to me like excessive interrupts, packet collisions etc….just looks like a dead NIC
dev.igb.4.%desc: Intel(R) PRO/1000 Network Connection version - 2.4.0 dev.igb.4.%driver: igb dev.igb.4.%location: slot=20 function=0 handle=\_SB_.PCI0.D01B dev.igb.4.%pnpinfo: vendor=0x8086 device=0x1f41 subvendor=0x15d9 subdevice=0x1f41 class=0x020000 dev.igb.4.%parent: pci0 dev.igb.4.nvm: -1 dev.igb.4.enable_aim: 1 dev.igb.4.fc: 3 dev.igb.4.rx_processing_limit: 100 dev.igb.4.dmac: 0 dev.igb.4.eee_disabled: 0 dev.igb.4.link_irq: 403 dev.igb.4.dropped: 0 dev.igb.4.tx_dma_fail: 0 dev.igb.4.rx_overruns: 0 dev.igb.4.watchdog_timeouts: 0 dev.igb.4.device_control: 1477706305 dev.igb.4.rx_control: 67141634 dev.igb.4.interrupt_mask: 4 dev.igb.4.extended_int_mask: 2147484159 dev.igb.4.tx_buf_alloc: 0 dev.igb.4.rx_buf_alloc: 0 dev.igb.4.fc_high_water: 33168 dev.igb.4.fc_low_water: 33152 dev.igb.4.queue0.no_desc_avail: 0 dev.igb.4.queue0.tx_packets: 2546184 dev.igb.4.queue0.rx_packets: 202589 dev.igb.4.queue0.rx_bytes: 0 dev.igb.4.queue0.lro_queued: 0 dev.igb.4.queue0.lro_flushed: 0 dev.igb.4.queue1.no_desc_avail: 0 dev.igb.4.queue1.tx_packets: 3569953 dev.igb.4.queue1.rx_packets: 122795 dev.igb.4.queue1.rx_bytes: 0 dev.igb.4.queue1.lro_queued: 0 dev.igb.4.queue1.lro_flushed: 0 dev.igb.4.queue2.no_desc_avail: 0 dev.igb.4.queue2.tx_packets: 995675 dev.igb.4.queue2.rx_packets: 31067808 dev.igb.4.queue2.rx_bytes: 0 dev.igb.4.queue2.lro_queued: 0 dev.igb.4.queue2.lro_flushed: 0 dev.igb.4.queue3.no_desc_avail: 0 dev.igb.4.queue3.tx_packets: 3674232 dev.igb.4.queue3.rx_packets: 88985 dev.igb.4.queue3.rx_bytes: 0 dev.igb.4.queue3.lro_queued: 0 dev.igb.4.queue3.lro_flushed: 0 dev.igb.4.queue4.no_desc_avail: 0 dev.igb.4.queue4.tx_packets: 4509460 dev.igb.4.queue4.rx_packets: 80735 dev.igb.4.queue4.rx_bytes: 0 dev.igb.4.queue4.lro_queued: 0 dev.igb.4.queue4.lro_flushed: 0 dev.igb.4.queue5.no_desc_avail: 0 dev.igb.4.queue5.tx_packets: 3015023 dev.igb.4.queue5.rx_packets: 95970 dev.igb.4.queue5.rx_bytes: 0 dev.igb.4.queue5.lro_queued: 0 dev.igb.4.queue5.lro_flushed: 0 dev.igb.4.queue6.no_desc_avail: 0 dev.igb.4.queue6.tx_packets: 2100212 dev.igb.4.queue6.rx_packets: 114919 dev.igb.4.queue6.rx_bytes: 0 dev.igb.4.queue6.lro_queued: 0 dev.igb.4.queue6.lro_flushed: 0 dev.igb.4.queue7.no_desc_avail: 0 dev.igb.4.queue7.tx_packets: 4101655 dev.igb.4.queue7.rx_packets: 105344 dev.igb.4.queue7.rx_bytes: 0 dev.igb.4.queue7.lro_queued: 0 dev.igb.4.queue7.lro_flushed: 0 dev.igb.4.mac_stats.excess_coll: 0 dev.igb.4.mac_stats.single_coll: 0 dev.igb.4.mac_stats.multiple_coll: 0 dev.igb.4.mac_stats.late_coll: 0 dev.igb.4.mac_stats.collision_count: 0 dev.igb.4.mac_stats.symbol_errors: 0 dev.igb.4.mac_stats.sequence_errors: 0 dev.igb.4.mac_stats.defer_count: 0 dev.igb.4.mac_stats.missed_packets: 0 dev.igb.4.mac_stats.recv_no_buff: 0 dev.igb.4.mac_stats.recv_undersize: 0 dev.igb.4.mac_stats.recv_fragmented: 0 dev.igb.4.mac_stats.recv_oversize: 0 dev.igb.4.mac_stats.recv_jabber: 0 dev.igb.4.mac_stats.recv_errs: 0 dev.igb.4.mac_stats.crc_errs: 0 dev.igb.4.mac_stats.alignment_errs: 0 dev.igb.4.mac_stats.coll_ext_errs: 0 dev.igb.4.mac_stats.xon_recvd: 0 dev.igb.4.mac_stats.xon_txd: 0 dev.igb.4.mac_stats.xoff_recvd: 0 dev.igb.4.mac_stats.xoff_txd: 0 dev.igb.4.mac_stats.total_pkts_recvd: 31875273 dev.igb.4.mac_stats.good_pkts_recvd: 31875273 dev.igb.4.mac_stats.bcast_pkts_recvd: 183 dev.igb.4.mac_stats.mcast_pkts_recvd: 0 dev.igb.4.mac_stats.rx_frames_64: 56499 dev.igb.4.mac_stats.rx_frames_65_127: 118706 dev.igb.4.mac_stats.rx_frames_128_255: 4268691 dev.igb.4.mac_stats.rx_frames_256_511: 308408 dev.igb.4.mac_stats.rx_frames_512_1023: 456179 dev.igb.4.mac_stats.rx_frames_1024_1522: 26666790 dev.igb.4.mac_stats.good_octets_recvd: 40798660342 dev.igb.4.mac_stats.good_octets_txd: 11294455422 dev.igb.4.mac_stats.total_pkts_txd: 24517003 dev.igb.4.mac_stats.good_pkts_txd: 24517003 dev.igb.4.mac_stats.bcast_pkts_txd: 6105 dev.igb.4.mac_stats.mcast_pkts_txd: 443 dev.igb.4.mac_stats.tx_frames_64: 282372 dev.igb.4.mac_stats.tx_frames_65_127: 291318 dev.igb.4.mac_stats.tx_frames_128_255: 17645482 dev.igb.4.mac_stats.tx_frames_256_511: 373380 dev.igb.4.mac_stats.tx_frames_512_1023: 166562 dev.igb.4.mac_stats.tx_frames_1024_1522: 5757889 dev.igb.4.mac_stats.tso_txd: 0 dev.igb.4.mac_stats.tso_ctx_fail: 0 dev.igb.4.interrupts.asserts: 48596954 dev.igb.4.interrupts.rx_pkt_timer: 31874824 dev.igb.4.interrupts.rx_abs_timer: 0 dev.igb.4.interrupts.tx_pkt_timer: 0 dev.igb.4.interrupts.tx_abs_timer: 0 dev.igb.4.interrupts.tx_queue_empty: 24512231 dev.igb.4.interrupts.tx_queue_min_thresh: 34287030 dev.igb.4.interrupts.rx_desc_min_thresh: 0 dev.igb.4.interrupts.rx_overrun: 0 dev.igb.4.host.breaker_tx_pkt: 0 dev.igb.4.host.host_tx_pkt_discard: 0 dev.igb.4.host.rx_pkt: 273 dev.igb.4.host.breaker_rx_pkts: 0 dev.igb.4.host.breaker_rx_pkt_drop: 0 dev.igb.4.host.tx_good_pkt: 161 dev.igb.4.host.breaker_tx_pkt_drop: 0 dev.igb.4.host.rx_good_bytes: 40798644311 dev.igb.4.host.tx_good_bytes: 11291998600 dev.igb.4.host.length_errors: 0 dev.igb.4.host.serdes_violation_pkt: 0 dev.igb.4.host.header_redir_missed: 0
packet capture during failure to obtain address (this repeats etc)…...
[2.2-RELEASE][admin@pfsense.irj972.local.lan]/root: tcpdump -i igb4 -n -e\ tcpdump: verbose output suppressed, use -v or -vv for full protocol decode\ listening on igb4, link-type EN10MB (Ethernet), capture size 65535 bytes\ capability mode sandbox enabled\ 09:57:35.418721 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:57:43.603163 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:57:58.628668 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:58:10.300655 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:58:18.123653 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:58:23.094394 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 09:58:24.173118 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 09:58:35.209295 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:58:35.209350 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:58:38.615139 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:58:44.051533 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:58:57.754659 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:59:06.751717 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:59:13.412166 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:59:18.731217 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 09:59:26.140115 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:59:31.140733 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 09:59:32.223387 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 09:59:43.099670 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:59:43.099728 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:59:44.077917 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:59:46.088837 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:59:49.509144 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 09:59:57.149148 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:00:06.264730 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:00:19.273645 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:00:19.351130 0c:c4:7a:07:d3:4c > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 86.6.30.1 tell 86.6.31.151, length 46\ 10:00:19.359733 0c:c4:7a:07:d3:4c > 00:21:55:cb:93:da, ethertype IPv4 (0x0800), length 60: 86.6.31.151.445 > 88.245.242.252.51368: Flags [R.], seq 0, ack 1015706558, win 0, length 0\ 10:00:19.965117 0c:c4:7a:07:d3:4c > 00:21:55:cb:93:da, ethertype IPv4 (0x0800), length 60: 86.6.31.151.445 > 88.245.242.252.51368: Flags [R.], seq 0, ack 1, win 0, length 0\ 10:00:35.208162 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:00:39.237932 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 10:00:40.269642 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 10:00:51.191110 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:00:51.191172 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:00:56.973637 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:01:03.394341 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:01:15.448021 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:01:21.341961 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:01:29.763758 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:01:47.534392 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 10:01:48.616219 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 10:01:59.638511 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:01:59.638551 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:02:01.667631 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:02:06.584692 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:02:19.586637 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:02:39.337677 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:02:56.233386 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 10:02:57.305638 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 10:03:08.335765 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:03:08.335819 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:03:10.351151 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:03:11.922575 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:03:13.305153 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:03:21.435318 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:03:33.726134 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:03:52.614881 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:04:00.458869 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:04:05.499490 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 10:04:06.578226 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ plug\ 10:04:32.843841 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:04:32.843897 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:04:33.201847 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:04:35.327642 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:04:36.352665 0c:c4:7a:07:d3:4c > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 590: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 0c:c4:7a:07:d3:4c, length 548\ 10:04:36.452722 0c:c4:7a:07:d3:4c > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 590: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 0c:c4:7a:07:d3:4c, length 548\ 10:04:36.823898 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:04:37.792892 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:04:37.802532 0c:c4:7a:07:d3:4c > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 86.6.30.1 tell 86.6.31.151, length 46\ 10:04:37.809831 0c:c4:7a:07:d3:4c > 00:21:55:cb:93:da, ethertype IPv4 (0x0800), length 98: 86.6.31.151 > 86.6.30.1: ICMP echo request, id 15952, seq 0, length 64\ 10:04:39.662854 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:04:40.149097 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:04:43.492885 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:04:51.112925 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:04:53.403180 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:05:01.454914 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:05:06.433017 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:05:16.568944 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:05:31.006009 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 10:05:32.084215 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 10:05:36.763168 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:05:42.991497 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:05:42.991550 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:05:46.123125 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:05:54.073136 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:06:05.435315 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:06:25.476416 0c:c4:7a:07:d3:4c > 00:21:55:cb:93:da, ethertype IPv4 (0x0800), length 60: 86.6.31.151.8090 > 85.25.103.50.48863: Flags [R.], seq 0, ack 2944169830, win 0, length 0\ 10:06:26.950954 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:06:30.473116 0c:c4:7a:07:d3:4c > 00:21:55:cb:93:da, ethertype ARP (0x0806), length 60: Request who-has 86.6.30.1 tell 86.6.31.151, length 46\ 10:06:34.705132 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:06:35.403556 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:06:39.277953 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 10:06:40.359274 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 10:06:51.435339 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:06:51.435393 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:06:56.447039 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:07:04.150797 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:07:15.495195 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:07:32.669738 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:07:48.579671 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 10:07:49.658274 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 10:08:00.441290 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:08:00.441346 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:08:02.588120 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:08:06.757112 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:08:17.938616 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:08:31.434178 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:08:31.636630 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:08:52.910109 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:08:57.189611 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 10:08:58.206293 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 10:09:09.252909 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:09:09.252965 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:09:12.441116 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:09:21.008108 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:09:31.759107 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:09:47.408872 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:09:59.455601 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:10:05.158687 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.20 tell 86.5.163.20, length 28\ 10:10:06.240233 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 86.5.163.1 tell 86.5.163.20, length 28\ 10:10:17.329407 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:10:17.329464 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:10:22.501600 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:10:23.884892 0c:c4:7a:07:d3:4c > 33:33:00:01:00:02, ethertype IPv6 (0x86dd), length 98: fe80::ec4:7aff:fe07:d34c.546 > ff02::1:2.547: dhcp6 inf-req\ 10:10:27.418331 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:10:37.353627 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ 10:10:46.575595 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ ^X10:11:02.327114 00:22:68:1f:74:81 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:22:68:1f:74:81, length 300\ ^X^C\ 128 packets captured\ 128 packets received by filter\ 0 packets dropped by kernel}
so none of this brought my WAN interface back up so I REBOOTED pfsense,
the netstat upon reboot….
igb4 has disappeared completely....
igb5 seems to have acquired the 10.10.10.1 address pfBlockerNG is using for DNSBL filtering (still in beta)[2.2-RELEASE][admin@pfsense.irj972.local.lan]/root: netstat -rn\ Routing tables\ \ Internet:\ Destination Gateway Flags Netif Expire\ 10.10.10.1 link#8 UHS lo0\ 10.10.10.1/32 link#8 U igb5\ 127.0.0.1 link#13 UH lo0\ 192.168.10.0/24 link#1 U igb0\ 192.168.10.1 link#1 UHS lo0\ 192.168.50.0/24 link#8 U igb5\ 192.168.50.1 link#8 UHS lo0\ 192.168.60.0/24 link#9 U igb6\ 192.168.60.1 link#9 UHS lo0\ 192.168.70.0/24 link#10 U igb7\ 192.168.70.1 link#10 UHS lo0\ 192.168.101.0/24 link#5 U ix0\ 192.168.101.1 link#5 UHS lo0\ 192.168.110.0/24 link#6 U ix1\ 192.168.110.1 link#6 UHS lo0\ 192.168.210.0/24 192.168.210.2 UGS ovpns2\ 192.168.210.1 link#15 UHS lo0\ 192.168.210.2 link#15 UH ovpns2\ \ Internet6:\ Destination Gateway Flags Netif Expire\ ::1 link#13 UH lo0\ fe80::%igb0/64 link#1 U igb0\ fe80::a236:9fff:fe2d:eba8%igb0 link#1 UHS lo0\ fe80::%ix0/64 link#5 U ix0\ fe80::92e2:baff:fe37:fba4%ix0 link#5 UHS lo0\ fe80::%ix1/64 link#6 U ix1\ fe80::92e2:baff:fe37:fba5%ix1 link#6 UHS lo0\ fe80::%igb4/64 link#7 U igb4\ fe80::222:68ff:fe1f:7481%igb4 link#7 UHS lo0\ fe80::%igb5/64 link#8 U igb5\ fe80::ec4:7aff:fe07:d551%igb5 link#8 UHS lo0\ fe80::%igb6/64 link#9 U igb6\ fe80::ec4:7aff:fe07:d552%igb6 link#9 UHS lo0\ fe80::%igb7/64 link#10 U igb7\ fe80::ec4:7aff:fe07:d553%igb7 link#10 UHS lo0\ fe80::%lo0/64 link#13 U lo0\ fe80::1%lo0 link#13 UHS lo0\ fe80::%ovpns2/64 link#15 U ovpns2\ fe80::a236:9fff:fe2d:eba8%ovpns2 link#15 UHS lo0\ fe80::%ovpnc1/64 link#16 U ovpnc1\ fe80::a236:9fff:fe2d:eba8%ovpnc1 link#16 UHS lo0\ ff01::%igb0/32 fe80::a236:9fff:fe2d:eba8%igb0 U igb0\ ff01::%ix0/32 fe80::92e2:baff:fe37:fba4%ix0 U ix0\ ff01::%ix1/32 fe80::92e2:baff:fe37:fba5%ix1 U ix1\ ff01::%igb4/32 fe80::222:68ff:fe1f:7481%igb4 U igb4\ ff01::%igb5/32 fe80::ec4:7aff:fe07:d551%igb5 U igb5\ ff01::%igb6/32 fe80::ec4:7aff:fe07:d552%igb6 U igb6\ ff01::%igb7/32 fe80::ec4:7aff:fe07:d553%igb7 U igb7\ ff01::%lo0/32 ::1 U lo0\ ff01::%ovpns2/32 fe80::a236:9fff:fe2d:eba8%ovpns2 U ovpns2\ ff01::%ovpnc1/32 fe80::a236:9fff:fe2d:eba8%ovpnc1 U ovpnc1\ ff02::%igb0/32 fe80::a236:9fff:fe2d:eba8%igb0 U igb0\ ff02::%ix0/32 fe80::92e2:baff:fe37:fba4%ix0 U ix0\ ff02::%ix1/32 fe80::92e2:baff:fe37:fba5%ix1 U ix1\ ff02::%igb4/32 fe80::222:68ff:fe1f:7481%igb4 U igb4\ ff02::%igb5/32 fe80::ec4:7aff:fe07:d551%igb5 U igb5\ ff02::%igb6/32 fe80::ec4:7aff:fe07:d552%igb6 U igb6\ ff02::%igb7/32 fe80::ec4:7aff:fe07:d553%igb7 U igb7\ ff02::%lo0/32 ::1 U lo0\ ff02::%ovpns2/32 fe80::a236:9fff:fe2d:eba8%ovpns2 U ovpns2\ ff02::%ovpnc1/32 fe80::a236:9fff:fe2d:eba8%ovpnc1 U ovpnc1}
this is the dhcpd.log upon reboot, DHCP seems to be working fine on LANs, just WAN still dead.
(I recall the LED's on pfsense and my modem are on but steady and not blinking at this stage. )Mar 2 10:15:02 pfsense dhclient: PREINIT\ Mar 2 10:15:02 pfsense dhclient: EXPIRE\ Mar 2 10:15:02 pfsense dhclient: Deleting old routes\ Mar 2 10:15:02 pfsense dhclient: PREINIT\ Mar 2 10:15:02 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 1\ Mar 2 10:15:02 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 2\ Mar 2 10:15:04 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 2\ Mar 2 10:15:06 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 2\ Mar 2 10:15:08 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 4\ Mar 2 10:15:12 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 4\ Mar 2 10:15:16 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 11\ Mar 2 10:15:27 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 7\ Mar 2 10:15:35 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 9\ Mar 2 10:15:44 pfsense dhclient[17672]: DHCPDISCOVER on igb4 to 255.255.255.255 port 67 interval 14\ Mar 2 10:15:58 pfsense dhclient[17672]: No DHCPOFFERS received.\ Mar 2 10:15:58 pfsense dhclient[17672]: Trying recorded lease 86.5.163.20\ Mar 2 10:15:58 pfsense dhclient: TIMEOUT\ Mar 2 10:15:58 pfsense dhclient: Starting add_new_address()\ Mar 2 10:15:58 pfsense dhclient: ifconfig igb4 inet 86.5.163.20 netmask 255.255.255.0 broadcast 255.255.255.255\ Mar 2 10:15:58 pfsense dhclient: New IP Address (igb4): 86.5.163.20\ Mar 2 10:15:58 pfsense dhclient: New Subnet Mask (igb4): 255.255.255.0\ Mar 2 10:15:58 pfsense dhclient: New Broadcast Address (igb4): 255.255.255.255\ Mar 2 10:15:58 pfsense dhclient: New Routers (igb4): 86.5.163.1\ Mar 2 10:15:59 pfsense dhclient: New Routers (igb4): 86.5.163.1\ Mar 2 10:16:00 pfsense dhclient: Deleting old routes\ Mar 2 10:16:00 pfsense dhclient[17672]: bound: renewal in 238503 seconds.\ Mar 2 10:16:16 pfsense dhcpd: Internet Systems Consortium DHCP Server 4.2.6\ Mar 2 10:16:16 pfsense dhcpd: Copyright 2004-2014 Internet Systems Consortium.\ Mar 2 10:16:16 pfsense dhcpd: All rights reserved.\ Mar 2 10:16:16 pfsense dhcpd: For info, please visit https://www.isc.org/software/dhcp/\ Mar 2 10:16:16 pfsense dhcpd: Internet Systems Consortium DHCP Server 4.2.6\ Mar 2 10:16:16 pfsense dhcpd: Copyright 2004-2014 Internet Systems Consortium.\ Mar 2 10:16:16 pfsense dhcpd: All rights reserved.\ Mar 2 10:16:16 pfsense dhcpd: For info, please visit https://www.isc.org/software/dhcp/\ Mar 2 10:16:16 pfsense dhcpd: Wrote 0 deleted host decls to leases file.\ Mar 2 10:16:16 pfsense dhcpd: Wrote 0 new dynamic host decls to leases file.\ Mar 2 10:16:16 pfsense dhcpd: Wrote 45 leases to leases file.\ Mar 2 10:16:16 pfsense dhcpd: Listening on BPF/igb0/a0:36:9f:2d:eb:a8/192.168.10.0/24\ Mar 2 10:16:16 pfsense dhcpd: Sending on BPF/igb0/a0:36:9f:2d:eb:a8/192.168.10.0/24\ Mar 2 10:16:16 pfsense dhcpd: Listening on BPF/ix1/90:e2:ba:37:fb:a5/192.168.110.0/24\ Mar 2 10:16:16 pfsense dhcpd: Sending on BPF/ix1/90:e2:ba:37:fb:a5/192.168.110.0/24\ Mar 2 10:16:16 pfsense dhcpd: Listening on BPF/ix0/90:e2:ba:37:fb:a4/192.168.101.0/24\ Mar 2 10:16:16 pfsense dhcpd: Sending on BPF/ix0/90:e2:ba:37:fb:a4/192.168.101.0/24\ Mar 2 10:16:16 pfsense dhcpd: Listening on BPF/igb6/0c:c4:7a:07:d5:52/192.168.60.0/24\ Mar 2 10:16:16 pfsense dhcpd: Sending on BPF/igb6/0c:c4:7a:07:d5:52/192.168.60.0/24\ Mar 2 10:16:16 pfsense dhcpd: Listening on BPF/igb7/0c:c4:7a:07:d5:53/192.168.70.0/24\ Mar 2 10:16:16 pfsense dhcpd: Sending on BPF/igb7/0c:c4:7a:07:d5:53/192.168.70.0/24\ Mar 2 10:16:16 pfsense dhcpd: Listening on BPF/igb5/0c:c4:7a:07:d5:51/192.168.50.0/24\ Mar 2 10:16:16 pfsense dhcpd: Sending on BPF/igb5/0c:c4:7a:07:d5:51/192.168.50.0/24\ Mar 2 10:16:16 pfsense dhcpd: Sending on Socket/fallback/fallback-net\ Mar 2 10:16:17 pfsense dhcpd: DHCPDISCOVER from 88:1f:a1:28:e5:db (irj-extreme) via igb7\ Mar 2 10:16:18 pfsense dhcpd: DHCPDISCOVER from 90:e2:ba:38:2e:3c via ix1\ Mar 2 10:16:18 pfsense dhcpd: DHCPOFFER on 192.168.110.10 to 90:e2:ba:38:2e:3c via ix1\ Mar 2 10:16:18 pfsense dhcpd: DHCPOFFER on 192.168.70.154 to 88:1f:a1:28:e5:db (irj-extreme) via igb7\ Mar 2 10:16:18 pfsense dhcpd: DHCPREQUEST for 192.168.70.154 (192.168.70.1) from 88:1f:a1:28:e5:db (irj-extreme) via igb7\ Mar 2 10:16:18 pfsense dhcpd: DHCPACK on 192.168.70.154 to 88:1f:a1:28:e5:db (irj-extreme) via igb7\ Mar 2 10:16:20 pfsense dhcpd: DHCPREQUEST for 192.168.110.10 (192.168.110.1) from 90:e2:ba:38:2e:3c via ix1\ Mar 2 10:16:20 pfsense dhcpd: DHCPACK on 192.168.110.10 to 90:e2:ba:38:2e:3c via ix1\ Mar 2 10:16:20 pfsense dhcpd: DHCPDISCOVER from 28:37:37:41:a0:f3 (ae111) via igb5\ Mar 2 10:16:21 pfsense dhcpd: DHCPOFFER on 192.168.50.135 to 28:37:37:41:a0:f3 (ae111) via igb5\ Mar 2 10:16:21 pfsense dhcpd: DHCPREQUEST for 192.168.50.135 (192.168.50.1) from 28:37:37:41:a0:f3 (ae111) via igb5\ Mar 2 10:16:21 pfsense dhcpd: DHCPACK on 192.168.50.135 to 28:37:37:41:a0:f3 (ae111) via igb5}
that didn't fix it, so I rebooted the modem, leaving pfsense up…..