Static wan IP stops working after a power cycle

[Jsingh04]

So i have built a pfsense box on an hp t730 with a broadcom 95719 4port nic.

Everything worked fine the first time with a static ip on the wan port. But as soon there was a power cycle or a restart the internet stops working.

When I switch to Dhcp everything starting working normally again. Any idea what's wrong?

[SteveITS]

@jsingh04 Can you post the log from the bootup?

[Jsingh04]

@steveits
This is the system log file
system.log.txt

This is the dmesg.boot file

Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.3-STABLE RELENG_2_6_0-n226742-1285d6d205f pfSense amd64
FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
VT(efifb): resolution 800x600
CPU: AMD RX-427BB with AMD Radeon(tm) R7 Graphics    (2695.02-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x630f01  Family=0x15  Model=0x30  Stepping=1
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x3e98320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0xfebbfff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,XOP,SKINIT,WDT,LWP,FMA4,TCE,NodeId,TBM,Topology,PCXC,PNXC,<b25>,DBE,PTSC>
  Structured Extended Features=0x9<FSGSBASE,BMI1>
  XSAVE Features=0x1<XSAVEOPT>
  AMD Extended Feature Extensions ID EBX=0x1000
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=65536
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 7124004864 (6793 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <HPQOEM SLIC-WKS>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: unblocking device.
Firmware Warning (ACPI): Optional FADT field Pm2ControlBlock has valid Length but zero Address: 0x0000000000000000/0x1 (20200430/tbfadt-796)
ioapic0 <Version 2.1> irqs 0-23 on motherboard
ioapic1 <Version 2.1> irqs 24-55 on motherboard
Launching APs: 3 1 2
Timecounter "TSC-low" frequency 1347510905 Hz quality 1000
random: entropy device external interface
ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff80739000, 0) error 1
ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff807390b0, 0) error 1
ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80739160, 0) error 1
iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff807609f0, 0) error 1
iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff80760aa0, 0) error 1
iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80760b50, 0) error 1
wlan: mac acl policy registered
WARNING: Device "g_ctl" is Giant locked and may be deleted before FreeBSD 14.0.
WARNING: Device "pci" is Giant locked and may be deleted before FreeBSD 14.0.
module_register_init: MOD_LOAD (vesa, 0xffffffff8140a210, 0) error 19
WARNING: Device "kbd" is Giant locked and may be deleted before FreeBSD 14.0.
kbd1 at kbdmux0
[ath_hal] loaded
WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 14.0.
000.000056 [4344] netmap_init               netmap: loaded module
mlx5en: Mellanox Ethernet driver 3.6.0 (December 2020)
nexus0
efirtc0: <EFI Realtime Clock> on motherboard
efirtc0: registered as a time-of-day clock, resolution 1.000000s
cryptosoft0: <software crypto> on motherboard
acpi0: <HPQOEM SLIC-WKS> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 450
Event timer "HPET1" frequency 14318180 Hz quality 450
Event timer "HPET2" frequency 14318180 Hz quality 450
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <base peripheral, IOMMU> at device 0.2 (no driver attached)
vgapci0: <VGA-compatible display> port 0xf000-0xf0ff mem 0xd0000000-0xdfffffff,0xe0000000-0xe07fffff,0xfeb00000-0xfeb3ffff at device 1.0 on pci0
vgapci0: Boot video device
hdac0: <ATI (0x1308) HDA Controller> mem 0xfeb64000-0xfeb67fff at device 1.1 on pci0
pcib1: <ACPI PCI-PCI bridge> at device 2.1 on pci0
pci1: <ACPI PCI bus> on pcib1
bge0: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5719001> mem 0xe08b0000-0xe08bffff,0xe08a0000-0xe08affff,0xe0890000-0xe089ffff at device 0.0 on pci1
bge0: APE FW version: NCSI v1.2.46.0
bge0: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus0: <MII bus> on bge0
brgphy0: <BCM5719C 1000BASE-T media interface> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Using defaults for TSO: 65518/35/2048
bge0: Ethernet address: 00:0a:f7:7a:93:84
bge1: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5719001> mem 0xe0880000-0xe088ffff,0xe0870000-0xe087ffff,0xe0860000-0xe086ffff at device 0.1 on pci1
bge1: APE FW version: NCSI v1.2.46.0
bge1: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus1: <MII bus> on bge1
brgphy1: <BCM5719C 1000BASE-T media interface> PHY 2 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Using defaults for TSO: 65518/35/2048
bge1: Ethernet address: 00:0a:f7:7a:93:85
bge2: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5719001> mem 0xe0850000-0xe085ffff,0xe0840000-0xe084ffff,0xe0830000-0xe083ffff at device 0.2 on pci1
bge2: APE FW version: NCSI v1.2.46.0
bge2: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus2: <MII bus> on bge2
brgphy2: <BCM5719C 1000BASE-T media interface> PHY 3 on miibus2
brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge2: Using defaults for TSO: 65518/35/2048
bge2: Ethernet address: 00:0a:f7:7a:93:86
bge3: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5719001> mem 0xe0820000-0xe082ffff,0xe0810000-0xe081ffff,0xe0800000-0xe080ffff at device 0.3 on pci1
bge3: APE FW version: NCSI v1.2.46.0
bge3: CHIP ID 0x05719001; ASIC REV 0x5719; CHIP REV 0x57190; PCI-E
miibus3: <MII bus> on bge3
brgphy3: <BCM5719C 1000BASE-T media interface> PHY 4 on miibus3
brgphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge3: Using defaults for TSO: 65518/35/2048
bge3: Ethernet address: 00:0a:f7:7a:93:87
pcib2: <ACPI PCI-PCI bridge> at device 3.2 on pci0
pci2: <ACPI PCI bus> on pcib2
re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0xe000-0xe0ff mem 0xfe904000-0xfe904fff,0xfe900000-0xfe903fff at device 0.0 on pci2
re0: Using 1 MSI-X message
re0: Chip rev. 0x54000000
re0: MAC rev. 0x00100000
miibus4: <MII bus> on re0
rgephy0: <RTL8251/8153 1000BASE-T media interface> PHY 1 on miibus4
rgephy0:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
re0: Using defaults for TSO: 65518/35/2048
re0: Ethernet address: 7c:d3:0a:7f:3c:bd
re0: netmap queues/slots: TX 1/256, RX 1/256
xhci0: <AMD FCH USB 3.0 controller> mem 0xfeb6a000-0xfeb6bfff at device 16.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
xhci1: <AMD FCH USB 3.0 controller> mem 0xfeb68000-0xfeb69fff at device 16.1 on pci0
xhci1: 32 bytes context size, 64-bit DMA
usbus1 on xhci1
usbus1: 5.0Gbps Super Speed USB v3.0
ahci0: <AMD Hudson-2 AHCI SATA controller> port 0xf190-0xf197,0xf180-0xf183,0xf170-0xf177,0xf160-0xf163,0xf150-0xf15f mem 0xfeb70000-0xfeb707ff at device 17.0 on pci0
ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ohci0: <AMD FCH USB Controller> mem 0xfeb6f000-0xfeb6ffff at device 18.0 on pci0
usbus2 on ohci0
usbus2: 12Mbps Full Speed USB v1.0
ehci0: <AMD FCH USB 2.0 controller> mem 0xfeb6e000-0xfeb6e0ff at device 18.2 on pci0
usbus3: EHCI version 1.0
usbus3 on ehci0
usbus3: 480Mbps High Speed USB v2.0
ohci1: <AMD FCH USB Controller> mem 0xfeb6d000-0xfeb6dfff at device 19.0 on pci0
usbus4 on ohci1
usbus4: 12Mbps Full Speed USB v1.0
ehci1: <AMD FCH USB 2.0 controller> mem 0xfeb6c000-0xfeb6c0ff at device 19.2 on pci0
usbus5: EHCI version 1.0
usbus5 on ehci1
usbus5: 480Mbps High Speed USB v2.0
atapci0: <AMD Hudson-2 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf100-0xf10f at device 20.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
hdac1: <AMD Hudson-2 HDA Controller> mem 0xfeb60000-0xfeb63fff at device 20.2 on pci0
isab0: <PCI-ISA bridge> at device 20.3 on pci0
isa0: <ISA bus> on isab0
pcib3: <ACPI PCI-PCI bridge> at device 20.4 on pci0
pci3: <ACPI PCI bus> on pcib3
acpi_button0: <Power Button> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
driver bug: Unable to set devclass (class: atkbdc devname: (unknown))
ppc0: <Parallel port> port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/8 bytes threshold
ppbus0: <Parallel port bus> on ppc0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
hwpstate0: <Cool`n'Quiet 2.0> on cpu0
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
hdacc0: <ATI R6xx HDA CODEC> at cad 0 on hdac0
hdaa0: <ATI R6xx Audio Function Group> at nid 1 on hdacc0
pcm0: <ATI R6xx (HDMI)> at nid 3 on hdaa0
pcm1: <ATI R6xx (HDMI)> at nid 5 on hdaa0
pcm2: <ATI R6xx (HDMI)> at nid 7 on hdaa0
pcm3: <ATI R6xx (HDMI)> at nid 9 on hdaa0
hdacc1: <Realtek ALC221 HDA CODEC> at cad 0 on hdac1
hdaa1: <Realtek ALC221 Audio Function Group> at nid 1 on hdacc1
pcm4: <Realtek ALC221 (Analog 2.0+HP/2.0)> at nid 20,33 and 24,27 on hdaa1
pcm5: <Realtek ALC221 (Front Analog Headphones)> at nid 26 on hdaa1
ugen2.1: <AMD OHCI root HUB> at usbus2
ugen1.1: <0x1022 XHCI root HUB> at usbus1
ugen0.1: <0x1022 XHCI root HUB> at usbus0
uhub0: <0x1022 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
ugen5.1: <AMD EHCI root HUB> at usbus5
uhub1: <0x1022 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub2: ugen3.1: <AMD EHCI root HUB> at usbus3
<AMD EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus5
Trying to mount root from ufs:/dev/ufsid/4d1e709bd7be6799 [rw,noatime]...
uhub3: <AMD EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
Root mount waiting for: usbus0 usbus1 CAM usbus2 usbus3 usbus4 usbus5
uhub4: <AMD OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen4.1: <AMD OHCI root HUB> at usbus4
uhub5: <AMD OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
uhub4: 5 ports with 5 removable, self powered
uhub5: 5 ports with 5 removable, self powered
uhub0: 4 ports with 4 removable, self powered
uhub1: 4 ports with 4 removable, self powered
Root mount waiting for: CAM usbus3 usbus5
uhub2: 5 ports with 5 removable, self powered
uhub3: 5 ports with 5 removable, self powered
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <Simmtronic M2 2280 128GB U0514A0> ACS-2 ATA SATA 3.x device
ada0: Serial Number AA000050504445037230
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
ada0: Command Queueing enabled
ada0: 122104MB (250069680 512 byte sectors)
mountroot: waiting for device /dev/ufsid/4d1e709bd7be6799...
WARNING: / was not properly dismounted
WARNING: /: mount pending error: blocks 352 files 1
CPU: AMD RX-427BB with AMD Radeon(tm) R7 Graphics    (2695.02-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x630f01  Family=0x15  Model=0x30  Stepping=1
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x3e98320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0xfebbfff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,XOP,SKINIT,WDT,LWP,FMA4,TCE,NodeId,TBM,Topology,PCXC,PNXC,<b25>,DBE,PTSC>
  Structured Extended Features=0x9<FSGSBASE,BMI1>
  XSAVE Features=0x1<XSAVEOPT>
  AMD Extended Feature Extensions ID EBX=0x1000
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=65536
  TSC: P-state invariant, performance statistics

[stephenw10]

Is it using the same WAN IP and gateway in both configurations?

After you reboot with a static IP how are you testing the connectivity? How does it fail?

Steve

[Jsingh04]

@stephenw10

I have two boxes with same hardware both facing this issue.
One is running behind the ISP router and the other one behind a Unifi USG.
The DHCP and static IP addresses are different.

I test the connectivity just by opening google. When I ping google, it shows a name resolution error. I have installed team viewer on some virtual machines behind this firewall, even they are no longer accessible. When I ping the upstream USG IP, thats is reachable.

As soon as I switch to DHCP they start working again.

[stephenw10]

@jsingh04 said in Static wan IP stops working after a power cycle:

it shows a name resolution error

Then you have a DNS problem.
When you set the WAN as DHCP it probably pulls some external DNS servers that the firewall itself can use if it's own DNS resolver is not working.
When you look at you system log you will note that initially the date/time is wrong. The boot log shows there is an RTC present but it seems to be incorrect. Probably the battery needs replacing.

When you boot it with a static IP set after a power cycle the clock will be wrong and that leads to a scenario where Unbound fails to start because it's cert is invalid or it see results as invalid because DNSsec is enabled (by default). That means ntpd cannot resolve any external servers and the time cannot be updated.

So do one (or more) of:
Fix the RTC battery.
Add at least one external DNS server when you use a static WAN.
Disable DNSSec in Unbound.
Add a local NTP server that can be reached by IP address.

Steve