Gateway Not Active when connecting to 4g comcast backup gateway

stephenw10 · Dec 9, 2019, 11:25 AM

Hmm, well hard to see how that could work for something else but fail for pfSense. Unless perhaps it is conflicting with another subnet already defined in pfSense. Can we see the routing table? netstat -rn4

Steve

ercoupeflyer · Dec 9, 2019, 4:04 PM

Here is the current routing table. It shows opt1 active and sending thru gateway. but nothing received at the ip address!
that is 192.168.165.0 link, lan is 192.168.240.0 link

default 75.xx.xx.1 UGS 4932186 1500 re1
8.8.8.8 75.xx.xx.1 UGHS 102206 1500 re1
75.xx.xx.1/29 link#3 U 1139243 1500 re1
75.xx.xx.xx link#3 UHS 0 16384 lo0
75.75.75.75 75.xx.xx.xx UGHS 120330 1500 re1
127.0.0.1 link#6 UH 196310 16384 lo0
192.168.165.0/24 link#4 U 1128088 1500 re2
192.168.165.99 link#4 UHS 0 16384 lo0
192.168.240.0/24 link#1 U 30680502 1500 re0
192.168.240.1 link#1 UHS 68350 16384 lo0

ercoupeflyer · Dec 9, 2019, 4:23 PM

I just did a capture on the wan address for 192.168.165.0 segment. It looks like the router is trying to send the 165 segment packets thru the wan default gateway instead of the 165 segment gateway

trace wan: 75.xx.xx.1 address

08:15:31.306519 IP 192.168.165.99 > 192.168.165.1: ICMP echo request, id 22912, seq 25339, length 8
08:15:31.838758 IP 192.168.165.99 > 192.168.165.1: ICMP echo request, id 22912, seq 25340, length 8
08:15:32.370995 IP 192.168.165.99 > 192.168.165.1: ICMP echo request, id 22912, seq 25341, length 8

trace opt1 - backupwan 192.168.165.1 address

----- Blank ---------

Ummm

stephenw10 · Dec 9, 2019, 5:13 PM

Hmm, it really doesn't like 192.168.165.1 as a gateway. I expect to see a static route for it on link via re2.
But even without that 192.168.165.1 is inside a directly connected subnet.

What do you see in the system log when you connect that interface up and it pulls an DHCP lease?

Do you have the monitoring IP set as default currently, it's choosing to ping that rather than you have set it?

Steve

ercoupeflyer · Dec 9, 2019, 5:23 PM

hi, its using the default as the monitor ip. 192.168.165.1, this is pingable so should work. it pulls the ip just fine, everything looks correctly setup....
ummm

stephenw10 · Dec 9, 2019, 5:32 PM

I expect it to add a route there though and it isn't. If it's rejecting it there should be some messages in the log.
What does it show in the system log or dhcp log when you connect it?

Steve

ercoupeflyer · Dec 9, 2019, 9:42 PM

Here is what I have discovered so far after more detailed testing

if I set the 192.168.165.x interface to not monitor and assume it is up, then set the wan interface to down it will start using the backup wan.

Once the wan gateway is back up it never switches back because the monitor does not work.
I then set no monitor on the wan interface, makes it look like it is up, once that happens i can set the wan interface back to monitor and it works just fine again.

It always uses the working gateway to monitor the ip address for both interfaces.
each gateway sends a ping are routed thru the current working interface,
IE the other gateway to identify if it is up.

This of course fails when the active gateway goes down. so the gateway never recovers.

So summary, the backup gateway always uses the active gateway to ping, fails somehow in mapping nat or firewall.
After the primary gateway fails, does not recover because the backup gateway is always used.. pings never work thru the backup wan so it never sees its online.

The pings don't use the gateway port they are monitoring, instead they use the active gateway...... fails

This is a big problem

confusing

stephenw10 · Dec 9, 2019, 11:32 PM

Yes, that should never happen. The gateway should appear as a static route via the local link. Any monitoring IPs are added as a static route via the gateway they are set on to avoid exactly what you're seeing.

Are there really no errors shown in the logs when it pulls this and tries to add the routes?

Steve

ercoupeflyer · Dec 10, 2019, 12:02 AM

I have set static address for the back wan gateway. So no dhcp at all, still same problems,

What logs are most relevant to this, I dont see any errors, but can capture and post here as the interface goes up and down. Ie i can disable then enable the interface.

ercoupeflyer · Dec 10, 2019, 3:49 AM

Here is the startup log

Dec 9 19:37:48 vssgateway.xxxxxxxx.com nginx: 2019/12/09 19:37:48 [error] 8994#100115: send() failed (54: Connection reset by peer)
Dec 9 19:37:48 syslogd kernel boot file is /boot/kernel/kernel
Dec 9 19:37:48 syslogd exiting on signal 15
Dec 9 19:37:48 check_reload_status Syncing firewall
Dec 9 19:36:16 login login on ttyv0 as root
Dec 9 19:36:15 php-fpm 33733 /rc.filter_configure_sync: 33733MONITOR: WANGW is available now, adding to routing group FailOver 8.8.4.4|74.xx.xx.x3|WANGW|33.432ms|6.73ms|10%|none
Dec 9 19:36:14 php-fpm 340 /rc.start_packages: [lightsquid] Updating cronjobs...
Dec 9 19:36:14 check_reload_status Syncing firewall
Dec 9 19:36:14 check_reload_status Syncing firewall
Dec 9 19:36:14 php-fpm 340 /rc.start_packages: [lightsquid] Removing old cronjobs...
Dec 9 19:36:14 php-fpm 340 /rc.start_packages: [lightsquid] Successfully created '/usr/local/etc/lightsquid/lightsquid.cfg' configuration file.
Dec 9 19:36:14 php-fpm 340 /rc.start_packages: [lightsquid] Loaded default '/usr/local/etc/lightsquid/lightsquid.cfg.sample' configuration file.
Dec 9 19:36:14 check_reload_status Reloading filter
Dec 9 19:36:13 php-fpm 340 /rc.start_packages: [squid] Starting a proxy monitor script
Dec 9 19:36:13 php-fpm 340 /rc.start_packages: [squid] Reloading for configuration sync...
Dec 9 19:36:11 php-fpm 340 /rc.start_packages: [squid] Stopping any running proxy monitors
Dec 9 19:36:11 php-fpm 340 /rc.start_packages: [squid] Removing freshclam cronjob.
Dec 9 19:36:11 php-fpm 340 /rc.start_packages: [squid] Antivirus features disabled.
Dec 9 19:36:11 php-fpm 340 /rc.start_packages: [squid] Adding cronjobs ...
Dec 9 19:36:11 php-fpm 340 /rc.start_packages: [squid] - squid_resync function call pr:1 bp: rpc:no
Dec 9 19:36:09 check_reload_status Reloading filter
Dec 9 19:36:09 check_reload_status Restarting OpenVPN tunnels/interfaces
Dec 9 19:36:09 check_reload_status Restarting ipsec tunnels
Dec 9 19:36:09 check_reload_status updating dyndns WANGW
Dec 9 19:36:09 rc.gateway_alarm 60823 >>> Gateway alarm: WANGW (Addr:8.8.4.4 Alarm:0 RTT:33.176ms RTTsd:6.027ms Loss:13%)
Dec 9 19:36:07 php-fpm 33733 /index.php: Successful login for user 'admin' from: 192.168.240.6 (Local Database)
Dec 9 19:36:00 check_reload_status Reloading filter
Dec 9 19:35:59 php-fpm 340 /rc.start_packages: [squid] Starting a proxy monitor script
Dec 9 19:35:58 php-fpm 340 /rc.start_packages: [squid] Starting service...
Dec 9 19:35:56 php-fpm 340 /rc.start_packages: [squid] Removing freshclam cronjob.
Dec 9 19:35:56 php-fpm 340 /rc.start_packages: [squid] Antivirus features disabled.
Dec 9 19:35:56 php-fpm 340 /rc.start_packages: [squid] Adding cronjobs ...
Dec 9 19:35:55 php-fpm 340 /rc.start_packages: [squid] - squid_resync function call pr: bp: rpc:no
Dec 9 19:35:55 php-fpm 340 /rc.start_packages: Restarting/Starting all packages.
Dec 9 19:35:55 kernel done.
Dec 9 19:35:55 syslogd kernel boot file is /boot/kernel/kernel
Dec 9 19:35:54 syslogd exiting on signal 15
Dec 9 19:35:54 kernel done.
Dec 9 19:35:54 php-cgi rc.bootup: Creating rrd update script
Dec 9 19:35:54 snmpd 45060 disk_OS_get_disks: adding device 'cd0' to device list
Dec 9 19:35:54 snmpd 45060 disk_OS_get_disks: adding device 'ada0' to device list
Dec 9 19:35:49 kernel .done.
Dec 9 19:35:49 php-cgi rc.bootup: [squid] Installed but not started. Not installing 'filter' rules.
Dec 9 19:35:49 kernel ..
Dec 9 19:35:49 kernel .
Dec 9 19:35:49 php-cgi rc.bootup: [squid] Installed but not started. Not installing 'pfearly' rules.
Dec 9 19:35:49 php-cgi rc.bootup: [squid] Installed but not started. Not installing 'nat' rules.
Dec 9 19:35:49 check_reload_status Updating all dyndns
Dec 9 19:35:49 php-cgi rc.bootup: NTPD is starting up.
Dec 9 19:35:49 kernel done.
Dec 9 19:35:49 kernel done.
Dec 9 19:35:48 kernel done.
Dec 9 19:35:47 php-cgi rc.bootup: sync unbound done.
Dec 9 19:35:47 php-fpm 340 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'filter' rules.
Dec 9 19:35:47 php-fpm 340 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'pfearly' rules.
Dec 9 19:35:47 php-fpm 340 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'nat' rules.
Dec 9 19:35:47 php-fpm 341 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'filter' rules.
Dec 9 19:35:47 php-fpm 341 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'pfearly' rules.
Dec 9 19:35:47 php-fpm 341 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'nat' rules.
Dec 9 19:35:46 kernel re2: link state changed to UP
Dec 9 19:35:46 check_reload_status Linkup starting re2
Dec 9 19:35:46 check_reload_status Linkup starting re1
Dec 9 19:35:46 kernel re1: link state changed to UP
Dec 9 19:35:46 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:46 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:46 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:46 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:46 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:46 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:45 check_reload_status Linkup starting re0
Dec 9 19:35:45 kernel re0: link state changed to UP
Dec 9 19:35:45 check_reload_status Reloading filter
Dec 9 19:35:45 check_reload_status Restarting OpenVPN tunnels/interfaces
Dec 9 19:35:45 check_reload_status Reloading filter
Dec 9 19:35:45 check_reload_status Restarting ipsec tunnels
Dec 9 19:35:45 check_reload_status Restarting OpenVPN tunnels/interfaces
Dec 9 19:35:45 check_reload_status updating dyndns BackupWanGW
Dec 9 19:35:45 check_reload_status Restarting ipsec tunnels
Dec 9 19:35:45 check_reload_status updating dyndns WANGW
Dec 9 19:35:45 rc.gateway_alarm 31196 >>> Gateway alarm: BackupWanGW (Addr:8.8.8.8 Alarm:1 RTT:0.000ms RTTsd:0.000ms Loss:100%)
Dec 9 19:35:45 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:45 rc.gateway_alarm 31647 >>> Gateway alarm: WANGW (Addr:8.8.4.4 Alarm:1 RTT:0.000ms RTTsd:0.000ms Loss:100%)
Dec 9 19:35:45 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:45 php-cgi rc.bootup: MONITOR: WANGW is down, omitting from routing group FailOver 8.8.4.4|74.xx.xx.x3|WANGW|0ms|0ms|100%|down
Dec 9 19:35:45 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:45 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:44 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:44 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:44 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:44 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:43 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:43 kernel arpresolve: can't allocate llinfo for 74.xx.xx.x1 on re1
Dec 9 19:35:43 kernel done.
Dec 9 19:35:43 php-cgi rc.bootup: Removing static route for monitor 8.8.8.8 and adding a new route through 192.168.165.1
Dec 9 19:35:43 kernel .done.
Dec 9 19:35:43 php-cgi rc.bootup: Removing static route for monitor 8.8.4.4 and adding a new route through 74.xx.xx.x1
Dec 9 19:35:43 kernel ..
Dec 9 19:35:43 php-cgi rc.bootup: [squid] Installed but not started. Not installing 'filter' rules.
Dec 9 19:35:43 kernel .
Dec 9 19:35:43 php-cgi rc.bootup: [squid] Installed but not started. Not installing 'pfearly' rules.
Dec 9 19:35:43 php-cgi rc.bootup: [squid] Installed but not started. Not installing 'nat' rules.
Dec 9 19:35:43 php-cgi rc.bootup: Default gateway setting fast wan as default.
Dec 9 19:35:43 php-cgi rc.bootup: Gateway, switch to: WANGW
Dec 9 19:35:42 php-cgi rc.bootup: Resyncing OpenVPN instances.
Dec 9 19:35:42 check_reload_status Linkup starting re2
Dec 9 19:35:42 check_reload_status Linkup starting re0
Dec 9 19:35:43 kernel pflog0: promiscuous mode enabled
Dec 9 19:35:43 kernel re2: link state changed to DOWN
Dec 9 19:35:43 kernel re0: link state changed to DOWN
Dec 9 19:35:43 kernel re1: link state changed to DOWN
Dec 9 19:35:43 kernel TSC: P-state invariant
Dec 9 19:35:43 kernel SVM: NP,NRIP,NAsids=64
Dec 9 19:35:43 kernel AMD Features2=0x37fd<LAHF,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT>
Dec 9 19:35:43 kernel AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
Dec 9 19:35:43 kernel Features2=0x802009<SSE3,MON,CX16,POPCNT>
Dec 9 19:35:43 kernel Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
Dec 9 19:35:43 kernel Origin="AuthenticAMD" Id=0x100f63 Family=0x10 Model=0x6 Stepping=3
Dec 9 19:35:43 kernel CPU: AMD Athlon(tm) II 170u Processor (2009.18-MHz K8-class CPU)
Dec 9 19:35:43 kernel random: unblocking device.
Dec 9 19:35:43 kernel Trying to mount root from ufs:/dev/gptid/e7f7db70-b78e-11e8-96e0-001ee5d7ba4f [rw]...
Dec 9 19:35:43 kernel ada0: 476940MB (976773168 512 byte sectors)
Dec 9 19:35:43 kernel ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
Dec 9 19:35:43 kernel ada0: Serial Number WD-WCAWF7860894
Dec 9 19:35:43 kernel cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed
Dec 9 19:35:43 kernel cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes)
Dec 9 19:35:43 kernel cd0: Serial Number 029CD088361
Dec 9 19:35:43 kernel cd0: <hp DVD-RAM GH60L RD01> Removable CD-ROM SCSI device
Dec 9 19:35:43 kernel cd0 at ata3 bus 0 scbus1 target 0 lun 0
Dec 9 19:35:43 kernel ada0: <WDC WD5000AAKS-60Z1A0 06.01D06> ATA8-ACS SATA 2.x device
Dec 9 19:35:43 kernel ada0 at ata2 bus 0 scbus0 target 0 lun 0
Dec 9 19:35:43 kernel ugen0.3: <vendor 0x2188 USB OPTICAL MOUSE> at usbus0
Dec 9 19:35:43 kernel uhid0: <SEM USB Keyboard, class 0/0, rev 1.10/1.10, addr 2> on usbus0
Dec 9 19:35:43 kernel uhid0 on uhub0
Dec 9 19:35:43 kernel kbd2 at ukbd0
Dec 9 19:35:43 kernel ukbd0: <SEM USB Keyboard, class 0/0, rev 1.10/1.10, addr 2> on usbus0
Dec 9 19:35:43 kernel ukbd0 on uhub0
Dec 9 19:35:43 kernel ugen0.2: <SEM USB Keyboard> at usbus0
Dec 9 19:35:43 kernel uhub1: 10 ports with 10 removable, self powered
Dec 9 19:35:43 kernel uhub0: 10 ports with 10 removable, self powered
Dec 9 19:35:43 kernel uhub1: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
Dec 9 19:35:43 kernel ugen1.1: <nVidia EHCI root HUB> at usbus1
Dec 9 19:35:43 kernel uhub0: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
Dec 9 19:35:43 kernel ugen0.1: <nVidia OHCI root HUB> at usbus0
Dec 9 19:35:43 kernel pcm1: <Realtek ALC888 (Analog)> at nid 27 and 22 on hdaa0
Dec 9 19:35:43 kernel pcm0: <Realtek ALC888 (Analog)> at nid 20 and 24,25 on hdaa0
Dec 9 19:35:43 kernel usbus1: 480Mbps High Speed USB v2.0
Dec 9 19:35:43 kernel usbus0: 12Mbps Full Speed USB v1.0
Dec 9 19:35:43 kernel hdaa0: <Realtek ALC888 Audio Function Group> at nid 1 on hdacc0
Dec 9 19:35:43 kernel hdacc0: <Realtek ALC888 HDA CODEC> at cad 0 on hdac0
Dec 9 19:35:43 kernel Timecounters tick every 1.000 msec
Dec 9 19:35:43 kernel hwpstate0: <Cool`n'Quiet 2.0> on cpu0
Dec 9 19:35:43 kernel ppc0: cannot reserve I/O port range
Dec 9 19:35:43 kernel atkbd0: [GIANT-LOCKED]
Dec 9 19:35:43 kernel kbd0 at atkbd0
Dec 9 19:35:43 kernel atkbd0: <AT Keyboard> irq 1 on atkbdc0
Dec 9 19:35:43 kernel atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
Dec 9 19:35:43 kernel orm0: <ISA Option ROM> at iomem 0xc0000-0xcefff on isa0
Dec 9 19:35:43 kernel acpi_tz0: <Thermal Zone> on acpi0
Dec 9 19:35:43 kernel vgapci0: Boot video device
Dec 9 19:35:43 kernel vgapci0: <VGA-compatible display> mem 0xfb000000-0xfbffffff,0xe0000000-0xefffffff,0xfc000000-0xfcffffff irq 23 at device 13.0 on pci0
Dec 9 19:35:43 kernel re2: netmap queues/slots: TX 1/256, RX 1/256
Dec 9 19:35:43 kernel re2: Ethernet address: 34:e8:948d:db
Dec 9 19:35:43 kernel re2: Using defaults for TSO: 65518/35/2048
Dec 9 19:35:43 kernel rgephy2: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
Dec 9 19:35:43 kernel rgephy2: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus3
Dec 9 19:35:43 kernel miibus3: <MII bus> on re2
Dec 9 19:35:43 kernel re2: MAC rev. 0x00200000
Dec 9 19:35:43 kernel re2: Chip rev. 0x2c000000
Dec 9 19:35:43 kernel re2: Using 1 MSI-X message
Dec 9 19:35:43 kernel re2: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0xbc00-0xbcff mem 0xfd9ff000-0xfd9fffff,0xfd8fc000-0xfd8fffff irq 16 at device 0.0 on pci4
Dec 9 19:35:43 kernel pci4: <ACPI PCI bus> on pcib4
Dec 9 19:35:43 kernel pcib4: <ACPI PCI-PCI bridge> at device 12.0 on pci0
Dec 9 19:35:43 kernel re1: netmap queues/slots: TX 1/256, RX 1/256
Dec 9 19:35:43 kernel re1: Ethernet address: a0:f3:c1:01:18:aa
Dec 9 19:35:43 kernel re1: Using defaults for TSO: 65518/35/2048
Dec 9 19:35:43 kernel rgephy1: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
Dec 9 19:35:43 kernel rgephy1: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus2
Dec 9 19:35:43 kernel miibus2: <MII bus> on re1
Dec 9 19:35:43 kernel re1: MAC rev. 0x00200000
Dec 9 19:35:43 kernel re1: Chip rev. 0x2c000000
Dec 9 19:35:43 kernel re1: Using 1 MSI-X message
Dec 9 19:35:43 kernel re1: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0xcc00-0xccff mem 0xfdbff000-0xfdbfffff,0xfdafc000-0xfdafffff irq 16 at device 0.0 on pci3
Dec 9 19:35:43 kernel pci3: <ACPI PCI bus> on pcib3
Dec 9 19:35:43 kernel pcib3: <ACPI PCI-PCI bridge> at device 11.0 on pci0
Dec 9 19:35:43 kernel pci2: <ACPI PCI bus> on pcib2
Dec 9 19:35:43 kernel pcib2: <ACPI PCI-PCI bridge> at device 9.0 on pci0
Dec 9 19:35:43 kernel ata5: <ATA channel> at channel 1 on atapci1
Dec 9 19:35:43 kernel ata4: <ATA channel> at channel 0 on atapci1
Dec 9 19:35:43 kernel atapci1: <nVidia nForce MCP61 SATA300 controller> port 0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xf200-0xf20f mem 0xfe02b000-0xfe02bfff irq 21 at device 8.1 on pci0
Dec 9 19:35:43 kernel ata3: <ATA channel> at channel 1 on atapci0
Dec 9 19:35:43 kernel ata2: <ATA channel> at channel 0 on atapci0
Dec 9 19:35:43 kernel atapci0: <nVidia nForce MCP61 SATA300 controller> port 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xf700-0xf70f mem 0xfe02c000-0xfe02cfff irq 20 at device 8.0 on pci0
Dec 9 19:35:43 kernel nfe0: Ethernet address: 70:71:bc:a0:84:5d
Dec 9 19:35:43 kernel rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow
Dec 9 19:35:43 kernel rlphy0: <RTL8201L 10/100 media interface> PHY 1 on miibus1
Dec 9 19:35:43 kernel miibus1: <MII bus> on nfe0
Dec 9 19:35:43 kernel nfe0: <NVIDIA nForce MCP61 Networking Adapter> port 0xfc00-0xfc07 mem 0xfe02d000-0xfe02dfff irq 22 at device 7.0 on pci0
Dec 9 19:35:43 kernel hdac0: <NVIDIA MCP61 HDA Controller> mem 0xfe024000-0xfe027fff irq 23 at device 5.0 on pci0
Dec 9 19:35:43 kernel re0: netmap queues/slots: TX 1/256, RX 1/256
Dec 9 19:35:43 kernel re0: Ethernet address: 00:1e:e5:d7:ba:4f

stephenw10 · Dec 10, 2019, 12:39 PM

Are those all individual Realtek NICs? Can you swap them?
I would be hesitant to use the nfe NIC but maybe just as a test.

If you connect the OPT interface to some other router with DHCP enabled does it fail the same way?

Steve

ercoupeflyer · Dec 10, 2019, 6:18 PM

I tried to swap them earlier and used the nfe nic, it had the same problem, so i added a new nic. and stopped using nfe nic. But it had exactly the same problem.

It does seem to have something to do with the 192.168. segment, and I am unable to reconfigure the 4g router provided by comcast. Is there some way to work around this issue or am I making a wrong assumption.

It's funny that the log says its creating a route for the monitor address 8.8.8.8 thru the correct gateway, yet when i do a capture the its not going out there it is going out the default wan gateway........

Can i setup some manual route to force this to work, shaking my head...... lol

Yes they are all individual realtek nics, and yes I have tried a netgear router which creates 192.168. segments and it fails as well. If I connect to the dhcp of the cable modem by comacast it issues a 10.1. address and after some waiting it seems to become active. but actually never sends anything thru. I have made sure the local address filter is turned off for that gateway re2.

I am no longer using dhcp on any interface!

stephenw10 · Dec 10, 2019, 10:18 PM

It looks like you have something configured using 192.168/16 somewhere that is conflicting. It's not in the routing table though.
I would open your config file and search it for 192.168 and see what pops out at this point. There will be a lot of entries since you're using that for LAN.

Steve