WAN Losing Connectivity / IP not reestablished



  • Hi;
    im having an issue on my WAN port; about 2 times a week, in the middle of the night, (not sure what time) my WAN interface loses connectivity with the internet.
    ive got a typical setup:  Comcast Cable Modem –> pfSense --> Switch
    pfSense is version 2.0.3

    i was able to grab the following out of the system log, and as far as i could tell (my log buffer is set to 500 lines...)
    this chunk of logging was repeated over and over for hours !

        Jun 13 08:54:51         php: : The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf re0 > tmp/re0_output > /tmp/re0_error_output' returned exit code '15', the output was ''
    
        Jun 13 08:54:51         dhclient[27229]: exiting.
        Jun 13 08:54:51         dhclient[27229]: exiting.
        Jun 13 08:54:51         dhclient[27229]: connection closed
        Jun 13 08:54:51         dhclient[27229]: connection closed
        Jun 13 08:54:51         php: : DEVD Ethernet detached event for wan
        Jun 13 08:54:51         dhclient[26632]: DHCPREQUEST on re0 to 255.255.255.255 port 67
        Jun 13 08:54:49         kernel: re0: link state changed to DOWN
        Jun 13 08:54:49         check_reload_status: Linkup starting re0
        Jun 13 08:54:49         dhclient[26632]: DHCPREQUEST on re0 to 255.255.255.255 port 67
        Jun 13 08:54:49         php: : HOTPLUG: Configuring interface wan
        Jun 13 08:54:49         php: : DEVD Ethernet attached event for wan
        Jun 13 08:54:47         dhclient[26632]: DHCPREQUEST on re0 to 255.255.255.255 port 67
        Jun 13 08:54:47         dhclient[26632]: DHCPREQUEST on re0 to 255.255.255.255 port 67
        Jun 13 08:54:47         dhclient: PREINIT
        Jun 13 08:54:46         kernel: re0: link state changed to UP
        Jun 13 08:54:46         check_reload_status: Linkup starting re0
    

    i've searched the forums already, and found im not the only one with an issue similar to this;
    there were a few 'solutions' thrown around, but they dont seem to have worked.
    the solutions i found were:

    1. disable sticky connections. (mine were enabled, and have since been disabled)
    2. disable MAC spoofing on WAN (i was not MAC spoofing)

    **edit
    cron is enabled, but it contains all the default jobs see here–> https://www.dropbox.com/s/nd2wfk7lajxa072/shot_130613_090843.png

    any suggestions would be helpful !



  • @ericab:

    Hi;
    im having an issue on my WAN port; about 2 times a week, in the middle of the night, (not sure what time) my WAN interface loses connectivity with the internet.
    ive got a typical setup:  Comcast Cable Modem –> pfSense --> Switch
    pfSense is version 2.0.3

    @ericab:

    i was able to grab the following out of the system log,

    
        Jun 13 08:54:49         kernel: re0: link state changed to DOWN
    . . .
        Jun 13 08:54:46         kernel: re0: link state changed to UP
    

    Suggests your WAN link lost connectivity with its upstream device - PERHAPS because that device (modem?) lost connectivity with its upstream device.



  • hmm, well i know for sure there are no power outages going on…maybe its a send/recieve signal issue for the modem itself ?
    i know its out of the scope of this forum, but do you by chance, know the acceptable power/SnR levels i should aim for ?
    its strange as i never had this issue before, and nothing about the cable modem/length of COAX cable's length to the drop point has changed...

    do you know of a way i can automatically trigger the pfSense box to reboot itself when "kernel: re0: link state changed to DOWN" pops up in the logs ?



  • @ericab:

    i know its out of the scope of this forum, but do you by chance, know the acceptable power/SnR levels i should aim for ?

    No, but perhaps tech support at your ISP does.

    @ericab:

    its strange as i never had this issue before, and nothing about the cable modem/length of COAX cable's length to the drop point has changed…

    Might be irrelevant if something upstream is flapping.

    @ericab:

    do you know of a way i can automatically trigger the pfSense box to reboot itself when "kernel: re0: link state changed to DOWN" pops up in the logs ?

    Not directly. I suspect you might find that is a bit "brutal": it might lock you out of your box. (suppose you have an UP -> DOWN transition regularly during startup.)

    Link UP is a report that the NIC has seen carrier signal from its peer. PERHAPS when the modem loses synchronisation with its peer the modem disables its downstream interface (to your pfSense box) causing your WAN link to go down (due to loss of carrier signal). Putting a switch (a $10 switch would do) between the modem and your WAN interface would maintain the carrier signal to the WAN link as long as the switch has power. If you still see the link flapping then it is likely the problem is around your WAN interface such as faulty cable or connector or driver bug. Maybe you have some other way of monitoring the link between WAN interface and modem to determine if the link really is going down. I don't have a cable service so I can't test this theory.

    Perhaps you could try using a snapshot build of pfSense 1.2. The snapshot builds have much more current device drivers than the 2.0.x builds which are based on FreeBSD 8.1 which was released almost three years ago.



  • thanks wallybob;
    i'm going to explore all these avenues, and will report back in a week or so.



  • all is well after adding a smiple switch between the modem, and the pfsense box !



  • @ericab:

    all is well after adding a smiple switch between the modem, and the pfsense box !

    Can you elaborate on "all is well"? You have had the switch in there for a few days and haven't seen the pfSense WAN link flap?



  • yes; sorry..

    adding the switch between the Modem and the pfSense box has, as of now (almost a week in) prevented my WAN interface from losing connectivity with the internet. i no longer have system logs full of the error messages in my OP. all seems to be well.
    if anything changes, ill be sure to reply to this post; but as far as i can tell wallabybob; you were correct in the issue being loss of the carrier signal to my WAN NIC.



  • It might be worth talking with tech support at your ISP. Does the modem's LAN port flapping indicate a fault in the modem or a fault "upstream"? If the latter, does that suggest a degrading physical connection upstream?



  • alright i can ask about the first one; but as for the second; i have triple checked my DOCSIS 3.0 signal levels are well with in tolerated bounds. ill explorer it a bit further with my ISP, but im going to have to tear my hair out if i have to deal with Taiwanese level 1 tech support for more then 5 seconds.

    ill keep this thread updated; but for anyone in the same situation; put a powered switch inbetween your cable modem, and the pfSense box, and you may notice your problem has been solved.
    thanks to wallabybob for his assistance !


Log in to reply