Default route changing randomly



  • I have a config with two WAN ports, one goes to the Internet with IPV6 and IPV4, the other to a bunch of internal private 10.0.0.0 networks with IPV4.  Both are NAT, and both get their IP and gateways via DHCP.  The Internet WAN port gateway is set as default, and I have gateway monitoring disabled.  There is a static route sending 10.0.0.0/8 traffic to the private WAN.

    On a random basis, sometimes once a week, sometimes a couple times a day, the default route switches from the Internet to the internal private WAN network and I lose connectivity to the Internet.  Rebooting pfsense will set things back to normal.  I can't figure out what triggers the route to change, nor how to prevent it.

    I've tried replacement hardware, and get the same result, so it isn't a hardware problem.

    I have resorted to a script that monitors the route table every 5 minutes and reboots the device when it sees the default route change.  This works, but, ugh.

    What am I doing wrong?



  • Not enough details to work with.

    Provide a network schematic (include IP subnets)

    Check system logs around the time it hops.


  • Rebel Alliance Global Moderator

    "the other to a bunch of internal private 10.0.0.0 networks with IPV4.  Both are NAT,"

    Why would you be natting to internal rfc1918 networks?  Post up your gateway section.. To route to other networks, this would not be "wan" connection with nat - this would normally be just a gateway you setup in pfsense and setup routes.

    Do you have both gateways you get via dhcp as "default"?



  • "Why would you be natting to internal rfc1918 networks?"

    Because I connect to a wireless network that I don't manage that uses rfc1918 IPs.  Each wireless node (router) in the network gets configured with a random 10.0.0.0/29 network address during initial setup on each node.  The routing for these nodes is managed with OLSR on the wireless network.  Pfsense apparently used to have a plugin for OLSR, but doesn't any longer and I cannot add routes for my internal LAN to OLSR.  Nodes can come and go without notification or coordination on this network, so I can't reasonably maintain an accurate static route list, so I have a generic static 10.0.0.0/8 route out to that network interface to cover all wireless networks.    I'm only allocated a /29 on the wireless network, and I provide services from multiple internal LAN IPs, so I have NAT configured so it only consumes one wireless IP.  This is on my OPT1 interface, and the IP and gateway are provided via DHCP from the wireless node.

    I'm open to suggestions for better ways to do this, but this is the only way I could see getting it to work with the restrictions I have.

    My internal LAN is 10.10.6.0/24.  This works fine because the LAN interface's /24 route is more specific than the wireless /8, so things route properly.

    The WAN port connects to my ISP, and is a 73.x.x.x/24 which is provided via DHCP from my cable modem.

    So to recap:

    To internet
    73.x.x.1 (gateway)
      |
    73.x.x.x/24
      WAN
    +–---------+
    | pfsense    | LAN--10.10.6.1/24----To internal LAN
    +-----------+
    OPT1
    10.117.100.157/29
      |
    10.117.100.153 (gateway)
      To a couple dozen or so random 10.x.x.x/29 networks routed by OLSR

    "Do you have both gateways you get via dhcp as "default"?"  "Post up your gateway section"
    The only gateway that is set default is the WAN (internet) side.

    This is from my /conf/config.xml file:
    <gateways><gateway_item><interface>opt1</interface>
                            <gateway>dynamic</gateway>
                            <name>MESH_NMT_DHCP</name>
                            <weight>1</weight>
                            <ipprotocol>inet</ipprotocol>

    <monitor_disable></monitor_disable></gateway_item>
                    <gateway_item><interface>wan</interface>
                            <gateway>dynamic</gateway>
                            <name>WAN_DHCP</name>
                            <weight>1</weight>
                            <ipprotocol>inet</ipprotocol>

    <monitor_disable><defaultgw><latencyhigh>1500</latencyhigh>
                            <losshigh>100</losshigh></defaultgw></monitor_disable></gateway_item>
                    <gateway_item><interface>wan</interface>
                            <gateway>dynamic</gateway>
                            <name>WAN_DHCP6</name>
                            <weight>1</weight>
                            <ipprotocol>inet6</ipprotocol>

    <monitor_disable><defaultgw></defaultgw></monitor_disable></gateway_item></gateways>

    and just for info:
    <staticroutes><route><network>10.0.0.0/8</network>
                            <gateway>MESH_NMT_DHCP</gateway></route></staticroutes>

    Normally netstat -nr shows this:
    Internet:
    Destination        Gateway            Flags      Netif Expire
    default            73.x.x.1        UGS        em0
    10.0.0.0/8        10.117.100.153    UGS        em2
    10.10.6.0/24      link#2            U          em1
    10.10.6.1          link#2            UHS        lo0
    10.117.100.152/29  link#3            U          em2
    10.117.100.153    10.117.100.153    UGHS        em2
    10.117.100.157    link#3            UHS        lo0
    73.x.x.0/24    link#1            U          em0
    73.x.x.x      link#1            UHS        lo0
    75.75.75.75        73.x.x.1        UGHS        em0
    75.75.76.76        73.x.x.1        UGHS        em0
    127.0.0.1          link#8            UH          lo0
    172.16.0.0/12      10.117.100.153    UGS        em2

    When it goes bad I see this:
    Internet:
    Destination        Gateway            Flags      Netif Expire
    default            10.117.100.153    UGS        em2
    10.0.0.0/8        10.117.100.153    UGS        em2
    10.10.6.0/24      link#2            U          em1
    10.10.6.1          link#2            UHS        lo0
    10.117.100.152/29  link#3            U          em2
    10.117.100.153    10.117.100.153    UGHS        em2
    10.117.100.157    link#3            UHS        lo0
    73.x.x.0/24    link#1            U          em0
    73.x.x.x      link#1            UHS        lo0
    75.75.75.75        73.x.x.1        UGHS        em0
    75.75.76.76        73.x.x.1        UGHS        em0
    127.0.0.1          link#8            UH          lo0
    172.16.0.0/12      10.117.100.153    UGS        em2

    I've looked through the various logs when the problem happens, and I don't see anything obviously wrong. 
    I've played with various values and ultimately disabled gateway monitoring to make sure that isn't causing the problem.