Dual wan- wan1 down - no web browsing



  • Hello All,

    Sytem info:

    pfSense 1.2.3
    dual wan - 2 ISP's
    wan Load Balancing setup
    wan Failover setup
    Squid
    Squidguard

    Still new to pfSense FYI.

    Problem:

    Everything on our new pfSense setup is working fine,,,except,,, when I unplug the WAN ethernet cable web browsing stops.
    I can, though, ping google.com & i do get the correct ping response/ip number. This is both from the pfSense machine as well as a client machine on the network.
    If I unplug the Opt1/second  isp ethernet cable(and have wan plugged in)  web browsing continues.

    Setup info:

    1. I have setup the Load balancing setup to use each isp's gateway as the monitor address.
    2. I have setup the two failover pools to use one of each of the isp dns servers as the monitor ip address. ( Everything here shows green as expected as long as both ethernet cables are plugged in ,FYI.)
    3. In the dns servers listed in the web-ui I have one for isp1 and one for isp2.
    4. When I unplug either of the two wan ethernet cables I can still ping both public gateways of the two isp's as expected.( I believe this is correct functionality?)
    5. I have setup in the firewall lan rules to use the load balancer as the gateway (not default).

    I am very familiar with linux speak but not at all familiar with FreeBSD in some of the nomenclature. In other words when I go to look at Diagnostics>Routes in the web-ui I am not sure what I should be seeing in regards to the 'flags' entries. I am thinking this is were I need to be looking in regards to when one of the two wan links is down this routing table should change to some degree,and I don't know what in fact I should be seeing here change.

    Edit:
    I wanted to add a screenshot of my Diagnostics>Routes.

    I am puzzled as to why when looking at this,for my wan2/fxp3 the .34 address is showing as interface 'lo' rather than what it should be of fxp3. The .33 address is the gateway for .34 as described and it shows as being the fxp3? The mac addresses listed at each of the these entries are correct (the .33 address is actually the dsl router,but shows fxp3 ?). Do I possibly need to change the dsl router to bridge mode? I never had to mess with this setting with previous firewall,FYI.
    I am sure someone that has done several installs of pfSense can decypher this easily.

    At this point I am stumped in regards how to troubleshoot this scenario.
    Any ideas appreciated.

    Thank You,
    Barry



  • Rebel Alliance Developer Netgate

    1. When I unplug either of the two wan ethernet cables I can still ping both public gateways of the two isp's as expected.( I believe this is correct functionality?)

    If you unplug a WAN you should not still be able to ping its gateway. Do you see any alarms in the system log when you unplug a wan?

    And I can't tell from your screencap but it doesn't look like your DNS servers have static routes. I don't see a static route for any IPs on WAN1 (where the default route is)



  • jimp,

    Thank You much for the reply. I do not have any staic route for either of the dns servers. I must have misunderstood the doc on the multi wan setup. I took it to be if you used the dns servers for each of the two wan monitor ip's you didnt need to do a static route entry.
    I'll  go back through the multi-wan doc on here and look it over again,
    What you say makes sense in that if no static route no dns being served to the still working wan link.
    OK. I did check again and the downed wan link I can not ping that wans gateway.

    my static route would be Interface WAN > dest=d.n.s.ip/32  gateway = wan.gateway.ip

    Barry


  • Rebel Alliance Developer Netgate

    Normally setting a monitor IP will add a route automatically. I'm not sure why that did not appear in the route output you showed.

    That static route sounds fine.



  • jimp,

    If you look at the screenshot i attached at line # two(opt1/wan2/fxp3) you can see the .242(dns-ip) shows a route to .33(gateway). This is without any static route added manually.
    What puzzles me(if you look a few lines down) is the .34 address which is in fact the opt1/wan2 (public-static ipaddress)shows as interface 'lo'? If I look at the same Diagnostics>Routes today that entry is not there. I have not changed a thing or even rebooted the pfSense machine.
    I did manually add the static route just earlier today for the opt1/wan2 for dnsip/32 to opt1 gateway, & when I unplug wan ethernet I can no longer ping opt1>dns ip address? strange.
    I know this gets very frustrating for someone to try and give help,without knowing so many variables involved in each pfSense setup.

    Wanted to add:
    I am seeing the exact same behavior on two pfSense boxes that I set up for two different school buildings.
    I may add that at each of the buildings:
    WAN= plugs into an bridge that is fed to the building from a "wireless consortium of a few local schools" that is equivilant to a t1 line. This has a static IP.
    OPT1= plugs into a convention dsl router that is supplied by Frontier.This is a static IP.
    Didn't know if this info may help someone with a possible idea.
    My problem is we have an internal email server so I can not have either one of the two down for any length of time, as I will be  getting phone calls big time! Everyone now has blackberries that hit the email server every 5 mins!,,,:(.
    I'll try and go in on a Sunday eve. and do some extensive hit and miss with this delima.

    It's a puzzler that when the WAN ethernet is unplugged the OPT1 dns server is no longer pingable,and hence no web browsing? This is with or without adding the two wan's dns servers to a static route.

    Thank You,
    Barry



  • More info on the setup.

    Should i be seeing on any client on the network my actual internal ip address? This is what the client machines sees as their ip address.
    Web browsing does work fine, but it seems each client should be seeing one of the two public ip addresses that pfSense box runs on?
    I thought I was fairly familiar with Squid (and Squidguard) but I am guessing this needs to be adjusted somewere to make the clients see their public ip address?
    I have not enabled nat reflection. Should I enable this?

    Thank you,
    Barry


  • Rebel Alliance Developer Netgate

    I didn't notice you were using squid before (an oversight on my part) – squid does not work with multi-WAN on 1.2.x. It only routes out WAN -- which is why you are not able to browse when WAN1 is down.

    Your local network clients will only talk to your pfSense box on the LAN side. They don't talk directly to the 'public' IPs in most cases. (NAT reflection makes it look like they do, but they really don't) That will have no bearing on squid being compatible with multi-WAN setups.

    It is possible to load balance squid in the 2.0 beta, but it's a bit complex yet and not very intuitive. There are some threads in the 2.0 forum with input on how to do this.



  • jimp,

    Thank You much for the good explanation. At least now I know what the prob is. Soo, If I simply disable Squid( or uninstall Squid/Squidguard),,the clients should be able to web browse OK? I am guessing with my setup.
    That's a bummer as for a school scenario we have to have some sort of content filtering in place.
    I must have missed this altogether in that Squid will only work on WAN(1).
    I'll do some better searching next time I guess.

    Thank You,
    Barry


  • Rebel Alliance Developer Netgate

    If you use a separate squid server (in a DMZ) you could get that to balance and still have your content filtering. It's just that the squid process runs on the router itself, and the policy routing needed for multi-wan only works in 1.2.3 when traffic enters the LAN, not when it leaves WAN.



  • jimp,

    is there anyway I could enter a lan rule that if WAN went down that port 80 would bypass squid and come/go directly out OPT1? This will still make web browsing transparent(although unfiltered) for users and wouldn't have to babysit the pfSense box should WAN go down?

    Thanks,
    Barry


  • Rebel Alliance Developer Netgate

    No, that isn't possible. The transparent redirect is a NAT rule, not a firewall rule, and can't be overridden in that manner.



  • That's a bummer as for a school scenario we have to have some sort of content filtering in place.

    Have you tried using OpenDNS.



  • Hi Perry,

    Thanks for the suggestion. I have 'heard' a lot about OpenDNS,but never really checked into how it works. I'll do a search here in the forums to see what I can find. Have you set up a pfSense box to work with OpeDNS,and squid to work on either of the two WAN links?

    Thanks,
    Barry



  • Hello All,

    Thanks to All of suggestions made. I have decided a pretty easy workaround for our setup is to ( if WAN goes down) to do Squid,uncheck allow users on proxy, Squidguard,uncheck enable SquidGuard, Do status>Services, stop Squidguard,stop Squid.
    This will allow web browsing,on WAN2 although unfiltered of course. This will be much simpler as our previous firewall setup,as with it,if either WAN went down I had to physically go to server room and readjust wires on the firewall. ( I have to be a two remote buildings besides here during the day.
    Once the WAN is restored I can reset Squid back to original setup.
    With this setup I can at least WEB-UI into the pfsense from any building I am at,and make changes and web browsing will be restored in just a few minutes after I start getting phone calls telling me internet is down,,,:)
    BTW: I did try the Opendns suggestion and I got the same results as with the two ISP dns servers.
    I may eventually try pfSense 2.0 and see if I can get Squid to work on both WANs.
    I have quite a time getting everything ironed out on these two pfSense boxes at two buildings I would like to leave them as they are for right now.

    Thank You,
    Barry Cisna


Locked