Please HELP!! 4WAN to 1LAN LB wont work!



  • Hello everyone,

    I have been head-desking here for about 2 weeks. I work IT for a couple hotels in my area and one of them is using a pfSense router. They are upgrading their systems and have 4 modems on 4 different gateways they would like to tie together for load balancing (no modem bonding or anything like that). Super simple network setup…

    4 Modems (Each on their own gateway)

    4 WAN pfSense (Netgate FW-7541D)

    Load balance and captive portal in pfSense

    Out to one LAN port going to a switch that is connected to 4 wireless AP.

    As you can see its really not complicated at all...

    Only thing is I have never used pfSense before, I have plently of networking experience and for the last 2 weeks I've been reading guides on how to setup MultiWAN in pfSense, sadly nothing I try works. I cannot even get 2 WAN ports to work. I have tried configuring and re-configuring firewall rules, gateway groups, etc...

    I need some help. Only the default WAN and default LAN port work. My 4 added WAN ports do not function so I know it is my configuration and not the router itself.

    Thanks for any help you guys might be able to offer. I'm at my wits end.

    --Fluidic



  • Try to calm down and don't double post.
    Make sure you have NAT rules for the LAN subnet on each WAN address. You can then test by changing the default LAN rule to use one of the other gateways. If that works, you can create load balanced gateways and send the traffic there.



  • In the router under - Firewall>NAT – I do not see any rules there at all. Every tab is empty.

    I have removed all my firewall rules and gateway groups as well and set the router back to defaults as a starting point.

    I am basically going to need this explained to me like I'm a 3 year old because like I said, I have never used pfSense before.

    Edit: At this point I can't even figure out how to get a new interface (WAN port) to work correctly by itself.

    Netgate has 6 ports (eth0 - eth5)
    Default ports are eth4 (LAN) and eth5 (WAN)
    When setting up an additional interface (eth0) to be a WAN port I cannot get the gateway to show as online.
    I have tested each of the 4 modems by direct connecting them to my laptop and programming the static IP info to my nic. Each modem performs flawlessly.
    Each modem also works correctly when plugged into the default (eth5) WAN port and programmed for static.
    I cannot get any modem to work on my new (eth0) WAN port.

    Hope the additional info helps.

    -Fluidic



  • @Fluidic:

    In the router under - Firewall>NAT – I do not see any rules there at all. Every tab is empty.

    Ok. That's fine. It means you are using automatic OB NAT. As long as your WAN interfaces have a gateway defined and your LAN doesn't, everything should work.
    @Fluidic:

    Netgate has 6 ports (eth0 - eth5)
    Default ports are eth4 (LAN) and eth5 (WAN)

    Really? That sounds like Linux terminology. I would expect them to be something like em0-em5
    The Interfaces should be easy to configure. If they do not show in the GUI, go to interfaces, assign.
    Go to the OPTx interface, enable it, and select 'static IPv4' Then enter the IP, subnet mask, and gateway. Save the gateway, then save the Interface. (You have public IPs, correct?) At this point, if the gateway modem is up, you should see the status of the gateway as up. Try only plugging one in at once so you can verify you have the correct port patched to the correct modem.



  • Sorry Linux background…

    Consider eth0 to be em0 in this case they are exactly the same. Under Interfaces I have em0-5
    It looks like this:

    em0=WAN1
    em1=WAN2
    em2=WAN3
    em3=WAN4
    em4=LAN1 (Default - 192.168.1.1)
    em5=WAN5 (Default set by Netgate this interface works)

    All my modems are static IPs, each one has a different gateway.

    Lets say the IPs for the modems are

    10.11.12.13
    9.10.11.12
    8.9.10.11
    7.8.9.10

    The gateways from my ISP always end in 254 so...

    10.11.12.13 would have a gateway of 10.11.12.254
    9.10.11.12 would have a gateway of 9.10.11.254
    and so on.

    The subnet from the ISP is always 255.255.255.0 (24)

    When I setup the interface I'm selecting Static IPv4, entering 10.11.12.13 as the IP, selecting 24 after the /, and putting the gateway in as 10.11.12.254, I then leave both the boxes checked for blocking private networks. Save the changes.

    What now? What is OPTx? I don't see this anywhere...

    -Fluidic



  • After setting the static IP/net/gateway for the new interfaces I then go to Status>Gateways to check on the status of the new interface.

    Every time I setup a new interface the gateway shows 100% packet loss OFFLINE.

    However if I use the same settings on em5 (the default WAN port) and test the same modem on that port the gateway will show as online with 5% packet loss.

    -Fluidic



  • Doesn't make a lot of sense to me. Did you buy this from the store pre-loaded? You may be eligible for tech support if so.



  • @Fluidic:

    What now? What is OPTx? I don't see this anywhere…

    OPTx is an interface that is not the default WAN or LAN. In other words, the ones you named WANx were OPTx originally. You did rename them yourself, right? Or, as dotdash is asking, did you buy a pre-built solution?



  • @timthetortoise:

    @Fluidic:

    What now? What is OPTx? I don't see this anywhere…

    OPTx is an interface that is not the default WAN or LAN. In other words, the ones you named WANx were OPTx originally. You did rename them yourself, right? Or, as dotdash is asking, did you buy a pre-built solution?

    Oooooh. Ok. Yes I did rename them myself to keep organized, I have labeled my modems WAN1, WAN2, WAN3, WAN4 then labeled them the same in the router so I could tell at a glance which was which.

    For organization I wanted WAN1 to plug into em0 (labeled as 1 on the physical router itself) which is why they are named that way. I did the same with my switch for the wireless APs (Port 1 is AP1, port 2 is AP2 and so on) then to top it off the cat6 wiring is all color coded to signify different appliances. (eg. Yellow cat6 is a modem, red cat6 is from the router to the switch, blue from the switch to the APs.)

    But anyway, my organization aside. Netgate promised me that getting 4 modems on 4 different gateways into the router would be very easy. :(

    I also purchased premium tech support with this router (over $1000CAD paid for this product/support). I'm going to hold off on bashing the ever loving beans out of Netgate as my ticket with them is still open. However the ticket has been open for a few days and I'm not holding my breath.

    -Fluidic



  • Some more info:

    Netgate got back to me on my ticket… NOT happy with the news I was given... Apparently pfSense isn't able to use multi-WAN if the modems are on the same SUBNET?!?!

    Every subnet from my ISP is 255.255.255.0 - When I called to ask if they are able to change the subnet I was told they only offer 255.255.255.0

    This makes absolutely zero sense to me, I have setup load balancing and fail over in many other kinds of routers. In those routers there was no requirement to even have separate gateways, let alone different gateways AND subnets...

    So now I have 4 modems on 4 different gateways, with 4 different IPs... Anyone have any ideas about the subnet thing?

    -Fluidic



  • The same subnet does not just mean the subnet mask. e.g. If your modems were on 1.2.3.4/24 1.2.3.5/24, etc. that would not work. If they were on 1.2.5.6/24 and 1.2.6.7/24 that would work, as they would be on two separate subnets. You could have two modems with 255.255.255.0 (/24) subnets as long as the first three octets (1.2.3) were NOT identical. I would guess your problem is that they all use the same gateway. That will not work. Sorry your ISP sucks. In the US, at least, that would not be considered a business-class connection. If other providers are available, the best situation is to have connections from different providers.



  • @dotdash:

    The same subnet does not just mean the subnet mask. e.g. If your modems were on 1.2.3.4/24 1.2.3.5/24, etc. that would not work. If they were on 1.2.5.6/24 and 1.2.6.7/24 that would work, as they would be on two separate subnets. You could have two modems with 255.255.255.0 (/24) subnets as long as the first three octets (1.2.3) were NOT identical. I would guess your problem is that they all use the same gateway. That will not work. Sorry your ISP sucks. In the US, at least, that would not be considered a business-class connection. If other providers are available, the best situation is to have connections from different providers.

    The IPs are like this…

    204.16.64.69  - IP on WAN1
    204.16.64.254- Gateway on WAN1
    255.255.255.0- Subnet on WAN1

    69.71.11.30  - IP on WAN2
    69.71.11.254- Gateway on WAN2
    255.255.255.0- Subnet on WAN2

    71.39.111.14  - IP on WAN3
    71.39.111.254- Gateway on WAN3
    255.255.255.0- Subnet on WAN3

    64.9.42.92 - IP on WAN4
    64.9.42.254 - Gateway on WAN4
    255.255.255.0 - Subnet on WAN4

    They are all completely separate IPs on different gateways, the only thing that is the same is the fact that they all use 255.255.255.0 as the subnet... This still will not work?

    -Fluidic



  • Assuming your real numbers are similar, they should work. (I'm assuming an ISP that was assigning IPs from a /24 would not be capable of offering IP space from four different providers. It is within the realm of possibility that I am in error)
    The important points would be that you are getting public addresses, that they are not using the same gateway, and that the IPs are not on the same subnet. (they could have the same subnet mask).



  • @Fluidic:

    @dotdash:

    The same subnet does not just mean the subnet mask. e.g. If your modems were on 1.2.3.4/24 1.2.3.5/24, etc. that would not work. If they were on 1.2.5.6/24 and 1.2.6.7/24 that would work, as they would be on two separate subnets. You could have two modems with 255.255.255.0 (/24) subnets as long as the first three octets (1.2.3) were NOT identical. I would guess your problem is that they all use the same gateway. That will not work. Sorry your ISP sucks. In the US, at least, that would not be considered a business-class connection. If other providers are available, the best situation is to have connections from different providers.

    The IPs are like this…

    204.16.64.69  - IP on WAN1
    204.16.64.254- Gateway on WAN1
    255.255.255.0- Subnet on WAN1

    69.71.11.30  - IP on WAN2
    69.71.11.254- Gateway on WAN2
    255.255.255.0- Subnet on WAN2

    71.39.111.14  - IP on WAN3
    71.39.111.254- Gateway on WAN3
    255.255.255.0- Subnet on WAN3

    64.9.42.92 - IP on WAN4
    64.9.42.254 - Gateway on WAN4
    255.255.255.0 - Subnet on WAN4

    They are all completely separate IPs on different gateways, the only thing that is the same is the fact that they all use 255.255.255.0 as the subnet... This still will not work?

    -Fluidic

    That will work fine. The subnet mask in combination with the IP address is what determines your subnet, and none of them are the same. Let Netgate know that they are all indeed different subnets.