3 WAN to 1 LAN



  • I am currently using 1 WAN and 1 LAN with two virtual IP:s.

    My current setup is the following:

    GATEWAY1
          |
       WAN1 (Interface IP: 100.100.100.100, Virtual IP: 100.100.100.101, Virtual IP: 100.100.100.102)
          |
     PFSENSE
          |
        LAN1 (10.0.0.1)
          |
    S1–S2--S3 (S1 = Server1 10.0.0.10, S2 = Server2 10.0.0.11, S3 = Server3 10.0.0.12)

    NAT-rules:

    100.100.100.100 -> 10.0.0.10 (Port 80, 443)
    100.100.100.101 -> 10.0.0.11 (Port 80, 443)
    100.100.100.102 -> 10.0.0.12 (Port 80, 443)

    This setup is working perfectly fine and I can communicate with S1 on 100.100.100.100, S2 on 100.100.100.101 and S3 on 100.100.100.102

    I have been trying to change this setup to the following since my new Internet provider does not allow virtual IP:s, I am only allow to use one IP-address for each MAC address.

    My new setup would be the following:

    GATEWAY1                       GATEWAY1                       GATEWAY1
         |                                    |                                     |
      WAN1 (100.100.100.100)    WAN2 (100.100.100.101)     WAN3 (100.100.100.102)
         |                                    |                                     |
          ---------------------- PFSENSE-----------------------
                                               |
                                             LAN1 (10.0.0.1)
                                               |
                                        S1--S2--S3 (S1 = Server1 10.0.0.10, S2 = Server2 10.0.0.11, S3 = Server3 10.0.0.12)

    NAT-rules are the same:

    100.100.100.100 -> 10.0.0.10 (Port 80, 443)
    100.100.100.101 -> 10.0.0.11 (Port 80, 443)
    100.100.100.102 -> 10.0.0.12 (Port 80, 443)

    Is this a possible setup to do today with pfSense?

    In my test environment I have been able to setup this Multi-WAN and it is possible for S1, S2, S3 to communicate out from the network. Communicating with S1 works fine when accessing 100.100.100.100, but I am not able to communicate with S2 from 100.100.100.101 or S3 from 100.100.100.102

    Am I only missing some firewall rules or is my communication problems with S2 and S3 related to pfSense not allowing the same gateway on the WAN:s where each separate WAN have its unique MAC-address?

    Do I need to throw in some NAT devices, WAN2 -> NAT -> GATEWAY1 and WAN3 -> NAT -> GATEWAY1 for this setup to work?

    The servers, S1, S2, S3, have to be on the same LAN since they are communicating with each other using their internal IP:s.



  • That certainly is a unique problem. If you use CARP vip, each interface will have a unique MAC address iirc. A derivative of the original or something. This who use IP alias could tell you if that one does the same. ProxyARP I would imagine would use the same MAC for all.
    Here is mine as an example:

    (10.x.y.1) at 00:00:5e:xx:yy:83 [ether] on eth0 <–- this is the CARP VIP
    (10.x.y.2) at 00:30:48:zz:aa:c4 [ether] on eth0 <–- this is the REAL interface
    This is from the arp tables on a machine behind this FW.

    Personally, I would change providers (or in this case never signed up with them). It just seems very unpractical to ask for something like that. All firewalls I know work with VIPs. This would be a huge problem for me as I have 64 addresses in one location. There are only a very few systems can could even get that close for a port count. 
    Did they tell you why they want something as insane as that?



  • I'm going to run these three servers on my new home 100/100 Mbit fiber connection. My new Internet provider uses DHCP to lease IP-addresses and only one IP-address is leased for every unique MAC. But I'm allow to lease up to 10 IP-addresses using different MAC:s. Virtual IP:s (IP Alias) share the same MAC as the parent interface and that is the reason I can't use Virtual IP:s (IP Alias).

    But if I understand you correctly I can use Virtual IP:s (CARP) and every Virtual IP will get its own MAC, or I am wrong? If this works this would of course be the best option for me since I can keep my original setup! :)

    But regarding my question about Multi-WAN:

    Is it possible today using pfSense to connect three WAN to the same gateway (where each WAN are having a unique MAC address) and then direct the incoming traffic from each WAN to an internal IP-adresses using NAT?

    Or is my communication problem with S2 and S3 related to pfSense not allowing the three WAN:s to connect to the same gateway? Or is this a valid setup in pfSense and I only need to create some additional firewall rules besides the normal NAT-rules?



  • I did look into CARP VIP:s but I didn't get it to work. Maybe a little to complicated for me.

    But maybe I can run a separate instance of pfSense for each and every interface instead and then every server can have it's own gateway. Then I don't run into the problem with Multi-WAN. I'm running pfSense virtualized so this may be an option.

    GATEWAY1                       GATEWAY1                       GATEWAY1
         |                                    |                                     |
      WAN1 (100.100.100.100)    WAN1 (100.100.100.101)     WAN1 (100.100.100.102)
         |                                    |                                     |
    PFSENSE1                        PFSENSE2                         PFSENSE3
         |                                    |                                     |
          –---------------------- LAN1 ------------------------
                                               |
                                        S1--S2--S3 (S1 = Server1 10.0.0.10, S2 = Server2 10.0.0.11, S3 = Server3 10.0.0.12)



  • Well .. that is an option .. 3 FW to manage and no central way to traffic shape or manage.
    I am not well versed in multi-WAN to help with that.
    They sure do have a backward way of doing things at the ISP. Anyway .. good luck.



  • Well, it is for sure not an optimal setup but it may be the only one I can use if I don't get CARP VIP:s working correctly.
    And I am guessing the problem I'm having with the Multi-WAN setup is related to the gateway issue so that is probably a dead end if I don't throw in some NAT devices. But adding NAT devices will probably slow down transfers more than running three firewalls in parallel.



  • Yeah … I cannot get past 3 separate FWs in my head either. I keep thinking also 3 separate WAN interface with DHCP on and 1:1 NAT might actually work, but I don't really know. This is not a true multiwan setup any how so balancing and what not is does not come into play.



  • Yes, you are correct, balancing doesn't really matter in this case.

    I would like to thank you for your time since your answers somehow got me thinking of the 3 separate FWs. I don't really know how, but sometimes it really helps to just get some feedback to make you look at the problem from a different angle.

    So thank you again for taking your time. :)



  • Did you ever get this to work like you had planned…what where the results...where you happy with them?


Locked