Multi WAN - same ISP, same WAN Gateway (Utilizing two pfsense Boxes)
-
I have two WAN IP addresses which are provided by the same ISP (Same WAN Gateway). I want to have one IP for all internet traffic and the second IP for the Mail Server.
I understand that I can't have multiple WANs utilizing the same gateway so I would like to know if the following scenario is possible -
- Setup two seperate pfSense routers. (Lan ips 10.1.1.1 and 10.1.2.1)
- In the first pfSense Box (10.1.1.1) System: Routing: Edit gateway, I add on the LAN interface a Second Gateway (GW_WAN2) (Which is the LAN address of the second pfSense Box - 10.1.2.1)
- Create rule on the first pfSense Box (10.1.1.1) to route certain traffic (from the internal LAN source of pfsense box one 10.1.1.1) to the GW_WAN2
- Create forwarding rules on pfSense Box 2 to route inbound traffic to internal LAN of pfSense Box one.
Will this work properly?
I tried this scenario and created a Rule to route a Mail Server which was on the LAN of pfSense Box one (10.1.1.1) and routed it to GW_WAN2.
I could see the states being created in the Second pfSense box but the Mail Server mail was being rejected by every other Mail Server it was connecting to. One of the messages is below.5.2.1 : AOL will not accept delivery of this message. (in reply to end of DATA command)
Any advice?
-
I think you just need manual outbound NAT, on a single pfSense.
e.g. your WAN has multiple public IPs 11.22.33.2 and 11.22.33.3 with gateway 11.22.33.1 - make a manual outbound NAT rule that NATs traffic from your mail server IP on LAN to 11.22.33.3 and another rule that NATs the rest of the LAN traffic to 11.22.33.2 - then mail server outgoing connects to other mail servers will have a source address of 11.22.33.3
You can also add port forward/s for 11.22.33.3 to the mail server on LAN, and outside stuff can connect in to the mail server.
Have a look at this thread - it does similar for multiple LAN subnets: http://forum.pfsense.org/index.php/topic,63666.msg350577.html#msg350577 -
I have a very similar setup:
Setup:
WAN has IP1 11.11.11.1 and Gateway 20.1.1.1
OPT1 has IP2 11.11.11.2 and no Gateway (as I could not add Gateway with same IP 20.1.1.1)I need my mailserver LAN 192.168.1.2 to go out trough the OPT1 interface IP 11.11.11.2
I have Firewall NAT 1:1 defined as
OPT1 11.11.11.2 192.168.1.2 *But I can see in outgoing email transcripts the 11.11.11.1 IP instead of 11.11.11.2.
Is the 1:1 NAT not the right way to go? Should I remove 1:1 part and change NAT Outbound from Automatic Mode to Manual and then add Mapping? If yes, then what exactly do I place in the fields of new Manual Outbound form?
Thank you for response.
-
I have two WAN IPS both sharing the same Gateway address.
When I add the second WAN2 i cant select the same Gateway. Do I add a Gateway for WAN2 using the LAN address of the first Gateway?
When I enable OUTBOUND NAT Traffic is not going out WAN2 with the rule that I added. I assume its a Gateway issue?
Any suggestions?
Here is my outbound NAT
WAN2 10.1.1.9/32 * * * WAN2 address * NO
WAN 10.1.1.0/24 * * 500 WAN address * YES Auto created rule for ISAKMP - LAN to WAN
WAN 10.1.1.0/24 * * * WAN address * NO Auto created rule for LAN to WAN
WAN 127.0.0.0/8 * * * WAN address 1024:65535 NO Auto created rule for localhost to WAN
-
Hoping to get a suggestion on the issue above?
Appreciate the help.
-
Hi,
if this task is still open / for other if they find this thread by searching…
Normally an ISP should give you a single IP or a small network which you can define on WAN Interface and use it as is.
But if you got only 2+ single IPs or a 2nd separate network from your ISP then you must do a little "Trick" on WAN Interface:
-
Add within Firewall => Virtual IP an IP Alias for additional IPs/networks
-
Activate different public IP settings by deactivating automatic Outbound NAT
in Firewall => NAT => Outbound NAT -
You can clear a lot of unneccessary Outbound NAT rules but don't forget to have at least 1 outbound NAT which matches your LAN's to your default public IP.
-
Then you can add before this rule your explicit server outbound NAT rule to use the other public IP (in interface list you can select IP Aliases, CARP IPs and Host Aliases from Firewall Aliases)
If you use CARP with firewall master/slave setup you must have a network assigned by ISP.
For instance you got 198.51.100.0/28 assigned.Then normally
-
the ISP uses 198.51.100.1 as his GW IP and let you use the rest of IP as you need
(don't forget the last IP address of /28 is reserved broadcast address, too) -
the master FW should get e.g. 198.51.100.3,
-
the slave FW should get e.g. 198.51.100.4 and
-
the virtual CARP IP would be e.g. 198.51.100.2
and other public IPs for mailserver, pbx, webserver, … public IP for each internal LAN e.g. can be setup as additional CARP IPs / IP Aliases.
-
Normal behavior for CARP ist that each CARP fails "standalone". :(
So if either on WAN or LAN interface communication is dropped but not on other interface, then you got in a malfunction state till you cut off master firewall completely.
You can fix this partly by setting within
=> System => Advanced => System Tunables-
key: net.inet.carp.preempt
-
comment: CARP Interface group failover (group carp)
-
value: 1
but this helps only for physical recognized fails…
If there were WAN connection failures, network disconnections behind used switches or unknown network card issues that let stay network port up then CARP system cannot help to failover to slave firewalls.
There must be a carp group down/up event scripted for this functionality and I hope this task is on feature list ;) -
-
I read in one thread that IP Aliases could be setup on CARP IP so that it's much easier to handle a huge amount of virtual IPs without problems (you can setup only up to 256 CARP IPs in sum for all interfaces) but in my tests I had problem with NAT/ firewall rules (https was not NATted but I got pfSense webgui :( ).
If your ISP assign you a 2nd network, e.g. 192.0.2.0/28 then the setup is mostly equal:
-
you got ISP gateway 192.0.2.1 on his side,
-
your master firewall is 192.0.2.3,
-
your slave firewall is 192.0.2.4 and
-
virtual gatewway IP is 192.0.2.2
The big question here is often howto setup this on the same WAN interface…
The answer is normally easy - with Virtual IP Aliases.-
the ISP gateway for both networks is normally the same router, so for ISP it's not interesting if you sent packages to 192.0.2.1 or to 198.51.100.1 - the packages would find their way ;)
-
But if you need additional gateway it should be easy setup by System => Routes => Gateways
-
master/slave firewall IPs are setup as Virtual IPs
-
your gateway CARP IP can be then added to WAN interface because of IP Aliases there is a matching network found for CARP IP
Would be nice if someone with WIKI documentation rights could add the not yet available steps/ tipps to it. ;)
-
-
Thanks Reiner,
For my setup i have 2 assigned WAN IP addresses with the same gateway.
Im still unclear how to assign the IPs as you had indicated in your post. Could you provide an example utilizing the IPs listed below?
- WAN1 (10.10.10.1) and WAN2 (10.10.10.2) are static addresses as provided by my ISP with a GW say (20.20.20.1)
I define the two static Wan addresses in "Interfaces" WAN1 is using GW 20.20.20.1, What do I set as the WAN2 Gateway?
-
I understand setting the Manual Outboard NAT and adding a first new entry to NAT a single LAN address to WAN2 but need some help with the IP alias setup.
-
The internal LAN is say 10.40.40.0/255, and the single address that needs to use the outbound WAN2 would be 10.40.40.9
Really appreciate the help.
-
Hi,
btw… you can use my IP examples... The networks came extra from http://tools.ietf.org/html/rfc5737
3. Documentation Address Blocks The blocks 192.0.2.0/24 (TEST-NET-1), 198.51.100.0/24 (TEST-NET-2), and 203.0.113.0/24 (TEST-NET-3) are provided for use in documentation.;)
@BBcan17:
For my setup i have 2 assigned WAN IP addresses with the same gateway.
Im still unclear how to assign the IPs as you had indicated in your post. Could you provide an example utilizing the IPs listed below?
- WAN1 (10.10.10.1) and WAN2 (10.10.10.2) are static addresses as provided by my ISP with a GW say (20.20.20.1)
I define the two static Wan addresses in "Interfaces" WAN1 is using GW 20.20.20.1, What do I set as the WAN2 Gateway?
You should delete WAN2 Gateway because you don't need it ;)
I guess you have defined 10.10.10.1 in WAN1 interface and added now IP Alias 10.10.10.2 to interface WAN1 (which you can rename then back to WAN).@BBcan17:
- I understand setting the Manual Outboard NAT and adding a first new entry to NAT a single LAN address to WAN2 but need some help with the IP alias setup.
- The internal LAN is say 10.40.40.0/255, and the single address that needs to use the outbound WAN2 would be 10.40.40.9
Looks more like a mess… LAN/WAN with same network range? and /255 isn't possible - even with IPv6;)
So I try to setup some demo enviroment...
-
ISP has 198.51.100.1
-
you got 198.51.100.22 as 1st IP
-
you got 198.51.100.33 as 2nd IP
-
you have internally LAN 192.168.1.0/24 (default pfSense LAN range)
-
and your special server for 2nd public IP has IP 192.168.1.11
Then you have something setup in WAN interface that told pfSense to have own public IP 198.51.100.22 and set WAN GW to 198.51.100.1.
Now with your 2nd IP 198.51.100.33 you go to Firewall => Virtual IPs and add there a new IP… type IP Alias... Interface WAN, IP Address 198.51.100.33/32 and you can set a comment that its for your service xyz.
Now you have 2 possibilitys for incoming service xyz
-
setting up "Firewall" => "NAT" => "Port Forward" rules for each port/service with "Destination" (select => IP Alias: 198.51.100.33) to your server 192.168.1.11 with automatically generated firewall rules.
-
setting up "Firewall" => "NAT" => "1:1" for server 192.168.1.11 and open manually wanted port/services firewall rules (important: destination must be internal IP as by auto-generated NAT rules done)
With incoming NAT on 2nd IP address answer packets got automatically the 2nd public IP.
To set 2nd IP address for direct outgoing packets from your internal server .11 you need at least 2 rules in "Firewall" => "NAT" => "Outbound" (order is important):
-
Interface WAN, Protocol Any, Source: 192.168.1.11/32, Destination: Any, Translation: IP Alias: 198.51.100.33, Description: service xyz
-
Interface WAN, Protocol Any, Source: 192.168.1.0/24, Destination: Any, Translation: IP Alias: 198.51.100.22, Description: default out
Then all should work as expected ;)
Bests
-
Thanks Reiner,
I set that up now and it works fantastic. Thanks for your help. 8) 8) 8) Its very appreciated.
-
Hi Reiner,
My WAN2 is set as Static as provided by my ISP.
Question - How would you get a Virtual IP to work for a 2nd WAN that needs to login to get connected (ie PPOE)?
I also thought that my ISP set a MAC address to each of my IP so I am not sure how i have both of my WAN IP's utilizing the same Mac address?
-
Hi BBcan17
@BBcan17:
My WAN2 is set as Static as provided by my ISP.
Question - How would you get a Virtual IP to work for a 2nd WAN that needs to login to get connected (ie PPOE)?
I also thought that my ISP set a MAC address to each of my IP so I am not sure how i have both of my WAN IP's utilizing the same Mac address?
mmh, good question…
If I remember right this was discussed several times in forum and there is no easy solution yet possible:
The idea was to create an additional interface on pfSense (different network port, or VLAN) and put pfSense behind a stupid or VLAN capable switch to "join" both ports/VLANs to the needed 1 port of modem/leased line.So for pfsense you have different WAN Gateways with MAC Address assigned IPs... which uses the same modem/gateway on the "other side" of the switch.
-
This is not working in 2.1.4 when routing from OPT1 interface to WAN VIP