Assigning multiple WAN IP's to multiple interfaces

[dcol]

I do not want to run 4 servers off one NIC. The reason I am switching from my old firewall appliance is because of bandwidth issues and that device had 8 assignable ports which I only used 4.

I simply want to assign my 4 WAN IP's to the 4 NICs. I have a block of 8 IPs from the ISP on one WAN line.(Only use 4)
I am using Manual Outbound NAT with 4 Virtual IP's. All 4 servers can browse the internet but if you try to view a webpage on any of the servers, you get the pfsense webconfigurator. This is using the IP and not the domain name from a remote browser. If I use DNS, I get that DNS Rebind error.

So I am almost there, just need to tweak something and I tried every setting I can think of.

By the way, using NAT 1:1 does not work at all incoming or outgoing. Can't use port forwarding because I will be using the same port on multiple servers.

Temporarily I set the firewall on each interfaces to pass all traffic to eliminate any firewall blockage.

Really appreciate the help

[tim.mcmanus]

I've posted screen shots of my Port Forward, NAT, 1:1, Outbound NAT, and Rules.  As well as my LAN2 Interface.

Each of your Interfaces needs firewall rules created to communicate to other interfaces.  See my LAN2 Rules.  Port Forwarding, NAT, 1:1, and Outbound NAT are all more for external communications.  You should try first to get your server onto one of your SERVERx interfaces and then get that to go to the Internet.  So reset your router and get one server on one interface set up to at least communicate with the Internet.  That is low-hanging fruit.  If you can't do that, the rest doesn't matter.

The only interface that by default can communicate to the Internet is the first LAN interface.  All of the other interfaces lack rules to communicate anywhere else.  I suggest taking the LAN rules and copying them to SERVER1 and see if that gets your server to communicate to the Internet.

You can use the same port with multiple servers using virtual IPs.  See my screen shots to see how it's done.


[dcol]

Thanks for you response I believe it will be a great help when I try it tonight.

It looks like your configuration has 2 ISP WAN connections. One for the local network and one for your servers. WAN, WAN2
If you only have one ISP, then tell me what is plugged into the outer WAN interface
I assume you setup 3 Virtual IP's 96.57.99.139,140, and 141

using your IPs, here is my setup
10.0.1.1 - LAN - 96.57.99.138 - my 4th IP
10.0.2.1 - SERVER1 - 96.57.99.139 - Your LAN2
10.0.3.1 - SERVER2 - 96.57.99.140
10.0.4.1 - SERVER3 - 96.57.99.141
WAN - ISP connection with multi IP's

My NAT: Outbound would have only WAN to each subnet source and NAT address as WAN address
My SERVERx Firewall: Rules would be the same for each interface

[jahonix]

@dcol:

I do not want to run 4 servers off one NIC … because of bandwidth issues...

But they are all connected through one bottleneck WAN interface, right?
Do you use lots of local traffic to your servers? (Exchange Server with some Outlook clients does count.  ;-)

[tim.mcmanus]

96.57.99.138-141 are all on the same WAN2 interface.  They route to machines on my LAN and LAN2 (mostly LAN2).

WAN is a completely different WAN interface.  Yes, WAN and WAN2 are two independent and different WAN connections.

So if you follow the screen shots, a connection coming into WAN2 for address 96.57.99.140 would route (in your case) to SERVER2 (10.0.3.1).  You'll see this in the NAT screen where it comes into a public IP and then routes to a private IP.

You'll see a subsequent firewall rule to the NAT (they can be created at the same time, and I recommend this) for the WAN2 interface (which is the 96.57.99.138-141 interface) and routes that traffic to the destination server in LAN2.

LAN2 has firewall rules allowing any-to-any, so traffic can go in and out of that interface.  All of your SERVERx interfaces should be any-any to allow traffic into and out of the devices on that subnet.

However, as I stated in my initial post, start with the SERVERx rules and create the initial any-any rule FIRST.  Then see if the server can reach the Internet.  It should.  Once you've solved that issue, create the virtual IPs and then NATs, and it should just start working.

[dcol]

Thanks it is now working as expected, except the download speeds are 50% lower than when I was on the firewall appliance.
My system is a Dell with i5-4690 8GB memory, Intel i340-T4 quad NIC, and SSD drive. Should be faster internet speeds.

Is there any way to bring up the performance? Maybe there is some limiting setting somewhere.

[tim.mcmanus]

If you're not running any additional packages (and even if you were), you shouldn't see any impact to performance. Your specs seems. Rey good for the task at hand and then some.

How are you measuring download speeds and from where to where?

[dcol]

My normal bandwidth is 20Mbps Upload and 150Mbps Download. Since using pfsense I still get 20Mbps Up but only 40Mbps down. I figured it is some throttling on the downloads.

UPDATE
The servers bandwidth, clocking at 100Mbps down, is much higher than the LAN. I'd be one happy camper if the LAN did that well

I was hopping after I add a bunch of packages, which I have not done so far, I can maintain decent speeds.
I am going to post my settings shortly and lets see if anyone can spot any mistakes I may have made.

Thanks to you all, you have been a great help and frankly makes pfsense a better product.

[dcol]

Here are screenshots of my setting. Public IP's are partially masked for security.
Every setting not shown would be the default setting.

[tim.mcmanus]

Your MBUF usage is very high for a computer with your specs. Not sure why and not entirely sure it's at all related to your issue of speed. Everything else looks okay.

[dcol]

MBUF was high because of the Intel Quad NIC. I added kern.ipc.nmbclusters="1000000" to the loader.conf.local file and now the MBUF is down to 2%

Thanks for that catch.