What is best practice for my scenario

[mrcha0s]

Hello,
I'm planning to use pfSense in data center. I have 8 web servers. Mostly web traffic and 250 Mbit per second average. For DDOS attacks I'm using Cloudflare so function of firewall is NATing external ips to internal ips, allow web traffic and block all other traffic. Also I will have rules allow specific IPs to all ports.
Could tell me following hardware is good enough ?
Dell R230 with single Intel(R) Xeon(R) CPU E3-1220 v5 @ 3.00GHz and 32 or 64GB DDR4 ram.
In case of attack, Is it possible to (temporary 2-3 hours) handle 1Gigabit traffic with this hardware ?

In my scenario I have juniper ex4300 switch with 2 vlans.
Blue vlan for external ips. Yellow vlan for internal ips.
Blue vlan port 1 is come from datacenter and 45.46.44.0/24 subnet and gateway is 45.46.44.1
Blue vlan port 4 connected to firewall server nic1. It is WAN and ip is 45.46.44.2/24
Blue vlan port 2 and 3 I have servers and I directly connect them. I don’t want to use them behind firewall. I will give them ip like 45.46.44.3 and they directly connect to internet.
Yellow vlan port 1 is firewall server nic2. It is LAN and ip is 10.0.0.1/24
Yellow vlan other ports connected to my servers 10.0.0.2,3,4,5… and gateway is 10.0.0.1 (firewall)
In my scenario I have 20 external ips to use front of the firewall and 60 external ips to give servers behind to firewall and I want to use 10.0.0.0/24 subnet for servers behind the firewall.
I will create virtual IP for each ip I will use behind to firewall (45.46.44.11,12,20,25,30,50) and 1:1 NAT to internal IPs and Its not always match like 45.46.44.24 -> 10.0.0.24 some times it will be like 45.46.44.24 -> 10.0.0.9
Is adding 100-150 ips as a virtual ip effect performances ?
Could you advise me best practice for this scenario ?
Instead of this using bridge mode is more performant ? I will have two external ip subnets. One of them for servers front of the firewall other is for servers behind to firewall and pfSense for on bridge mode.
In all scenarios pfSense’s function will be; allow web traffic for all ips and block all ports. Some special ips can reach all ports.

Sorry about my English.
Thanks.

[JeGr]

As for the hardware, when reading datacenter E3 seems a little on the light side for me, but if you're only routing aroung 1gbps in sum (you didn't specify) that should work well. 32GB RAM should be more then enough if you don't deal with heavy hitter packages. Even with 8-16G I haven't seen that much trouble. 64G would probably a waste if you don't run RAM heavy packages.

as for your questions:

I will create virtual IP for each ip I will use behind to firewall (45.46.44.11,12,20,25,30,50) and 1:1 NAT to internal IPs and Its not always match like 45.46.44.24 -> 10.0.0.24 some times it will be like 45.46.44.24 -> 10.0.0.9

Do you have strict 1:1 matches from external to internal IPs? As you wrote sth about 60 external IPs to give to internal ones I wasn't sure.
If you just do a simple mapping of one external IP to one internal IP, 1:1 NAT (or BiNAT) is your friend. But don't try that if you want multiple external IPs to go to the same internal IP as this could be very confusing for the filter as to what IP to use when that internal IP initiates connections to the internet itself.

So for real 1:1 mappings, feel free to use BiNAT and don't forget the appropriate firewall rules. They aren't added automatically like port forwards so you have to manually add them (and remember the destination is the internal IP as the NAT rewrite happens before filtering it!).

Is adding 100-150 ips as a virtual ip effect performances ?

Not that I'm aware of. We hat around 64 at one time (as our old /24 subnet was configured with .1 being the ISPs gateway). But with getting our /22 RIPE assignment, we now get this /22 routed via a /29 transit network, so we don't have to configure virtual IPs anymore (if you get them routed and you only map them via NAT/BiNAT, you don't have to add alias IPs as the IPs get routed to pfSense anyway).

Could you advise me best practice for this scenario ?

That depends. If you have multiple ports or different configurations per IP, 1:1 NAT could be easier for you. If there are cases that you have to map multiple external IPs to the same internal IP, I'd go with port forwards + custom manual outbound NAT entries. It's more work (as you have to configure port forwards for incoming and outbound NAT for outgoing IP mapping) but it's more flexible and the firewall rules get linked and created automagically. Both are valid configurations.

Instead of this using bridge mode is more performant ? I will have two external ip subnets. One of them for servers front of the firewall other is for servers behind to firewall and pfSense for on bridge mode.

I'd never ever use bridge mode in that scenario. EVER. Just no!
There may be users that like bridge mode better, but I don't see more performance with that and only trouble with debugging and troubleshooting ahead. You'd have to torture me to set it up that way ;)

Also on a more serious note: you will miss out on the possibility to simply switch a server to a new one by simply pointing the NAT entry from <internal_oldIP> to <internal_newIP>. With webhosting or the likes this is really a nice thing to have (besides NAT being a damned thing that should die ^^). Rolling out a new version of a customer system/server? Just add it with a new internal IP, test it, change the 1:1 NAT or port forward entries -> profit :)

In all scenarios pfSense’s function will be; allow web traffic for all ips and block all ports. Some special ips can reach all ports.

Clean and simple, yes. Most common modus operandi for typical web VMs/server/containers etc.

[riahc3]

@mrcha0s said in What is best practice for my scenario:

I'm planning to use pfSense in data center.

I read everything else but this is pretty much all that is needed...

As much as people want to sell you pfSense on here, pfSense is not a enterprise product. Cisco, Palo Alto, Juniper, HPE, Dell, Huawei....Those are vendors that will sell you a enterprise router. pfSense at best is small to medium business.

Go with another solution other than pfSense (if you have the budget, Palo Alto. Really awesome product)

[stephenw10]

There are many, many people running pfSense in data centers in exactly this sort of setup with no issues.

That hardware is more than sufficient. I would agree it's probably overpowered for what you've specified.

Do you actually have the full /24 of public IPs? Really you don't want to be NATing that at all if you can avoid it. If would be far better to just use that subnet directly on the internal interface. That would require your provider to route it to you via another IP though. If they can do that that's what you should do.

Steve

[mrcha0s]

Thank you for your answers. @JeGr @riahc3 @stephenw10
Budget is a issue, so I go with pfsense. My future plan is using 2-3 pfsense on cheap hardware and split traffic to them. I will setup backup servers as a vm so they will share resource and in case of fail over they scale up dynamically. Now I'm testing my setup I have problems with HA. When I enable HA WAN traffic lost connection during long downloads and on fail over downloads interrupted. I'm investigating issue right now.

Thank you

[Derelict]

hardware in an HA pair should be identical. Having a physical primary and a VM secondary is only going to be grief for you and will likely never work as well as a matching pair would.

[stephenw10]

Is your provider really not able to just route that subnet to you? That would be a much better setup.

What is your reasoning for splitting the subnet between 3 firewall pairs?

And, yeah, VM Secondary to a physical Primary is probably going to be painful.

Steve

[riahc3]

@mrcha0s said in What is best practice for my scenario:

Thank you for your answers. @JeGr @riahc3 @stephenw10
Budget is a issue, so I go with pfsense. My future plan is using 2-3 pfsense on cheap hardware and split traffic to them. I will setup backup servers as a vm so they will share resource and in case of fail over they scale up dynamically. Now I'm testing my setup I have problems with HA. When I enable HA WAN traffic lost connection during long downloads and on fail over downloads interrupted. I'm investigating issue right now.

Thank you

If you are setting up a data center and will use this in production, then you should gear your budget towards networking as it is the backbone of any infrastructure.

And pfSense is not a enterprise ready solution.

I highly suggest you consider this because when your network goes down and pfSense does not meet your RTO...

[stephenw10]

It seems like you've had some sort of bad experience here but I assure you there are many many people using pfSense successfully in exactly this sort of setup.
Not really sure what you're contributing at this point.

Steve

[JeGr]

Ah it's Mr. Discord guy... again. Sorry have to do this:

And pfSense is not a enterprise ready solution.

Yeah bullshit. There! I said it.

So back to topic: Again, yes it is. Otherwise we've been doing things wrong for the last 10y or so and I don't see that when I look into our network or customer ;) Indeed I've seen running pfsense in datacenter environments lots of times now. Not because of some BS reason or budget, but because it works and does a fine job when you have your facts straight and your requirements written out beforehand. But running pfSense on old hardware some time and then start bickering about "lack of CLI" or "automation" or "no API" or "speed (10G is not enough)" ... some users clearly didn't do a very good job at the planning stage (should've gone for TNSR in the first place in that example, but failed to see that).

But I also don't understand the intention between using 3 "old/bad/cheap" Hardware boxes to spread the load versus buying one set of cluster/carp ready good hardware and run your show with it. And if your budget is that low while trying to run a REAL network in a DC? Then shout down the line to your boss because the one thing you do not want failing if your business depends on it is your network backbone. If it goes down - how much is it going to cost per hour? per day? per week? Does that justify being a cheapskate when it comes to the integral parts of the network/security?

Don't understand those bosses these days. Everyone needs internet, cloud, datacenter, yet they still think that things come magically from some wireless fairy-tale thingy somewhere that doesn't need money, nor power, nor configuration. Just sparkles and sunshine...

[Derelict]

@JeGr

[mrcha0s]

This is my experimental project. I have been using servers for more than 10 years but I always use vm, dedicated, cloud etc... 4 month ago I started to buy hardware and using colocation service. Last 2 months I'm researching firewalls and I see that fortigate, cisco asa, checkpoint, sophos are expensive for just packet filtering.
I dont want to use cisco,juniper, hpe products because I know they make discount %75-%90 if you are a whale. I will buy only two so I will get listing price.
Fortigate 100E ~3000 USD in Turkish market I need buy two for HA so its 6000 USD and in 5 years licenses will expire.
According to datasheet Fortigate 100E have 7.5Gbps firewall throughput. My aim is to get 7.5 Gbps firewall throughput with pfsense. Im using used Dell servers because they are cheap.
In my scenario I have two challenges;

1. Too many 1:1 NATs, its because architecture of my system. I will explain later with diagrams.
2. Using virtual machines for HA slaves.
   If cannot get expected results I will try Netgate hardware instead of buying fortigate, sophos, etc... Probably netgate hardware will cost me twice because of Turkish custom taxes and other delivery costs.
   In a 2 week I will share my performance tests without firewall, with pfsense, cyberroam, juniper ssg and fortigate 100D
   Thank you.

[Derelict]

While we would love for you to buy Netgate hardware it is not absolutely necessary.

Two of the Dell 220s with the same setup for NICs, RAM, etc should work fine as an HA cluster.

Not sure about getting the throughput you are looking for from them though. When you start trying to get more traffic through the software it really matters more how many packets per second need to be forwarded, not the overall throughput.

[mrcha0s]

@Derelict I'm not sure about netgate hardware but I want to try TNSR. I haven't review tnsr yet but if TNSR uses accelerator chip or different hardware than generic servers I can invest on it.

[Derelict]

The main performance difference between TNSR software and pfSense software is that TNSR software utilizes Vector Packet Processing and DPDK so there is much less overhead in forwarding packets through the device. It is more of a software difference than hardware acceleration. Both leverage hardware acceleration such as AES-NI.

https://www.youtube.com/watch?v=nAY1SV74S88

[stephenw10]

@mrcha0s said in What is best practice for my scenario:

Too many 1:1 NATs, its because architecture of my system. I will explain later with diagrams.

Yes, that's going to get awkward quickly. Even if they charge you more it would be worth it to have your ISP toute the whole subnet to you so you can use the IPs directly IMO.

Steve

[Derelict]

It is much easier to do it on subnet boundaries instead of human, decimal boundaries. such as a /26 of .64-.127 instead of .101-.150.

You can accomplish the former with a single 1:1 NAT entry. The latter would take several.

Routed subnets would be much better, but would put more reliance on upstream to get right.

[JeGr]

@Derelict said in What is best practice for my scenario:

@JeGr

Thanks for the laugh!

And yes, as @Derelict and @stephenw10 already stated, try getting them to route your subnet via a transit network. Sometimes one has to be very stubborn and persistent about it, but it pays off with any kind of box behind it to be far easier configuration-wise.