Hardware Check



  • Hello,

    I´am thinking of building an pfsense router as replacement for our old office router. The Hardware should be able to handle the traffic between about 4-5 gbit networks (only firewall/Routing) and 2x 100Mbit internet connections (Firewall and maybe Snort).

    We also want to use 2 OVPN connections with the maximum of 100mb each and be able to have Road warriors to log in.

    I know that that will require a server class motherboard and some good intel nics but I am very unsure about the cpu and the ram requirements and I was not able to find some “real” numbers.

    I was thinking of using the following processor: INTEL Xeon E3-1275 v3. Do you think that processor is suitable for that usage or overkill?

    As for RAM I was thinking about using 8GB DDR3-1600 ECCM CL11.

    I would be very thankful for some information’s.


  • Netgate Administrator

    Probably overkill. What total bandwidth do you require between your internal networks?

    200Mbps OpenVPN with Snort running I'm unsure about.

    Steve



  • RAM is fine. For just 100-200Mbps internet a Xeon is an overkill.

    For 200Mbps on OpenVPN with Snort may be the work for a Xeon.. yes. Depends on your encryption settings. Use 128-bit first and see how it works out. If you feel you need 256-bit (which is just an overkill) then test it out and see if you are satisfied with it. I recommend going with 128-bit as it take a lot of CPU cycles (depending on VPN users) for encrypting/encrypting data in OpenVPN. More the users, lower will be the throughput. For 5-10 users it should be fine.



  • Thank you for your answers.

    @stephenw10

    We will have 2-3 internal networks that will only have a very moderate traffic (Telephone and service network to Workstation Network for example). But we will also have 2 Networks that will probably have heavier load between them (Gbit level).

    @asterix

    Are the 5-10 Users the total number of users behind the pfsense or the number of Roadwarriror users caming over vpn?


  • Netgate Administrator

    Ok, so you need 1000Mbps between two internal networks but not between several internal networks at the same time? If you did need a total of 2 or even 3 Gbps that would ramp up the requirements considerably.

    The VPN question is interesting, I don't have much experience of multiple clients. I guess there is some overhead but do 10 remote client at 10Mbps really need that much more CPU that 1 at 100Mbps? Asterix?

    Steve



  • I'd expect 10 separate connects to work better than 1 big fast connection.  (I've had more  - Never noticed a problem)



  • @stephenw10:

    Ok, so you need 1000Mbps between two internal networks but not between several internal networks at the same time? If you did need a total of 2 or even 3 Gbps that would ramp up the requirements considerably.

    The VPN question is interesting, I don't have much experience of multiple clients. I guess there is some overhead but do 10 remote client at 10Mbps really need that much more CPU that 1 at 100Mbps? Asterix?

    Steve

    Well, I have configured close to 50 VPN users (doesn't matter if they are road warriors.. VPN users are just VPN users :) ) and CPU takes a toll depending on the encryption settings. 10 VPN users for even an i3 is a walk in the park for it (well kinda.. lol) .. but I am skeptical on 200Mbps VPN bandwidth requirements. That would require the CPU to really work its way to process that kind of throughput.

    If you have the Xeon server on hand then I would recommend you do a test to ensure it works up to your requirements. I believe it should be fine. That processor can support 500+ network users/devices easily behind pfSense… take a note here.. I mentioned behind pfSense. VPN users require additional CPU processing so generally categorized as "VPN users" and not "network users" as they come in from the WAN side. Well there are exceptions for them as well and it can get complicated where there could be internal network VPN users between sites connected by separate subnets of the same larger network. For that scenario you would be better off having multiple NICs supporting internal networks and configure VPN on those NICs and not the WAN. Makes sense ????

    I am a visual person.. If you could create a high level visual diagram of your network requirements it would be of great help for us to recommend the best config.


  • Netgate Administrator

    But is the processing power required increased significantly by having multiple connections, 10 X 20Mbps. rather than say 2 x 100Mbps? It's not something I've looked into.

    Steve



  • @stephenw10:

    But is the processing power required increased significantly by having multiple connections, 10 X 20Mbps. rather than say 2 x 100Mbps? It's not something I've looked into.

    Steve

    Steve, I edited/added more info on my post above not realizing you were responding as well.. You may wanna read my post again.

    On your question …

    Yes  :)

    More the users more cycles are needed to separate and encrypt/decrypt data for each user and that affects throughput. This is purely based on the fact that VPN is configured on the WAN side.

    Internal site to site VPN and VPN user connections would be way faster as they are on dedicated direct lines like almost being on the same switch.



  • I have made a simple drawing where you can see the networks and there bandwidth and attached it.




  • @Yada:

    I was thinking of using the following processor: INTEL Xeon E3-1275 v3. Do you think that processor is suitable for that usage or overkill?

    As for RAM I was thinking about using 8GB DDR3-1600 ECCM CL11.

    I would be very thankful for some information’s.

    Haswell i3 supports ECC and AES-NI at a much lower price, and will perform the same on single threaded things like pf. (same 3.X Ghz clockspeeds) Just make sure your motherboard has a bios update that supports them.

    If you truly need quad cores with eight threads, get the 1245 as the extra 0.1 Ghz on the 1275 adds nearly $100 to the price. IMO you do not need a quad.
    Another way to save money: if your motherboard has a built in vga controller (very common) buy a cheaper 12_0 model. Most OEMs sell the boards that let you use the intel gpu as 'workstation' instead of 'server'.



  • OK.. so anything behind your pfSense (internal network) connected via network switches has nothing to do with speeds. The internal network runs off your internal network switches. You can have 10Mbps or 100Mbps or 1000Mbps/Gibabit network and it won't tax your pfSense CPU as the data is ignored and only WAN data is processed. An i3 processor will fly for 100Mbps WAN .. or 2 x 100 Mbps

    If I am not mistaken, you want Snort but you do not say the number of users behind pfSense. I would recommend i5/i7 or higher like Xeon if you have a large user base. i3 will still be fine for 10-40 users. for 50+ I would go for an i5. For 100+ i7 and 200+ definitely a Xeon. This config is just for Snort as it will hog CPU cycles for processing all the connections coming in and going out the WAN. Plus if you add snort for LAN then it will strain the CPU even further.

    Your road warriors/vpn users … how many are those? Keep in mind that you can configure VPN on only 1 WAN at a time(unless I am missing something and CARP might be able to combine them... which I think it can't).. so you would need 2 different VPN connections to take advantage of 2 WANs. You need to load balance VPN users for both WAN connections... by dividing the users between the WAN... VPN User 1 connecting to WAN1 will not be able to connect to WAN2 and vice versa.



  • @asterix:

    OK.. so anything behind your pfSense (internal network) connected via network switches has nothing to do with speeds. The internal network runs off your internal network switches. You can have 10Mbps or 100Mbps or 1000Mbps/Gibabit network and it won't tax your pfSense CPU as the data is ignored and only WAN data is processed.

    What if you wanted the internal networks to be isolated with limited connectivity and utilize the pfsense firewall to do that? Wouldn't that still get processed by the CPU? Admittedly that would not be as taxing as WAN side processing involving NAT/Snort/VPN/etc. but it still needs to be considered, right? Or is it insignificant enough to be "lumped in" with the rest of the load?



  • @JoelC707:

    @asterix:

    OK.. so anything behind your pfSense (internal network) connected via network switches has nothing to do with speeds. The internal network runs off your internal network switches. You can have 10Mbps or 100Mbps or 1000Mbps/Gibabit network and it won't tax your pfSense CPU as the data is ignored and only WAN data is processed.

    What if you wanted the internal networks to be isolated with limited connectivity and utilize the pfsense firewall to do that? Wouldn't that still get processed by the CPU? Admittedly that would not be as taxing as WAN side processing involving NAT/Snort/VPN/etc. but it still needs to be considered, right? Or is it insignificant enough to be "lumped in" with the rest of the load?

    Yup.. as the internal network switches handle all the internal routing.



  • @asterix:

    Yup.. as the internal network switches handle all the internal routing.

    They won't if, as he says, he actually wants them routed/firewalled (e.g., each color in his picture on a different VLAN)?!



  • @asterix:

    If I am not mistaken, you want Snort but you do not say the number of users behind pfSense. I would recommend i5/i7 or higher like Xeon if you have a large user base. i3 will still be fine for 10-40 users. for 50+ I would go for an i5. For 100+ i7 and 200+ definitely a Xeon. This config is just for Snort as it will hog CPU cycles for processing all the connections coming in and going out the WAN. Plus if you add snort for LAN then it will strain the CPU even further.

    Correct me if I am wrong, but isn't snort still largely single threaded? I know thats one reason I am looking at bro a lot harder lately. (the main reason I think snort will soon be utter crap for regular users sounds a lot like cooking oil)

    If it is, the difference between i3/5/7/xeon is almost immaterial and you probably want the most recent core with the highest clockspeed. Going from nehalem –> sandy bridge -> ivy bridge -> haswell is roughly 5~10% gain per step given same clock.
    (not going to nitpick cache sizes or memory channels, but snort doesn't seem to care much compared to say, video encoding)

    From this perspective a nice cheap desktop haswell core will beat a dual socket sandy E5 xeon that costs 10 times as much.
    (can't find the cheap single socket ivy xeons yet, though if you have 2k+ to burn enjoy 8-12 cores per socket)

    Also, i5/i7 perform identical to same generation non-HT/HT E3 xeons respectively, just no ECC and maybe VT-d.


  • Netgate Administrator

    @JoelC707:

    What if you wanted the internal networks to be isolated with limited connectivity and utilize the pfsense firewall to do that? Wouldn't that still get processed by the CPU? Admittedly that would not be as taxing as WAN side processing involving NAT/Snort/VPN/etc. but it still needs to be considered, right? Or is it insignificant enough to be "lumped in" with the rest of the load?

    If you have multiple 'internal' interfaces segregating your network then that traffic is indeed processed and uses almost as much CPU as WAN-LAN traffic (assuming no NAT). It's not at all insignificant. That's why I asked about it.

    The diagram doesn't show any switches so it's hard to say quite what is intended.

    Steve



  • My internal network activity is many times close to terabytes. Especially during weekend backups. Have 4 separate interfaces out of which 3 are heavily used. I barely see any CPU spikes. I do see my managed switch getting hammered.



  • Is your switch layer 2 or 3? If it's layer 3 then it's handling all the routing most likely. If it's layer 2 then pfsense has to handle all the routing. I guess that's really the question the OP needs to answer, is their switch layer 3 (or do they have a separate router to handle the layer 3 duties)?



  • Layer 2



  • Terabytes per what, though?

    So you're saying you're generating gigabits of routed throughput (i.e., between subnets – or are the interfaces just bridged?), and your pfSense box is near idle while your L2 switch is busy? That just seems… wrong. Even cheap switches should be able to forward at line rate without breaking a sweat.


  • Netgate Administrator

    I agree, traffic between any two interfaces is filtered by pf. Traffic not involving WAN probably isn't NATed and probably not subject to Snort etc. The only way this isn't true is if you've disabled the firewall. Even bridged interfaces are filtered.

    Steve



  • I think you could avoid filtering traffic that is just forwarded between bridge members by setting net.link.bridge.pfil_member=0 and net.link.bridge.pfil_bridge=1, but yeah, pfSense seems to be set up the other way around by default.


  • Netgate Administrator

    Yes, the default is a filtering bridge. However I believe I read that even with bridge member filtering disabled there is still some processing takes place. Can't find that now of course.  ::)

    Steve



  • @razzfazz:

    Terabytes per what, though?

    So you're saying you're generating gigabits of routed throughput (i.e., between subnets – or are the interfaces just bridged?), and your pfSense box is near idle while your L2 switch is busy? That just seems… wrong. Even cheap switches should be able to forward at line rate without breaking a sweat.

    I have a Netgear GSM7248v2 48-port switch. Typical data transfers are between 20 -28 MB/sec across the subnets and each subnet is on it's own NIC. I wouldn't say the pfSense CPU is near idle.. but it's barely even noticeable. It has to be doing some processing but I have 2 physical Xeon CPUs and I suppose its a walk in the park for them.. ;)



  • Oh, OK, I thought we were talking about pushing throughput close to wire speed; less than 30MB/s isn't exactly what I'd consider "hammering" a gigabit switch.



  • @stephenw10:

    @JoelC707:

    What if you wanted the internal networks to be isolated with limited connectivity and utilize the pfsense firewall to do that? Wouldn't that still get processed by the CPU? Admittedly that would not be as taxing as WAN side processing involving NAT/Snort/VPN/etc. but it still needs to be considered, right? Or is it insignificant enough to be "lumped in" with the rest of the load?

    If you have multiple 'internal' interfaces segregating your network then that traffic is indeed processed and uses almost as much CPU as WAN-LAN traffic (assuming no NAT). It's not at all insignificant. That's why I asked about it.

    The diagram doesn't show any switches so it's hard to say quite what is intended.

    Steve

    Tanks for your answer. The networks will be splited in different subnets to different nic port on the pfsense.



  • @razzfazz:

    Oh, OK, I thought we were talking about pushing throughput close to wire speed; less than 30MB/s isn't exactly what I'd consider "hammering" a gigabit switch.

    Well 30MB/sec on each subnet. I have 2 NAS and they run backups at nights.. and they coincide with data transfers at some point. So the switch is working on 3 subnets at a time.. sometimes with smaller files it gets to 50-60MB/sec on one subnet alone. So techincally its handling lets say 30 + 50 + 60 MB/sec simultaneously.



  • Mine is rocking at about 4Mbps all day and night…

    Reliably too  ;D


Log in to reply