PfSense 2.2 SMP Performance



  • I suppose this is mostly directed at the developers, although I would welcome any comment from anyone who has done this sort of testing.

    I have a client that is about to purchase a campus for their business.  The buildings will be connected at 10gbps ethernet (singlemode fiber) at l2 (with room for up to a 40gbps lagg in the future, although I don't expect they will need it for a long time).  They will have a core chassis switch etc. (most likely a nexus 7xxx) in the NOC so I don't expect any issues with switching throughput.  I know that there is a 0% chance that pfSense will be able to do forwarding/packet inspection at 10gbps with the current single-thread pf implementation, but I have hope for when 2.2 stable comes out.

    I'm operating under the assumption that VLANs will add unnecessary overhead, so we will stack the gateway with as many intel/chelsio 10gbps nics as necessary.  Money is no object here, so if I need to stack a 4-socket server with 4 e7-4890 v2's with 512GB RAM, then that's what will happen (even that cost pales in comparison to what they are looking at purchasing from palo alto).

    I'm going to do jumbo frames where plausible, but for now lets assume that the worst-case MTU is 1500.

    They want full visibility of traffic between all of their RFC1918 subnets and a "single-pane" approach where plausible.  I think I can get them to do a separate inline squid3 box for web traffic inspection and MAYBE a separate snort box (although I really think IDS/IPS is overrated.. this is their requirement), but everything else e.g. VPN (site-to-site and road warrior, OpenVPN), policy based routing, mildly complicated outbound NAT rules, outbound filtering, etc. will be done at the gateway.

    Has anyone done 10gbps+ testing of SMP-friendly pf?  Does it scale well (up to 40gbps)?  Is there some other limitation that I'm not thinking of besides the CPU?

    Thank you in advance!



  • I don't have experience with such a high bandwidth configuration although I played with many modest gigabit configurations.
    But the point on which I would like to comment is about VLANs.

    Contrary to some belief out there VLANs hardly add any overhead to the packet processing capability of a switch or router. In fact from a router's perspective it's a lot better as some of the traffic (not intended to the router) never actually makes it to the router as the switch does the presorting of packets. So, in a VLAN environment the apparent router performance goes up a bit, and generally speaking VLANs are actually very good for the layer 2, layer 3 processing.
    In any event, if turning on VLAN functionality on your router suddenly makes your router look like it's overworking, it means that your hardware is underpowered or border line. But I bet it won't happen in your case given the hardware that you are contemplating.

    Halea



  • @haleakalas:

    I don't have experience with such a high bandwidth configuration although I played with many modest gigabit configurations.
    But the point on which I would like to comment is about VLANs.

    Contrary to some belief out there VLANs hardly add any overhead to the packet processing capability of a switch or router. In fact from a router's perspective it's a lot better as some of the traffic (not intended to the router) never actually makes it to the router as the switch does the presorting of packets. So, in a VLAN environment the apparent router performance goes up a bit, and generally speaking VLANs are actually very good for the layer 2, layer 3 processing.
    In any event, if turning on VLAN functionality on your router suddenly makes your router look like it's overworking, it means that your hardware is underpowered or border line. But I bet it won't happen in your case given the hardware that you are contemplating.

    Halea

    Hi Halea,

    You make a good point re: VLANs.  If 2.2 can handle the kind of pps I'm talking about, then the processing of the extra 32 bits (where applicable) probably won't make a difference in a negative manner.  If I can make sure that pfSense can handle this kind of traffic, I hopefully can do some really sweet performance testing and report back with pretty graphs.

    Thanks!


Log in to reply