Returning to the Age Old ECC Question: How Important is it?



  • Hey all,

    I've been running pfSense on my ESXi box (with ECC) using a passed through Intel dual port (to avoid any potential virtualization security issues) for many years now and have been very happy with it.

    Recently, I decided that it was time to break out pfSense to a dedicated bare metal box, because I still wasn't convinced that passing through the Ethernet really eliminated my virtualization related security concerns, AND  because it is pretty annoying that the network goes down whenever I have to tinker with/maintain the virtualized server.

    So, I decided to go for overkill power (to do lots of QoS on on a 160/160Mbps connection), which - to me - meant a relatively high clocked, but low wattage Haswell dual core.  As soon as I started poking around I found an open box deal on a Socket 1150 Supermicro X10SLV-Q motherboard on Newegg, and decided to jump on it, because it was a great deal, and there was only one left.

    I have had lots of Supermicro boards over the years and been very happy with them, and unfortunately this lulled me into a false sense of security to the point where I didn't spend enough time peering over the specs.  Apparently this particular Supermicro motherboard is an odd one that doesn't support ECC  (OOPS).

    My philosophy on ECC has always been that it is a waste on client systems, but I put it in all my servers that run 24/7 for stability and protection from data corruptions sake.  The question is, is ECC really that important on a router?  Can I get away with using this board without ECC without sacrificing stability/reliability significantly?

    My gut tells me it is not a big deal for a dedicated pfSense box for the following reasons:

    • Packets are not retained in RAM on the router, they just pass through.  They probably only spend on the order of Nanoseconds in RAM, and that's not very much time for any given packet to experience a flipped bit, even without ECC.

    • TCP packets have error correction built in.  If - in the unlikely event - in this microscopic amount of time a bit actually flips, a new packet will just be requested and sent.  No big deal.

    • UDP is more vulnerable, but still, refer to the first point.  Packets really aren't kept in RAM.  It's orders of magnitude more likely that something else between my router and the outside source of the packet results in it being corrupted, than the lack of ECC ram in my router

    • In my 25 years of building systems, I have never once had RAM go bad on me without an attributable cause (like a failed power supply, lightning strike, etc.)  RAM is usually either bad when you first get it (which is why you test it with Memtest86+ or equivalent upon install) or it tends to work indefinitely

    So what do the rest of you think?  Is this a fatal mistake?  Should I turn around and return the board as soon as I get it (and eat the restocking fee), or is this just fine for router/firewall purposes.  What would you do?

    I'd love to hear your opinions.

    Thanks,
    Matt



  • Packets are mostly disposable. UDP doesn't hugely care about loss and TCP is meant to handle loss. The Firewall's config can be easily backed-up and replace.

    The main benefit you get from ECC on a firewall is potential better up time and better detection of memory issues that could drive you wild. If you were talking about a file server, then it's incredibly important.


  • Netgate

    What he said, but ram density is lower too.



  • So what do the rest of you think?

    I personally would try to get even ECC RAM since the RAM speed was increasing much in the past and it
    is so fast enough for a really speedy system. In normal you can go with any RAM type you want as I see it right.

    Is this a fatal mistake?

    No, never, but and this will be also true, if you need routing performance one or more (Multi WAN) at 1 GBit/s
    at the WAN interface, it is suggest to run server grade hardware and based on this advice, it is more common
    to go with ECC RAM based on that CPUs that will be used in servers. Not more, but also not less. On the pfSense
    website is written something over that, here is a link to read about it. Link

    501+ Mbps Multiple cores at > 2.0GHz are required. Server class hardware with PCI-e network adapters.

    And on top of this I would say it all depends on the offered services and installed packets you are using
    together with pfSense. ECC is not really needed but good to have and a firewall is a 24/7 running device
    and so it could be nice to have. Here´s a fine talk about that 1 GBit/s routing and the amount of RAM and
    its speed frequency, but nothing about ECC vs. non ECC.

    Should I turn around and return the board as soon as I get it (and eat the restocking fee), or is this just fine for router/firewall purposes.  What would you do?

    Why only for inserting ECC RAM?

    I'd love to hear your opinions.

    Be happy with this board.



  • To get very technical about ECC memory; what it is is ram with an additional chip that provides an error check process. This error check process applies directly to data being written on and off the hard drive. For pfsense, it will ensure that the system runs relatively error free. For packet handling, very little benefit will be seen unfortunately since packet routing is usually handled in your NIC cards themselves. Plus, all your Internet traffic will simply pass nic to nic without being written to disc. Your logs though, will be error free.



  • Appreciate your input guys.  Goes to confirm what I thought I already knew.

    Just wanted to make sure in order to verify that I wasn't overlooking anything.

    My main server uses registered ECC ram, but it serves as my NAS using ZFS.  I don't want to take any risks when it comes to my data.



  • My main server uses registered ECC ram, but it serves as my NAS using ZFS. I don't want to take any risks when it comes to my data.

    ZFS is working based on checksumming the data for a higher data integrity and there fore it should be a
    better thing here to go with ECC RAM because if a failure will be once written to that file system it can´t
    be corrected then with ease so ECC RAM should here be a so called "must be". And yes for sure in pfSense
    all data will be flowing through the RAM or plain hitting the memory system and if there will be a really huge
    VPN usage and pending then on the art and wise of the VPN connections and their method or key lease time
    it would be good to know if this lease time were shortened by the admin guy to increase the security and
    saving much more RAM then too, to know that ECC RAM will be inside of your firewall, for sure it will be.

    But for the entire rest of us it is more a cosmetic thing or owed to the circumstance that server hardware
    or server grade hardware will be in usage and there fore that hardware is mostly using ECC RAM too.



  • If anything, I would check into your NICs first since they touch every single bit obviously. In the intel world only the server ethernet chipsets / cards have ECC buffers and they tend to use higher quality components anyways.
    Regardless though all modern networks are designed around assumed loss and errors. I imagine the shit that happens on the miles and miles of copper/fiber/microwave/whatever is many orders of magnitude worse than the few centimeters of copper inside your firewall.

    ECC on the system ram will mostly just give you higher potential uptime between random flipped-bit crashes IMO, it can't hurt but no big deal.



  • @Aluminum:

    ECC on the system ram will mostly just give you higher potential uptime between random flipped-bit crashes IMO, it can't hurt but no big deal.

    Yeah, that is what I thought, but it doesn't hurt to confirm with others on occasion.

    For me uptime has never been an issue.  My uptime is always governed by the need to reboot for a hardware/software upgrade, one power outage in the last 10 years the times I moved, never based on system instability :p


Log in to reply