Nice use of FreeBSD by NetFlix

  • Interesting followup to the original post, about Netflix's use of FreeBSD:

    Scott Long scottl at
    Wed Jun 6 00:16:55 UTC 2012

    Open Connect Appliance Software

    Netflix delivers streaming content using a combination of intelligent
    clients, a central control system, and a network of Open Connect appliances.

    When designing the Open Connect Appliance Software, we focused on these
    fundamental design goals:

    - Use of Open Source software
      - Ability to efficiently read from disk and write to network sockets
      - High-performance HTTP delivery
      - Ability to gather routing information via BGP

    Operating System

    For the operating system, we use FreeBSD <http:"">version
    9.0. This was selected for its balance of stability and features, a strong
    development community and staff expertise. We will contribute changes we
    make as part of our project to the community through the FreeBSD committers
    on our team.
    Web server

    Yes, we are indeed using FreeBSD at Netflix!  For those who are interested, I
    recently moved from Yahoo to Netflix to help support FreeBSD for them, and
    I'm definitely impressed with what is going on there.  Other than a few small
    changes, we're using stock FreeBSD 9, tracking the 9-stable branch on a
    regular basis.  Our chassis is a semi-custom 4U 19" form factor with thirty six
    3TB SATA disks and 2 SSDs.  Each disk has its own UFS+J filesystem, except for
    the SSDs that are mirrored together with gmirror.  The SSDs hold the OS image
    and cache some of the busiest content.  The other disks hold nothing but the
    audio and video files for our content streams.  We connect to the outside world
    via a twin-port Intel 10GBe optical NIC (only one port is active at the moment),
    and we use LSI MPT2 controllers for 32 of the 36 disks.  The other 4 disks
    connect to the onboard AHCI SATA controller.  All of the disks are
    direct-attach with no SAS backplanes or expanders.  Out-of-band management
    happens via IPMI on an on-board 1Gb NIC.  The entire system consumes
    around 500W of power, making it a very efficient appliance for its functionality.

    Netflix is also at the front of the internet pack with IPv6 roll-out, and FreeBSD
    plays an essential part of that.  We've been working hard on stabilizing the
    FreeBSD IPv6 stack for production-level traffic, and I recommend that all users
    of IPv6 update to the latest patches in 9-stable and 8-stable.
      Contact me
    directly if you have questions about this.  That said, we're excited about World
    IPv6 Day, and we're ready with AAAA DNS records and content service from both
    Amazon and the traditional CDNs as well as our OpenConnect network.

    From an advocacy standpoint, Netflix represents 30% of all North American
    internet traffic during peak hours, and FreeBSD is becoming an integral part
    of that metric as we shift traffic off of the traditional CDNs
    .  We're expanding
    quickly, which means that FreeBSD is once again a core part of the internet
    infrastructure.  As we find and fix stability and performance issues, we're
    aggressively pushing those changes into FreeBSD so that everyone can
    benefit from them, just as we benefit from the contributions of the rest of the
    FreeBSD ecosystem.  We're proud to be a part of the community, and look
    forward to a long-term relationship with FreeBSD.

    If you have any questions, let me know or follow the information links on the
    OpenConnect web site.