Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    ESXI - pfsense and FreeNAS

    Scheduled Pinned Locked Moved Virtualization
    24 Posts 9 Posters 8.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DerelictD Offline
      Derelict LAYER 8 Netgate
      last edited by

      I believe the main benefit to FreeNAS is ZFS RAIDZ2.  For that I like to give FreeNAS physical access to the disks.  This pretty much precludes me from running it virtualized.

      Chattanooga, Tennessee, USA
      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
      Do Not Chat For Help! NO_WAN_EGRESS(TM)

      1 Reply Last reply Reply Quote 0
      • johnpozJ Offline
        johnpoz LAYER 8 Global Moderator
        last edited by

        Says who?  Just raw map them, I give windows raw access to disks in my esxi box, its a simple command..  I just use stablebit drive pool, I see little use of raid of any sort in the home, or than maybe 0 there is little use of parity for anything storage items in the home.  But direct access to the disk is good for more than running raidz2, etc.

        "I have seen some people dont think its a good idea run both pfsense and freeNAS on the same hardware even if it is separate through ESXI"

        Who says this - nonsense.. I run all kinds of things as other vms on the same hardware my router pfsense is running on.. Not the freaking NSA secret headquarters or the dod nuke launch code storage facility ;)

        I get great performance to an from my storage vm to my network.. I like the raw map not only for the performance but that my vm os then has access to the smart info and can run scanner that keeps eye on the disk for me, etc.

        I would not install all kinds of services like that on my pfsense box, its suppose to be a firewall not running all kinds of other services - which is why doing it on say a esxi box is so great your other services you need like vm, plex box, whatever you just run as a different vm.

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 25.07.1 | Lab VMs 2.8, 25.07.1

        1 Reply Last reply Reply Quote 0
        • DerelictD Offline
          Derelict LAYER 8 Netgate
          last edited by

          @johnpoz:

          I see little use of raid of any sort in the home, or than maybe 0 there is little use of parity for anything storage items in the home.

          On this we'll just have to disagree I guess.

          Chattanooga, Tennessee, USA
          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
          Do Not Chat For Help! NO_WAN_EGRESS(TM)

          1 Reply Last reply Reply Quote 0
          • johnpozJ Offline
            johnpoz LAYER 8 Global Moderator
            last edited by

            What are you storing that requires parity?  I can understand backup for sure - this is given, I have multiple copies of home video and pictures.  Both local disks, and other media (multiple locations) and in the cloud.  But my video library for example in the TBs of data sure and the hell does not need to be available 24/7/365.. Its not mission critical if those files are available to the network, or even backed up.  Since have the original on optical, can always be reripped.  I have no concern for its replacement.  It might not even warrant replacement in the library since have really already seen the movie, etc.  Raid is not backup anyway, etc.  I don't see how it applies to the sort of files in a home?

            What sort of mission critical files do you have in the home that require the expenditure of cash to provide parity to mitigate down time on the loss of hardware?  Very curious! Is it a just a hobby and something you enjoy doing?  If so I understand that - but any dollar spend on providing parity could actually be spent on more storage if you ask me in the home.  Or actual backup??  Will your family disown you if the media library is offline if a disk crashes?

            If my whole esxi box burned up, to get back on network I could always fire up an old soho router laying around, or go to the store and get one, or fire up vm on my desktop to run pfsense on until such time esxi host could be rebuilt.  Worse case use hotspot on my phone, etc.  And that is if lost the whole thing, if disk crashed that contained VMs, I could reboot esxi via usb and use another disk for datastore, just move some files around or go to the store and pickup new ssd.  Not like anything would be down for any extended period.  So what justifies the cost of the spinning real time parity to restore files on loss of a disk?

            An intelligent man is sometimes forced to be drunk to spend time with his fools
            If you get confused: Listen to the Music Play
            Please don't Chat/PM me for help, unless mod related
            SG-4860 25.07.1 | Lab VMs 2.8, 25.07.1

            1 Reply Last reply Reply Quote 0
            • DerelictD Offline
              Derelict LAYER 8 Netgate
              last edited by

              Several terabytes of stuff I would rather not have to rebuild due to a simple disk failure.

              My home data storage policy is just different from yours.  You can stop trying to convince me I have built too much data protection into my home storage.

              Chattanooga, Tennessee, USA
              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
              Do Not Chat For Help! NO_WAN_EGRESS(TM)

              1 Reply Last reply Reply Quote 0
              • KOMK Offline
                KOM
                last edited by

                As someone who lost a lot of personal data due to a backup that validated just fine but then threw a data corruption error on restore after a disk went bad (thanks Norton Ghost!), I'll take RAID AND backup for $100, Alex.

                1 Reply Last reply Reply Quote 0
                • johnpozJ Offline
                  johnpoz LAYER 8 Global Moderator
                  last edited by

                  Agreed if you want parity, its your money and your files.. Go for it.. As to backup with Norton Ghost??  So an image, of the whole disk is not how I would backup "files"..

                  Maybe your files are different, but my online copy of all the xfiles episodes and star trek TOS while I like to have them at my fingertips don't justify cost of parity ;)

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 25.07.1 | Lab VMs 2.8, 25.07.1

                  1 Reply Last reply Reply Quote 0
                  • KOMK Offline
                    KOM
                    last edited by

                    So an image, of the whole disk is not how I would backup "files".

                    I prefer the flexibility of an image-based backup.  I can restore individual files with ease already, but a bare-metal recovery takes an hour, not several.

                    but my online copy of all the xfiles episodes and star trek TOS while I like to have them at my fingertips don't justify cost of parity

                    So where do you keep them then?  A stack of DVDs higher than your roof?  Hard disks are cheap.

                    1 Reply Last reply Reply Quote 0
                    • A Offline
                      attilahooper
                      last edited by

                      Too bad this thread devolved into a pissing match.

                      Parity and redundancy = good… When your data and time is important. And you want to be proactive.

                      Yoman, you still out there ? or did these tools scare you away ?

                      1 Reply Last reply Reply Quote 0
                      • M Offline
                        messerchmidt
                        last edited by

                        supoermicro atom c2758 with 16-32-64gb ddr3L ecc (depending on the freenas pool size)

                        1 Reply Last reply Reply Quote 0
                        • KOMK Offline
                          KOM
                          last edited by

                          or did these tools scare you away ?

                          KOM          Posts: 2595  Karma: +277/-10
                          johnpoz      Posts: 5473  Karma: +232/-40
                          Derelict    Posts: 3523  Karma: +390/-12
                          attilahooper Posts: 1    Karma: +0/-0

                          Hmmm…..  what was that about tools?  Get back to us when you've managed to actually help someone, ok?

                          1 Reply Last reply Reply Quote 0
                          • D Offline
                            doktornotor Banned
                            last edited by

                            @KOM:

                            attilahooper Posts: 1    Karma: +0/-0

                            That karma is already outdated…  ;D ;D ;D

                            1 Reply Last reply Reply Quote 0
                            • johnpozJ Offline
                              johnpoz LAYER 8 Global Moderator
                              last edited by

                              Pissing match?  It was a side discussion about the use of parity in home setting.  He has his views I have mine - sure and the F was not a pissing match.

                              An intelligent man is sometimes forced to be drunk to spend time with his fools
                              If you get confused: Listen to the Music Play
                              Please don't Chat/PM me for help, unless mod related
                              SG-4860 25.07.1 | Lab VMs 2.8, 25.07.1

                              1 Reply Last reply Reply Quote 0
                              • K Offline
                                Keljian
                                last edited by

                                Getting back on track: I run mdadm raid 6 at home using Ubuntu and pfsense on the same box using ESXi 6.

                                I have no drops in performance, and have sorted out pass through (vt-d/iommu) of a hard drive controller (m1015 in IT mode) to Ubuntu an one of the nics from an i350-T4 to the pfsense VM for the wan.

                                My hardware is as follows:
                                Asrock B85m Pro4 motherboard
                                14 gig of ram (using 7; 4 for Linux and caching, 2 for pfsense, extra is for work)
                                i5-4570t which I picked up from eBay cheap
                                M1015 hard drive controller (Lsi 9211-8t)
                                i350-t4 nic

                                Load is very low (1-10% usually). Measured load at idle with 7 disks spinning, 4 fans and an average power supply is 85w (45w with the disks spun down)

                                So yes it can be done

                                Key things to observe are:
                                1. Use a separate interface for the management network if possible
                                2. Passthrough your wan port directly to the pfsense if possible to prevent the hyper visor touching it for security reasons (see later posts in this thread)
                                3. Use the virtio package in pfsense and the core-VM package in Linux (I believe the same exists for bsd, hence the virtio package).

                                1 Reply Last reply Reply Quote 0
                                • F Offline
                                  frasse
                                  last edited by

                                  @Keljian:

                                  2. Passthrough your wan port directly to the pfsense if possible to prevent the hyper visor touching it for security reasons

                                  Is this really recommended?

                                  1 Reply Last reply Reply Quote 0
                                  • K Offline
                                    Keljian
                                    last edited by

                                    Don't know if it's recommended to others, but it makes perfect sense to me.

                                    Means any potential security issues with ESXi vswitches won't affect the other stuff on the box.

                                    1 Reply Last reply Reply Quote 0
                                    • johnpozJ Offline
                                      johnpoz LAYER 8 Global Moderator
                                      last edited by

                                      Security issues with vswitches?  On your wan?

                                      No I have never seen that recommended anywhere.  There is no issue with using a vswitch to connect your wan to pfsense.  Expect for those with really really tight tinfoil hats maybe ;)

                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                      If you get confused: Listen to the Music Play
                                      Please don't Chat/PM me for help, unless mod related
                                      SG-4860 25.07.1 | Lab VMs 2.8, 25.07.1

                                      1 Reply Last reply Reply Quote 0
                                      • K Offline
                                        Keljian
                                        last edited by

                                        straightens tinfoil hat

                                        Still if you can do it, why not? Are you intending to use that physical port for any other purpose at the same time?

                                        If you have pass through available, I don't see any logical reason not to, aside from a fringe case where pfsense doesn't have drivers for your nic where your hypervisor does.

                                        1 Reply Last reply Reply Quote 0
                                        • johnpozJ Offline
                                          johnpoz LAYER 8 Global Moderator
                                          last edited by

                                          Why not because it makes the setup more complicated - so why do it..  It doesn't buy you anything other than more complication if you ask me.. It sure not buying you any added "security"

                                          KISS

                                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                                          If you get confused: Listen to the Music Play
                                          Please don't Chat/PM me for help, unless mod related
                                          SG-4860 25.07.1 | Lab VMs 2.8, 25.07.1

                                          1 Reply Last reply Reply Quote 0
                                          • K Offline
                                            Keljian
                                            last edited by

                                            It is less trouble setting up direct path than it is a new vswitch, but hey, just go with what works for you

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.