ESX3 - NICs not showing as VLAN capable



  • Hello,

    I am a n00b to pfSense so please bear with me.

    I've got a Dell 2950 running ESX 3.x and I'm attempting to run pfSense 1.2 RC3 on a VM but having some issues trying to set up VLANs.  The 2950 has 2 on-board Broadcom NetXtreme II BCM5708 NICs as will as a dual-port Intel 82571EB PCI-E card.  ESX is reporting that the Broadcom is using the bnx2 driver while the Intel is using the e1000 driver.

    I'm assuming that ESX is passing this driver information to the VM and that is what is being presented to FreeBSD.  According to the pfSense hardware documentation the Broadcom NIC is supported (using the bge(4) driver, cant find the bnx2 driver listed) and the Intel NIC, while not directly supported, using the e1000 driver "makes" it supported.  It appears that the e1000 is VLAN capable as is the Broadcom NIC.

    My issue is that, in the web config, when I go to Interfaces –> Assign --> VLANs, there is nothing available under Parent Interfaces.  I'm currently running directly off the LiveCD (after finding the issue after initial install).  I've tried setting both the Intel NIC and the Broadcom NIC as the LAN interface when creating the VM and both yield the same result (no parent interfaces listed).  I've also tried creating additional networking options from the ESX side and adding those NICs to the VM config, same result.  Even if that did work, it limits me to only 2 additional NICs due to ESX hardware limitations. (You can only have 5 PCI devices, 1 is the SCSI controller, 1 WAN NIC, 1 LAN NIC, then the 2 additional virtual ones)

    This is a home/personal project so I'd like to avoid having to buy another NIC and/or having to dedicate another box for the firewall, but I'm not sure I have a real choice.  I'm attaching my ifconfig for your viewing pleasure.

    Any thoughts?

    Thanks in advance,
    Rick



  • With ESX, you probably need to setup VLANs in ESX and assign the pfSense interfaces to the networks you configure for the VLANs.



  • Yea, I've tried that with no change.

    In ESX, I set up two "base" vSwitches, one tied to one physical NIC for the WAN, the other tied to a different physical NIC for the LAN.  Both of these are, in turn, assigned to the VM.  From there, I tried adding VLAN ports to the LAN vSwitch with no effect.  I also tried creating vSwitches not tied to a physical adapter then adding those adapters to the VM, also with no change in the behavior for pfSense (no parent interfaces available).  In all cases I set the VLAN port as the optional port and it did show up in the web interface.

    I tried both of these configurations on both the Intel and Broadcom NICs.

    Thanks,
    Rick



  • Finally got it to work!  Here is the solution for anyone else who runs into it…

    • In ESX --> Configuration --> Networking, set up your WAN and LAN NICs (no need to VLAN here, just create a vSwitch for each physical NIC)
    • Create the VM and attach these 2 NICs to it
    • Boot the VM and install pfSense to disk (never got VLANs to work from the LiveCD)
    • Configure the LAN IP from the pfSense console and verify that you can connect to the web interface
    • Halt pfSense and shut down the VM
    • You need to alter the VMX file for the VM (I use FastSCP)
            - You will need to create an account in ESX with shell access, root doesnt seem to work to connect in v3.
            - In FastSCP, make sure you check the box to elevate the account to root access
    • Create a backup of the VMX file
    • Edit the VMX and add these two lines (I put them in the "middle" section with all of the other ethernet options, not sure if it matters)
            - ethernet0.virtualDev="e1000"
            - ethernet1.virtualDev="e1000"
    • Save the file and boot the VM
    • pfSense will ask you to re-assign the LAN and WAN interfaces, once you do you will see LAN has maintained its assigned IP
            - You will notice here that the interfaces are now called em0 and em1 where they were previously le0 and le1
    • Now connect to the web interface and both network interfaces now show as parent interfaces in the VLAN configuration section

    I should note that I used the Intel 82571EB ports for both the WAN and LAN interfaces.  This was not tested against the Broadcom NIC.

    Kudos to JLanders in this VMWare Community Thread for the hint: http://communities.vmware.com/thread/57678

    Hope this helps someone else.  I would like my Saturday back now.



  • Although the above solution did provide the ablity to show the NICs as VLAN capable in pfSense, it still presented an issue with the number of VLANs that I could configure.  In order to send multiple VLANs to the ESX vSwitch I would have to create a port group for each VLAN and add these to the pfSense VM.  ESX does not allow more than 5 PCI devices to be mapped to a VM so I could not assign more than 3 VLANs to any one VM. (the SCSI controller, 1 WAN, then 3 VLANs for the VMs)

    ESX2 offered a solution involving editing some files manually to allow the passing of VLAN tags to the gueast called VGT mode.  In ESX3 they made this much easier by simply specifying the VLAN tag as 4095.

    On the ESX side…
      - Using the VI Client, select the server and go to Configuration --> Networking
      - Click Add Networking to create a switch mapped to the NIC you are going to use as the LAN NIC for pfSense (In my case I used one of the Intel NICs)
      - Go to the properties of the vSwitch and set the VLAN tag for the port group to 4095.  You will notice that the VLAN ID will say "All" when you apply the change.
      - Make sure the this port group is assigned to the pfSense VM and edit the VMX file to use the e1000 device (see above postings)

    From here you should be all set to add all of the VLANs you need from the pfSense interface.  You will, of course, need to configure the port you are using on your physical switch that goes to the pfSense LAN NIC (Dell 5012 in my case) as a tagged port.

    Two excellent ESX3 VLAN'ing documents...
      - http://www.vmware.com/pdf/esx3_vlan_wp.pdf
      - http://download3.vmware.com/vmworld/2006/tac9689-b.pdf

    Hope all of this helps someone else...

    Rick


Log in to reply