Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Strange bug with more than 03 VMXNET3 adapters

    Scheduled Pinned Locked Moved
    2.2 Snapshot Feedback and Problems - RETIRED
    3
    6
    1.8k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      abcslayer
      last edited by

      Hi, I've met a strange bug:
      My setup:

      pfSense with 01 WAN, 02 LANs (total 03 VMXNET3 adapters) each connect to different virtual switches inside ESXi 5.5, only the switch with WAN interface has a physical adapter connect to (my purpose is using pfSense to isolate all VMs in two different virtual LAN switches). WAN is DHCP with reserved IP address on physical router.

      • Today I decide to add 02 more VMXNET3 adapters, each connect to a different virtual switch, so total I got 05 VMXNET3 adapters connect to 05 different virtual switches. After reboot the WAN interface nolonger gets IP address from physical router. All other LAN interfaces were failed too (I set static IP but VM can not ping to pfSense any more, DHCP from pfSense to VMs also failed). I tried to set static IP to the WAN it is also failed (the WAN IP display blank space in the console). The 02 new adapters were not setup/assigned, just added to pfSense VM.

      • If I remove the 02 extra adapters, the WAN interface work, everything comes back to normal state.

      • If I re-add 02 extra adapters -> it failed again.

      • If I add 02 extra E1000 adapter -> WAN still works, thing seems to be OK.

      I use the latest snapshot of pfSense. This is a quite strange bug.

      2.2-BETA (amd64)
      built on Sun Oct 12 11:15:43 CDT 2014
      FreeBSD 10.1-RC2

      1 Reply Last reply Reply Quote 0
      • P
        phil.davis
        last edited by

        It sounds like the WAN, LAN1 and LAN2 are not being connected to the same virtual devices. Now that you have 5 devices, maybe the virtual device that WAN is associated with is no longer the one that goes out to the real world.
        What makes me say this is that when you add a different virtual device, the settings are not mixed up. There are still 3 VMXNET3 virtual devices, so pfSense finds them the way it always did, and then the E1000 adapters happily sit there unassigned.
        Maybe just play with changing the interface device assignments until you find the ones that go where they should?

        As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
        If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

        1 Reply Last reply Reply Quote 0
        • A
          abcslayer
          last edited by

          I have checked again today by adding just 01 more VMXNET3 adapter to the yesterday instance (pfSense has 03 VMXNET3 + 02 E1000). The symptom happened again, WAN stopped working.

          The WAN is still pointing to the correct adapter (vmx0), the adapter has correct MAC address so there is no chance that ESXi change the order of the adapters.

          I has no clue about what happen with this system now.

          –--------
          P/S: other interfaces seem to be not working too. It can only ping the local static address of itself (e.g: 192.168.1.1, 192.168.2.1), can not ping any other static addresses on its LANs.

          1 Reply Last reply Reply Quote 0
          • A
            abcslayer
            last edited by

            I am using the latest snapshot, today I need to add more interfaces to pfSense and forgot about the VMXNET03 bug. It takes me sometime to reboot other routers & pfSense :'( before I remember about this trouble.

            2.2-BETA (amd64)
            built on Fri Oct 31 04:59:06 CDT 2014
            FreeBSD 10.1-RC4

            Remove the extra VMXNET03 and add E1000, the system with 03 VMXNET03 and 03 E1000 is running now.  :-X

            1 Reply Last reply Reply Quote 0
            • C
              cmb
              last edited by

              I have at least one 2.2 VM in ESXi 5.5 with 4 vmxnet3 NICs and they all work fine. Anything atypical you're doing networking-wise at the host level? What does packet capture show on the affected interface when it's not working?

              1 Reply Last reply Reply Quote 0
              • A
                abcslayer
                last edited by

                hi, there is nothing special about my config:

                • I put pfSense in a VM, there are three active adapters in which each adapter of pfSense is connected to a vswitch. Currently there are one WAN (its vswitch has a physical adapter connected to) and two LAN.

                • I plan to add two PPPoE connections to my pfSense to replace current physical router so I add two more adapters and config it to use PPPoE. The issue happened right when I add two more VMXNET3 and reboot pfSense. Then I change them to E1000 -> pfSense acted normally. I did report to this forum at that point.

                • Last weekend, I decided to add one more adapter for another LAN. Added VMXNET3 -> failed. Changed to E1000 -> OK.

                The VM is nanobsd and is installed with 2.2 snapshot about 1 month ago.
                I did not capture the traffic at the failed moment. I will try to do it again when I have chance (the pfSense currently handle NAT for some VMs now) I think basic features of pf is stable enough for using.

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post