Is E1000E better supported than VMXNET3 in pfSense?

[senseivita]

When I upgraded to 2.5 I was basically forced to stop using pfSense because it added up with some other issues, I moved back temporarily to an old Ubiquiti Networks UniFi firewall.

Recently I had been noticing errors in NICs on pfSense even before 2.5 when using VMXNET3 on vSphere. To work around them I'd just pass through PCI NICs to pfSense.

After fixing a few things for which I needed a stable network I decided to double-NAT, static route or maybe do a transparent firewall to aid UniFi, I would figure it out later. I picked an unused OPNsense VM I already had. OPNsense recommends using E1000E NICs over VMXNET3, I though about changing them but them whole MAC reassignment (I keep control over it) as well as the curiosity convinced me me not to. Even though reassigning interfaces later in GUI is a PITA and I would have to pay for it.

Installed pfSense, restored a backup, fixed interfaces, and for the first time in long, long time pfSense did reboot and reinstalled packages and went back to business without further fixing as it's supposed to.

There are no NIC errors, there are no passthrough PCI NICs, there are no tunnels yet but there are I think heavy workloads: Suricata in at least 5 interfaces, pfBlockerNG, DNS Resolver performing DoH (DNS upstream for domain controllers), catch-all frontends on HAProxy, Squid (not transparent), Avahi, to test my theory I installed ntopng and still good. I went back to pfSense after this, I'm grateful for that other firewall as a fallback but not as my only option. I plugged it off this time letting it sit there in the rack hopefully just wasting space in my very constrained 24U -ish rack.

I also noticed that when doing interface changes, or rules changes, the GUI hangs a whole lot less than before. It's like a dream firewall. It's using only one E1000E interface plus another of the same type for the WAN side, the latter is VLANed to the ONT, not that it needs to because there's no IP in it, it uses PPPoE, it doesn't conflict in any way.

As you can see above, the VM hardware is very unremarkable, the only difference from my previous pfSense VMs is the NIC type.

Some notable (I think) settings:

I remember reading something specific to VMXNET3 in the pfSense docs in a positive way but I can't find it anymore.

[Rico]

@skilledinept said in Is E1000E better supported than VMXNET3 in pfSense?:

I remember reading something specific to VMXNET3 in the pfSense docs in a positive way but I can't find it anymore.

For best performance, we recommend using VMXNET 3 type of adapters instead of E1000. ?
https://docs.netgate.com/pfsense/en/latest/virtualization/virtualizing-pfsense-with-vmware-vsphere-esxi.html

-Rico

[senseivita]

Thanks! I knew I wasn't imagining things. I don't get the performance why impact though.

Maybe it's the physical NICs that are not good with the paravirtualized NIC--but--other machines would have bad performance too, right?

I'll just be grateful it works for now. Thanks to you too! :)

[pfsensation]

@skilledinept said in Is E1000E better supported than VMXNET3 in pfSense?:

Thanks! I knew I wasn't imagining things. I don't get the performance why impact though.

Maybe it's the physical NICs that are not good with the paravirtualized NIC--but--other machines would have bad performance too, right?

I'll just be grateful it works for now. Thanks to you too! :)

I was literally going to make a forum post and thought I'd do some research and found your thread!

I've been having the same issues with VMXNET3 after introducing VLANs into my home network, I would see drops from time to time on pfSense. I am using a supported Intel NIC and have not previously had any issues for over a year. I have checked everything I could in terms of speed/duplex mismatches, bad cables, NIC, switches but had no joy. I was only able to fix this using E1000. I did see that it was using a bit more CPU usage and typically the speeds not being as consistent, therefore I'd much rather use VMXNET3 if possible. E1000 fixes the packet drops. This seems to be something specific to pfSense and ESXi - I don't have any problems with any of my other VM's which run on VMXNET3, all works perfectly.

I have 3 VMXNET adapters connected to my pfSense VM - WAN/WAN2/LAN. The VLANs are running over the LAN adapter, those appear to be the ones having issues.

@skilledinept Are you using VLANs? I think possibly this could be isolated to using VLANs with VMNET3. I have no drops on my WAN interfaces where I have no VLANs.

NOTE: No drops are observed switch side, vswitch security settings are all set to allow. Running pfSense 2.4.5-RELEASE (amd64)