Issue adding interfaces under ESXi

MikeKulls

I've been running pfSense under ESXi for about 4 years now with great success. Short description of the problem is that I add a third LAN interface to pfSense via the ESXi gui, reboot pfSense and the WAN stops working. It refuses to get an IP from the modem. The original 2 LANs are working fine. The interesting thing is this happens before I have assigned the new interface in pfSense, so the new interface is unused. Should that be a problem? I remove the new interface in ESXi, reboot pfSense and everything starts working.

Thanks

johnpoz

where did you add the new interface? Sounds like pfsense picked up this new interface as your wan that was not correct.

FauxShow

I've had issues where the order of the interfaces was wrong after I added a few VMXNet3 adapters. It looked like nothing was working correctly at first, but it just didn't map the interfaces through in the correct order.

KOM

I've never had any such problems with pfSense & ESXi, but I always add & configure in this order: WAN, LAN, OPT1, OPT2…

johnpoz

I normally setup specific mac for my interfaces in esxi so I know for SURE which one is suppose to be which and not have to worry about order they come up, etc..

I could prob get more creative in the numbers ;) But this makes it real simple, so as long as someone else on my isp and on my same network gets the bright idea of using the same exact mac I won't have any issues with that 00:00:01 ;) But then again im on 24/7/365 and its working.. So they are the ones that would have the problem if they tried to use that same mac.

em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:01

em1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:02

em2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:03

em3: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:00:00:04</rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast>

KOM

I normally setup specific mac for my interfaces in esxi so I know for SURE which one is suppose to be which

em0 = :01
em1 = :02
em2 = :03

Clear as mud… ;D

Reminds me of when I was supporting optical jukebox software. The developers in their wisdom decided to index the jukebox shelves starting at 1, but index the drives starting at 0. So when you read an event log message saying that drive 2 had a problem with the media in shelf 34, you had to try and remember, was that really drive 2 shelf 34, or drive 2 shelf 35, or drive 3 shelf 34, or drive 3 shelf 35...

johnpoz

Well you can not use 0 for a mac, not that I am aware of? Maybe you could ;)

Yeah interfaces always start at 0.. But its simple enough for me to look in esxi on which interface is on which vswitch, and then just easy numbers 1,2,3,4 to look at.

KOM

Well you can not use 0 for a mac, not that I am aware of?

Do I have to think of everything? Obviously, the solution is to not have an em0…

MikeKulls

Can't you use 00:00:00 for the last 6 digits?

johnpoz

Maybe you can - will give it a try when get a chance.. That would make numbers match up.. How do not have a 0 interface though? I don't think that is possible either. Other than having one and not using it ;)

But could also just change to A,B,C and D for the last of the mac address - just need something that is very easy to see which one is which so can make sure pick the right one when setting up pfsense..

KOM

How do not have a 0 interface though? I don't think that is possible either.

It was a joke.

l4k3k3m4n

I have the same problem, when I add a new interface and reboot the order is mixed and I have to manually remap everything.
Is there a fix to this problem?

Another question:

Is it better to use multiple untagged interfaces in pfsense and do the VLAN separation on ESX vswitch ?
Or just use one trunked interface on ESX vswitch and create VLAN interfaces in pfsense?

Thanks.

heper

personally i just trunk it straight in the VM and play with vlans inside the VM.
no clue if there's any performance difference in doing the tagging/untagging on the esx level.

i find it easier to manage, by doing vlans in pfsense. This also solves the adding/removing interfaces "regulary'. (i don't add/remove physical/virtual interfaces a lot. i do regulary need more or less vlans)

sebyp

Hi there gents,

I'm experiencing the same issue here, after I'm adding a new "physical" (from pfSense's point of view) interface things go kaboom!

I've been thinking about using VLAN's as well, but my only question is: will it work on multiple hosts?

I have a pretty straight setup with two hosts, connected via switches. Of course, for inter-host communication I'm mapping the virtual networks to VLANS.

Is there a possibility to add "virtual dot1q tags" inside the actual virtual networks?

Thanks,
Sebastian