Can't DHCP from Cable modem
-
if you're MAC spoofing in VMware, you must enable promiscuous mode. That sounds more like the issue described most recently where the DHCP responses are on the physical wire, but aren't seen in the VM, which means they're getting dropped within ESX. ESX will drop traffic destined to anything other than VM's assigned virtual MAC address if the vswitch isn't in promiscuous mode.
-
It would be interesting to see what ESXi thinks is the MAC address of that interface.
If you're comfortable with using SSH into your ESXi host and running a script you could try this:
http://www.virtuallyghetto.com/2011/05/how-to-query-for-macs-on-internal.html
I haven't tried it but I'm pretty sure tcpdump is also included ESXi.
-
OK, thanks to stephenw10 I have worked around the problem. Something is still going wrong but at least I have IP's on all my WAN interfaces for the first time!
What I did was left the Cisco 2960 interface (that all WAN's attach) trunked but set the native VLAN to 900. Then set WAN1 to the parent pfSense interface (non VLAN) and it instantly picked up a public IP address from the cable system.
So basically I'm sending WAN1 packets untagged, while WAN2 and WAN3 packets are still tagged. WAN2 and WAN3 seem to still work fine but have not thoroughly tested that yet.
Will do more testing tomorrow. I'm going to try spoofing the MAC on my other DHCP WAN interface and see if it quits working.
-
Interesting. The setup you have now is exactly what I would be trying to avoid!
Sending VLAN tagged and non-tagged traffic to the same pfSense interface can cause problems. However it doesn't always by any means. If it's working for you…
An interesting result though, as though the tagging/untagging process is causing packets not to get through.
I know pretty much nothing about vmware outside the concepts. I was suggesting it may be possible to create a virtual switch that accepts the trunk from the Cisco switch and untags the VLAN traffic to three interfaces internally which then feed pfSense. That way you remove the VLAN handling from pfSense.
Steve
-
I know pretty much nothing about vmware outside the concepts. I was suggesting it may be possible to create a virtual switch that accepts the trunk from the Cisco switch and untags the VLAN traffic to three interfaces internally which then feed pfSense. That way you remove the VLAN handling from pfSense.
Steve
Steve, this can easily be done in VMware. I "thought" it would have been easier to just trunk everything on a single interface but has turned out not the case. From what I have seen so far separate interfaces and no tagging seems the best solution. At work I have over 40 virtual servers (mostly Microsoft :-[) and have never had any problem like this with VMware before.
-
There is an 'issue' with FreeBSD and VLAN handling that can cause a problem when you have tagged and untagged traffic on the same interface. I am not familiar with it and I have never run into it myself but I know it exists. It shouldn't be a problem for you since originally you only had tagged traffic on the interface. In another thread I was trying to diagnose a problem and another user found that his switch was sending untagged packets to his pfSense box accidentally. To eliminate that as a possible cause of trouble I was suggesting you move VLAN handling away from pfSense.
However since you are now definitely sending untagged traffic, and it's working, I'm not sure what to suggest! :-\Steve
Edit: Out of interest I looked further into this and can find nothing so now I'm unsure where this advice came from or what it applies to. Certainly general advice within pfSense is to avoid tagged and untagged traffic on the same interface but whether that is due to FreeBSD or VLANs in general I don't know.
-
OK, problem is something to do with MAC spoofing. I spoofed the MAC on WAN3 and it stopped working. Even after clearing the MAC spoofing box it still would not work until a system reboot which got it working again non-spoofed.
One thing I did notice that was odd, while both WAN1 and WAN2 were spoofed (and not working) ifconfig in pfSense down at the bottom showed the VLAN'ed interfaces with their spoofed MAC's, but oddly WAN2 (which I have never spoofed) showed the same MAC as the spoofed WAN1 MAC. WAN3 showed it's last spoofed MAC even though at this time the spoofing box was cleared on the interface web page. After the reboot everything looked as expected, WAN1 showed it's spoofed MAC, while WAN2 and WAN3 showed the VMware interface MAC.
The problems seems to be spoofing and tagging don't work together. I can spoof non-tagged, and can tag non-spoofed, but can NOT spoof tagged.
Anyone else spoofing and tagging together?
-
you can do any of that, just that VMware complicates things because of its MAC restrictions. When you remove MAC spoofing it doesn't go back to the hardware MAC because there is no way to determine what that MAC is until you reboot. When you spoof on a VLAN parent NIC, it sets that for all the VLANs because of other complications with how that functions and the way many NICs work.
-
Why can't you use the default (hardware) MAC address on WAN1?
-
Why can't you use the default (hardware) MAC address on WAN1?
I could, but that is not what I want to do.
In the past I have always used spoofed the same easy to remember MAC towards the cable modem. This way when I want/need to switch internal devices I can simply shut down one, spoof the same MAC and bring up the other device anywhere on the cable modem VLAN without having to go to and rebooting the cable modem time and time again.
-
My experience was MAC spoofing and VLAN tagging does not work together. To work around my problem I set the spoofed MAC address that I wanted my cable modem to see from my WAN interface, inside the VM setup inside VMware and removed the spoofed MAC address from pfSense. This way pfSense sees and uses my spoofed MAC at boot time as if it was a MAC address on a physical NIC. My cable modem sees and locks to my spoofed MAC and all 3 WAN interfaces works correctly on separate VLAN's on the same physical interface.
This setup has been working fine for a week now. I finally have IP's on all three WAN interfaces with the cable modem locked to the MAC address I need it to use.
Thanks to everyone for all your suggestions!
