Multiple pfSense VMs on the same ESXi host
I'm trying to understand if I have a problem or not. I have two pfSense VMs on a single ESXi host. The host has a single NIC.
Each pfSense VM listens on its own IP.
If I ping one of these: 208.xxx.xxx.72
64 bytes from 208.xxx.xxx.72: icmp_seq=7 ttl=54 time=31.352 ms
36 bytes from 208.xxx.xxx.68: Destination Host Unreachable
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 5400 1b73 0 0000 30 01 3ef1 192.168.1.10 208.xxx.xxx.72
If I ping the other one: 208.xxx.xxx.68 I get the exact same result, except that the .72 is unreachable.
My question is why am I seeing the 208.xxx.xxx.68 at all when pinging the 208.xxx.xxx.72 and vice-versa?
Am I missing some setting in the pfSense configuration as to how these should be?
I am having issues with one of these, and possibly both, though I am only seeing the problem right now on one, so I'm wondering if this might be related.
The 208.xxx.xxx.72 has an Exchange server behind it and there are some persons who can send an email to this IP and it never arrives. I am using the Postfix Forwarder on this, so I can tail the logs live and watch. Two individuals have been identified who can send an email to this server and it never arrives and it never even shows in the logs as being rejected for any reason. There is a load of email coming in fine and it seems to work except for these two, and possibly others yet to be identified.
I can't figure out what is happening to these certain individuals. They can send to other email systems I've setup and they come in fine, so they can send, and they've never had this with anyone else.
Baffled at this point. Thanks for any help.
Do you have VLANs setup? Are both the .68 and .72 on the same subnet? Did you clone the one box from the other? Check and see if you have the same MAC address for the two firewalls as that would cause all kinds of random issues.
Yes, I have VLANs on both.
.68 and .72 are on different subnets.
I did clone one from the other.
MAC address was set on one but on the other it was blank. This is on the WAN interface. Is there another place it could be set?
Hmm… If you do an ifconfig from the shell on each box, there should be a line showing "ether MAC ADDRESS". Just make sure that they have different addresses on them. Also, in the Vsphere Client when you look at the network settings of the host, I think there might be a place to specify a mac address there. Under VMWare server it shows my clients mac, and there is either manual or generated by host.
Ok. I see a problem. Both have identical MAC addresses.
How can these be changed? From reading further on that it seems I have to change the IP in the host machine and that should do it. But I've done that and it didn't. Since these are clones of each other I'm assuming that is how this came to be and that by changing the IP in UI somehow did not change this.
Can I just make up a MAC and put it it? I'm not sure what a valid MAC would be, but I can probably google for it.
That was it! With so few MACs on my system I just took the existing MAC in the pfSense UI and modified it slightly and that worked.
The ping is normal now and the missing email is coming.
Thanks bman212121, saved my butt on this one.
You're welcome :)