PfSense on esxi 6.7, can get it to work propperly.
-
It only works when i set the WAN on VM Network.
When i set it on WAN it wont work. -
Then it is related to the realteks. You somehow need to validate whether they operate properly in ESXI at all. Another option could be, as mentioned by @kiokoman, to change to vmxnet3 in pfsense-VM-settings of ESXI.
-
yeah in any case realtek card are never a good choice for this stuff
-
ok.
In the meantime i have tried to run PfSense directly installed on a SSD.
That is running fine, and working woth both the INTEL nic for the WAN and the realtek as the LAN.The devices that are attached trough a LAN kabel are getting 10.0.0.X and internet access,
When i tried it with windows 2019 its working perfect without a problem. So i think i need to consider to leave ESXI for what it is. or try a different hyperviser OS.i only want to use it for:
1x windows server machine
1x firewall machine
2-3 linux machines.Any thought?
-
change that realtek card to some intel they are cheap on ebay / amazon after all
-
Exactly what I would recommend as well. It would cost only a few bucks and will work right out of the box with ESXI. Otherwise you need to spend countless hours or days to get into another hypervisor. Do the math to your liking :)
-
Thank you both for the help :) i will buy a Intel card ;)
I just found one of these: https://ark.intel.com/content/www/us/en/ark/products/184824/intel-ethernet-network-adapter-i350-t4-for-ocp-3-0.html for €20 euro.
Will this one work?
-
With ESXi have you tried putting the LAN vSwitch and Port Group into promiscuous mode ?
-
ok so, i bought this https://ark.intel.com/content/www/us/en/ark/products/184824/intel-ethernet-network-adapter-i350-t4-for-ocp-3-0.html inserted it in the ESXI and booted up. It works perfectly.
So now it is running and working like this:
Fiber optic > Ubiquiti USG router WAN > Ubiquiti USG router LAN > network port NIC ESXI WAN > network port NIC ESXI LAN > Network switch and accespoint in the house. ( 10.0.0.X range. ) this is working perfect.
But i want to remove the Ubiquiti usg router so i can setup the fiber optic directly into the WAN port of the NIC of the ESXI.
so i'm working with T-Mobile here in here in the Netherlands. and i have added the VLAN 300 for internet into the PfSense on the interface of the WAN port of the NIC:
Under interfaces > assigment i have put the new VLAN300 into the WAN:
But i dont get a IP from the DHCP server of my ISP provider.When i set it back on the normal settings, everything is working perfect and i get a 192.168.1.2 from the Ubiquiti USG.
-
under esxi did you set the vswitch to vlanid 4095 ?
-
So i need to edit this one:
This is the nic port that is comming from the fiber optic.And can you tell me WHY i need to add VLAN 4095?
-
This is a specific of ESXI. If empty, VLAN are NOT supported. 4095 allows ALL VLAN numbers from the VM. I still get confused about VLAN, tagged, untagged, etc. But this should work,
-
indeed 4095 means that you set your vswitch as a trunk port letting any vlan pass through
-
Yes, this did the trick :)
-
so after a few day's working perfectly.
it now randomly stops working.In the PfSense VM i see this:
When i reboot the VM everything works again.
What can this be? -
try to reinstall open-vm-tools
maybe do a fsck also -
i have reinstalled the open-vm-tools and nothing changed
what you mean by fsck?And what about the new update?
i'm now on:2.4.4-RELEASE-p3 (amd64)
built on Wed May 15 18:53:44 EDT 2019
FreeBSD 11.2-RELEASE-p10 -
fsck: from console is option 5 and F key ( F: Reboot and run a filesystem check )
and yes, upgrade to 2.4.5 -
ok now i understand what you mean. i just did that a couple of hours ago and it seems to work again.
What about the update?after the update i need to reinstall vm-tools again? and another fsck?
-
nope, no need, make a backup of your config just in case and do the upgrade
-
Still this is not working fine. My internet disconnects every 4-5 hour.
So can you please check my settings and tell me if the are right:First off all, the network settings:
-
ESXI WAN settings: + VLAN ID 4095
-
ESXI LAN settings:
-
PfSense settings:
Cable managment:
WAN T-mobile > WAN Port intel NIC ESXI
LAN Port intel NIC > port 1 into standard network switch. ( not managed )
port 2 network switch > Onboard LAN port of ESXI server for managment.
The rest of the network ports of the switch are 2 Ubiquiti WIFI accespoint, and some computers.As you can see, i also installed OpnSense just to find our of PfSense was the problem.
But also OpnSense has the same problems. -
-
it's ok, was the cable removed during this screenshot i suppose? physical adapters are not green
if you are using igbn (native esxi driver) try to install net-igb (intel driver for esxi)also install open-vm-tools from pfsense packages if you didn't already
-
Yes, the cable is removed otherwise i dont have good internet ofcourse :)
Both the drivers are installed:
Also the package in PfSense for open-vm-tools is installed.
-
idk, settings are ok, maybe do a backup of your configuration ad reinstall a new clean vm with pfsense 2.4.5 or try 2.5.0
-
Can I get a Teamviewer into the box... then I will have your ESXi/pfsense running in 5 mins
-
@kiokoman
I have tried freshinstall of 2.4.5 and 2.5.0
I have tried backing up and restoring.Maybe te problem is the fact i dont enter a MAC-ADDRESS into the wan port of the T-mobile fiber?
and i leave the MTU also clear?@Cool_Corona
if you can do it in 5 minutes. you can also tell me how ;) -
that would be something specific to your isp
-
So i tried adding a MAC-ADDRESS but then there is no internet at all.
Where in PfSense can i find a log? so maybe i can see where the problem is occurring. -
all logs are under status / system log
what kind of connection is it? pppoe ? dhcp? static or dynamic ip? -
I have a cable modem and for testing purposes I have been switching between a pfSense VM and physical appliances.
Not sure why - but my connection is stable with the mac address used when the connection was first setup. This may be a provider thing...
Either way - if you want your pfSense VM to use a custom mac address on the WAN interface you'll need to allow forged transmits.
I'd also suggest you enable promiscuous mode.
https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.security.doc/GUID-7DC6486F-5400-44DF-8A62-6273798A2F80.html
and
https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.security.doc/GUID-92F3AB1F-B4C5-4F25-A010-8820D7250350.html
and
https://wahlnetwork.com/2013/04/29/how-the-vmware-forged-transmits-security-policy-works/
For good measure also allow mac changes.
Once its all working... you can revert these settings until you know exactly which setting breaks your environment.
To sum this up: on the WAN vSwitch and Portgroup you need to:
-
Allow Mac Changes
-
Allow Forged Transmits
-
Allow Promiscuous Mode
-
In pfsense - on the WAN Interface use a Mac Address that has previously worked with your provider
-
-
These where my settings when i was using the Ubiquiti USG router:
-
ok, again the internet stopt working and i pull this out the logfile:
Apr 6 14:32:33 check_reload_status updating dyndns wan
Apr 6 14:32:33 check_reload_status Syncing firewall
Apr 6 14:32:33 php-fpm 2504 /interfaces_assign.php: Creating rrd update script
Apr 6 14:32:45 check_reload_status Syncing firewall
Apr 6 14:32:49 check_reload_status Syncing firewall
Apr 6 14:32:52 login login on ttyv0 as root
Apr 6 14:33:06 login login on ttyv0 as root
Apr 6 14:33:52 php-fpm 2504 /interfaces.php: Gateway, none 'available' for inet, use the first one configured. 'WAN_DHCP'
Apr 6 14:33:52 php-fpm 2504 /interfaces.php: Gateway, none 'available' for inet6, use the first one configured. ''
Apr 6 14:33:52 check_reload_status Restarting ipsec tunnels
Apr 6 14:33:54 php-fpm 2504 /interfaces.php: Unbound /var/unbound/root.key file is corrupt, removing and recreating.
Apr 6 14:33:56 check_reload_status updating dyndns wan
Apr 6 14:33:56 kernel vlan0: changing name to 'vmx0.300'
Apr 6 14:33:58 check_reload_status Reloading filter
Apr 6 14:33:58 php-fpm 2504 /interfaces.php: Creating rrd update script
Apr 6 14:34:24 check_reload_status rc.newwanip starting vmx0.300
Apr 6 14:34:24 php-fpm 340 /interfaces_assign.php: Gateway, none 'available' for inet, use the first one configured. 'WAN_DHCP'
Apr 6 14:34:24 php-fpm 340 /interfaces_assign.php: Default gateway setting Interface WAN_DHCP Gateway as default.
Apr 6 14:34:24 php-fpm 340 /interfaces_assign.php: Gateway, none 'available' for inet6, use the first one configured. ''
Apr 6 14:34:24 check_reload_status Restarting ipsec tunnels
Apr 6 14:34:25 php-fpm 2504 /rc.newwanip: rc.newwanip: Info: starting on vmx0.300.
Apr 6 14:34:25 php-fpm 2504 /rc.newwanip: rc.newwanip: on (IP address: REMOVED FOR PRIVACY!!!!!!) (interface: []) (real interface: vmx0.300).
Apr 6 14:34:25 php-fpm 2504 /rc.newwanip: rc.newwanip called with empty interface.
Apr 6 14:34:25 php-fpm 2504 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> REMOVED FOR PRIVACY!!!!!! - Restarting packages.
Apr 6 14:34:25 check_reload_status Reloading filter
Apr 6 14:34:25 check_reload_status Starting packages
Apr 6 14:34:26 php-fpm 341 /rc.start_packages: Restarting/Starting all packages.
Apr 6 14:34:26 php-fpm 340 /interfaces_assign.php: Unbound /var/unbound/root.key file is corrupt, removing and recreating.
Apr 6 14:34:26 login login on ttyv0 as root
Apr 6 14:34:29 check_reload_status updating dyndns wan
Apr 6 14:34:29 check_reload_status Syncing firewall
Apr 6 14:34:29 php-fpm 340 /interfaces_assign.php: Creating rrd update script
Apr 6 14:35:01 pkg-static pfSense-repo upgraded: 2.4.5 -> 2.4.5_2
Apr 6 14:35:09 check_reload_status Syncing firewall
Apr 6 14:35:14 pkg-static fusefs-libs-2.9.9_1 installed
Apr 6 14:35:14 pkg-static libdnet-1.13_3 installed
Apr 6 14:35:14 pkg-static libmspack-0.10.1 installed
Apr 6 14:35:14 pkg-static open-vm-tools-nox11-11.0.1_2,2 installed
Apr 6 14:35:14 php /etc/rc.packages: Beginning package installation for Open-VM-Tools .
Apr 6 14:35:14 check_reload_status Syncing firewall
Apr 6 14:35:14 check_reload_status Syncing firewall
Apr 6 14:35:14 php /etc/rc.packages: Successfully installed package: Open-VM-Tools.
Apr 6 14:35:14 pkg-static pfSense-pkg-Open-VM-Tools-10.1.0_2,1 installed
Apr 6 14:35:16 check_reload_status Reloading filter
Apr 6 14:35:16 check_reload_status Starting packages
Apr 6 13:34:57 php-fpm 340 /rc.start_packages: Restarting/Starting all packages.
Apr 6 14:35:18 kernel VMware memory control driver initializedAfter a reboot of the ESXI host, everything is working again.
-
uhm check the dhcp log and the gateway log also
REMOVED FOR PRIVACY!!!!!! your ip is 2 lines below -
All files attached. :)
Hope this helps. -
No DHCPOFFERS received.
maybe try with e1000 driver instead of vmnet3
-
So i need to reset the Pfsense ESXI to default.
inside the esxi change both adapters from vmnet3 to e1000.
fire up PfSense and do it all over? -
no just stop pfsense, edit the vm change from vmxnet3 to e1000 restart the vm, do it only for wan.
when you start pfsense it will noticed the change and ask you for the new wan interface in the console -
ok perfect, i have done this now and i also cleared all the logs.
-
AND AGAIN, it stopt :( :( :(
New files are attached.DHCP LOG.txt GENERAL LOG.txt -
there is nothing wrong in the log this time, must be something on your esxi or the network card or the cable