Hello empbilly,

@empbilly:

Look at the link.
https://eliasmoraispereira.wordpress.com/2016/10/05/pfsense-virtualizacao-com-xenserver-criando-vlans/

Solved!

Thank you.

"PFsense in a VM because its possible to brake ESX host security and well defeat the purpose of the firewall."

How would they have access to the esxi managment.. You wouldn't expose esxi to the internet - but sure ok if they compromise your host then yeah every vm on the host would be open..  But the internet is only connected to pfsense WAN.  esxi management should be on a different physical interface all together, etc.  So how would they even get to esxi to compromise its security?

With Kom - can you point out these things sayings its not secure so we can take a look..  There is a lot of FUD out there.. And then again your not running a DOD facility are you?  You stated its for your home use, etc.. So as long as you don't put your vmkern exposed to the public side there shouldn't be any issues at all.

Well, I think I found the problem. I have a CIFS ISO Library mounted from our samba server to make ISOS available to our xenserver.

I mounted a local ISO Storage on the xenserver and the installation happened normally. I configured it with an admin user and the permissions of the folder are ok. But either way, it's solved.

(¹) What I do not understand is why other ISOs that are in this share, work in their respective installations and the pfsense ISO does not !? :D

I'll investigate why it does not work in CIFS ISO Library!

EDIT: Possible answer about question (¹)
The pfSense ISO is packaged so as not to allow it to be transmitted at network installation time. (by JackL)

why would you need to do this?  My modem has gone down in the past - have never had to renew dhcp lease on such an event.  My pfsense is on esxi.

So I've moved the pfsense boxes to VMware and it seems to be the same issue. I must be doing something wrong on the pfsense boxes, if someone can kindly drop me a hint on where I may need to look.

thanks

well the speeds are good, but just wandering what speed i might have? as when you powershell route where would search the command to rehabilitate  it on the windows server 2012r2?

@Draven666:

Ok, I'll cut and paste (and slightly modify!) a message that I posted on the unRAID forum because it concerns virtualizing both products (unRAID and pfSense).

Let's start from the beginning. I built a pfSense server 3 or 4 years ago and I'm now in the process of upgrading it because it still run on version 2.1. I can really see your reaction, I know…I'm a bad guy but hey, if it ain't broke, don't fix it. So, I have a couple of questions for the community before diving head first in the upgrade process. First, since I'll probably upgrade the pfSense host machine, I would really like to virtualize it under ESXi 6.5. Is that possible and secure? Then, will I have to passthrough a couple of dedicated NIC to pfSense or virtual ones will do the trick? Since I'll run unRAID from the same box and probably a couple of Windows and/or Linux VM so, what kind of hardware can support this setup? pfSense doesn't need much so I don't think that I need a really powerful machine for them (unRAID and pfSense) but I would like to have some feedback from others. I have on hand an AMD Phenom II X4 945 or 965 Black Edition (can't remember exactly, but I can confirm upon request) on an Asus M4A89GTD Pro/USB3 or an Intel Q6600 on a P5K. From what I have read on the web, both of these boards don't support passthrough so, I'm looking at the Asus M5A99FX PRO R2.0. I would really like to find a board compatible with passthrough that I can use one of the processor I have on hand so, I can cut down the cost a little bit. I'll probably throw 16 or 32GB RAM, depending on the feedback I'll receive of this post. And, am I forced to use ECC ram for ESXi or non-ECC will do just fine?

For now, that's about it. I would like to thank everyone who took the time to read and answer my post.

Have a nice day.

pfSense is commonly virtualized, the security is good and the performance is good. It works on KVM, ESXI and Hyper-V, but is easiest to setup (GUI-wise) on the last two. (well… probably runs on Xen too, but it's not nearly as popular as the other 3 hypervisors mentioned)

You can choose to passthrough dedicated NICs, which would theoretically increase security, as the NICs are not shared with any other VMs, nor does the Hypervisor do any packet routing for you via vSwitches, but you lose some flexibility in configuration, as well as if you ever wanted to build a 2nd server and seamlessly vMotion/migrate the pfSense instance to the other host if the original host requires maintenance. That and you get simple backups, snapshot capability, etc. Still the option is yours.

My setup is as follows:

WAN connection VLAN2 on physical switch, trunked to ESXi.
ESXi host with 1 NIC (in reality there are more, but you only NEED one for this particular config)
vSwitch with portgroup WAN on VLAN 2 & regular LAN portgroup on native VLAN (0)/None
pfSense receives WAN signal on the VLAN 2 port, routes it through the LAN connection (OPT1, etc)

This is commonly known as a router-on-a-stick configuration, using a single NIC.

If you don't want to mess with VLANs or don't have a managed switch, then two NICs will be required on the host. Create 1 vSwitch with dedicated NIC for WAN, to be used exclusively for pfSense, and plug the WAN connection into that .
Create one or more vSwitches the LAN/OPT1/OPT2 connections, with the desired VMs also plugged into that switch for internet access. The LAN vSwitch NIC will provide internet access for the rest. You can create a vSwitch without a physical NIC attached to it if you only want to provide Internet access to to the VMs connected to it, and not the network at large.

Those CPUs are fine for pfSense, through running hot and power-hungryy for 24/7 use, but if you are going to run other VMs on it, it's probably OK :)

Now, the coolest thing you can do with this setup if you have another ESXI host with proper licensing (or VMUG learning license, $200 a year):

1. Have 2 hosts running in vCenter (the enterprise mgmt server for ESXI), identical vSwitch configurations, and be able to do a live migration of your router from one physical host to another without dropping a single packet.

2. Implement HA (high availability) monitoring so if one host or your pfSense VM goes down, it is restarted automatically on the other host.

Anyway, I'm a fan of virtualizing it, but be sure to know what you are doing, and understand the caveats of hosting your router on a VM sharing resources with other VMs, on a physical host that MAY need maintenance at times.

Turns out it was the virtIO causing issues and switched over to Intel Virtual NIC's

@kapara:

Since disabling have you had any issues?

Nope it has been up and running since I did the disable and I have had ZERO issues.

Yes if i remember right it's the same if you enable Hyper-V on your Windows installation -> Windows becomes a VM

Also Hyper-V is not reachable from the outside if you disable "Allow management operation system to share this network adapter" on
the virtual switch that is your WAN.

",two different network segments try communicate with each other must be used NAT"

No why does this seem to be a common thought.. Why would 2 different network segments connected to the same router need to be natted??  Do they overlap?  You do not need to nat between rfc1918 networks..

If your using KVM, have you read through the sticky
https://forum.pfsense.org/index.php?topic=88467.0

And where ae you placing these rules?  The default lan rules are any any… So if you bring up a vlan - lan should be able to talk to anything on the vlan out of the box.  If you can not - then you prob have a problem with the box on the vlan having a firewall.  Or maybe the vlan is not even correctly connected to pfsense.

Post up your rules on lan and vlans..

And how is your switch configured.  I have a gs108ev3 as well in my av cabinet that I run multiple vlans on..

Thank you for posting this, it's extremely helpful. I'm hopeful changes in 2.4 will benefit virtio performance? I'm not in a position where I can just pass through a nic dedicated to pfsense and so am at the mercy of virtio.

1. I need to setup the 3 physical NICS (LAN, WAN, MGT or maybe DMZ) with static IPs
You only need virtual switching.  Just add as many network adapters as youd like through hyper-v manager and your vm settings.

2. IP address(s) for virtual switch(s) and what types (ext, int or private?)
for outbound(wan) traffic, use an external switch and create at least one external network adapter for your pfsense vm.  You don't have to share this with management os, but take not that your management os wont have access to it.  in your pfsense configureation, this will be assigned an ip either by dhcp or staticely to match the external network.  create an internal switch for all other vms and even your host.  create adapters for all your vms and configure vms with pfsense internal ip address as default gateway.

3. IP addresses within pfsense (LAN, WAN etc).
pretty much answer to 2.
4. Endstate:  I have a WAN link with firewall rules applied and isolated from everything else, LAN link for filtered internet access and a LINK for management of pfsense (web interface and isolated to a workstation only).

I would suggest keeping it in an isolated environment until you are comfortable with it.  Then when you are sure of your abilities to manage it, put it into production.

heper - it's [iperf client] <-> [pfsense VM] <-> [iperf server] that all sit on the same switch;

"iperf client", "pfsense VM" and "iperf server" are each on their own hardware.

I don't this would be considered as multiple L3 setup right?

Been through all those suggestions but I appreciate the responses. I currently have 4 cores assigned, none seem to pin even under heavy load. The I350 nic settings I've left at their defaults. With the Broadcom's I had VMQ's disabled. The IPSec offload is enabled on the virtual NIC's. I'm starting to think that some how the Layer3 configuration is playing a role in the issue. I'm going to do a bit more research and follow up.

Hello heper.
I have looked on status/system log/gateway and there are messages like these:
Feb 9 13:24:48 dpinger WANGW 192.168.9.254: sendto error: 64

The 192.168.9.254 is the ISP's modem address.

On the other hand, this logs are real-time? the last message has date Feb 9 13:24:48, however, if this error message is related with my trouble, should there be messages from today's date? the pfSense firewall is currently operating and failing at every moment.

For the rest,  I have not found anything relevant in other logs options.

Thank you.
Luis

so my emX interfaces will not be changed in vmxX ?

No, of course not.