PFSense 2.3, Xenserver & Traffic Shaping
So I came to the unhappy realization tonight that apparently traffic shaping is broken on pfsense running on Xenserver. However, I read a post in the forum indicating that you can disable PV Nics in the bootloader for PFSense which apparently switches it to an emulated NIC and allows altq traffic shaping and VLANs?
My current setup is something I built for a local camp. They have a Dell Poweredge 2950 that was donated to them by a local business. They have two (horrible) DSL connections that top out at 3mb each on a good day. The server had 2 onboard nics plus two more Intel Dual Gigabit NIC's installed. I've got 2 interfaces on one of the intel nics setup in Xenserver, and PFSense is the only virtual machine with those interfaces added. Then it uses the same interface that Xenserver uses for management for the LAN connection. Load balancing is working great. Got the VPN to work. Have squid3 running trying to help save bandwidth as much as possible. Their file server and domain controller also run on this Host as virtual machines.
They have Ubiquiti AP's setup with two SSID's, production and guest. Currently both on the same subnet, but the guest has client isolation enabled. However, it doesn't take much for a few kids with smartphones to saturate the DSL connections that the office is trying to use. What I want to do is setup the ubiquiti AP's to have a separate VLAN for guest access that we can then use traffic shaping to give that VLAN lower priority than the office VLAN (plus more security by having it firewalled off by itself)
I had a couple questions,
If I add the lines to the bootloader referenced here: https://forum.pfsense.org/index.php?topic=108156.0
Is that all that has to be done? Or is there something different that needs to be configured on XenServer? I guess what I'm wondering is if I set it and reboot, will PFSense reconfigure itself or just be broken at that point? Do I need to setup a new PFSense VM from scratch?
I understand that an emulated interface will have higher cpu usage. But, this is a dual quad core Xeon machine at 3.2ghz, so power is not lacking. Neither is memory at 64GB.
Thanks for any input or direction you could give me!
OK, so I spun up a test VM on Xenserver here at my home. Did an install similar to the camp, no traffic manager. Then, I went into loader.conf.local and added the hw.xen.disable_pv_nics = 1 line and rebooted. I of course had to re-assign my interfaces at the console. Afterwards, traffic queues and VLAN options appeared.
However, the NICS showed up as realtek NICS, at 100Mb. Is there any way to make PFSense recognize link speeds higher than 100mb when PV is disabled? In this case, it's not important, but it may be more important down the road.
While this doesn't answer any of your questions, something to be aware of is because FreeBSD only allows shaping egress, you can't shape Squid's downloads since it attaches to the WAN interface and the download traffic does not go out any interface.
Do yourself a favor - run the Vmware converter on the VM's , reload the server to free ESXi , import the VM's , call it a day.
Having used both Xenserver from 5.0 to 6.5 and Vmware products - I feel the support and underlying hypervisor is better in Vmware than Xenserver currently. Back when Xenserver came out they had Vmware by the balls on licensing (Xenserver licensed per socket back then and Vmware was per core) forcing Vmware to change gears to keep market share.
I ran it in production with 5 hosts in a Xencluster with an iSCSI SAN running Exchange / SQL / Sharepoint and other VM's it ran fine just had some quirks you had to get used to dealing with.
Switched to Vmware - never looked back.
There are threads in the virtualization forum about Xenserver , try searching there. Good luck with it.
OK, so PFSense only shapes egress. Makes perfect sense! So I will setup shaping on the wan ports for upload, and the lan ports for download.
If I assign an adapter just to pfsense to use for it's LAN connection, I should be able to traffic shape that. It will mean two connections back to the switch, but that's not an issue. I have unused interfaces left on this server.
As for ESXi, not going to happen. We need the ability to do VM backups and VMware free is so stripped down, you can't do much useful with it. Xenserver can do everything except vGPU (not needed) for free.
I'll try to transition to dedicated interfaces for PFSense, then disable the PV. See what happens from there.
As for reading up; yes, I've done quite a bit about PFSense on Xenserver, hence why I knew what I already did. Just trying to find a decent solution.
Thanks for the help so far. I'll report back with my progress.
You can backup VM's in the free ESXi , it's just not done in the same way as paid Vmware but there are products out there to do it. I use this one all the time - http://www.altaro.com/vm-backup/
With traffic shaping if you use floating rules on the WAN it creates an inverse rule for traffic going out the LAN no need to duplicate the rule set.
Did you ever work out how to get Pfsense to detect the nics as 1000 instead of 100?
I know this is a bit tab outdated but I have successfully found a way to use intel drivers instead.
This method works for xenserver 7.2 (might need some minor changes for 7.1 and below)
You have to modify the file /usr/libexec/xenopsd/qemu-dm-wrapper with following after the def main(argv) line:
argv = [arg.replace('rtl8139', 'e1000') for arg in argv]
This will use the intel drivers instead for all the VMs on the xenserver. I tried a few other ways but this seemed to be the most reliable and consistent for use in a production environment.