PfSense 2.3 and Xen/KVM/XenServer

corotte

i'm using Xenserver since a while (now on 6.5 SP1) and got the well known problem of tx checksum offloading as describe in the following
https://forum.pfsense.org/index.php?topic=88467.0

Now, i've upgraded to PfSense 2.3
I still haven't try to test by putting back on checksum offloading but since PfSense 2.3 is based on FreeBSD 10.3, does this issue apply to 2.3 ?

I just want to make sure before doing a wrong move.

rainer_d

Would also be interested in knowing this.
Turning off hw-checksumming is not really an option - unless these settings are also transfered once a VM is migrated to another physical server (or restarted there when the original server has crashed).
I would so love to have pfSense replace the default, pathetic software-defined networking in Apache Cloudstack (on XenServer).

fohdeesha

the issue still applies and always will as long as freebsd (pf) drops packets that don't have a checksum (invalid checksum) (which is usually a good decision for a router), as by default in xen, inter-vm traffic that never leaves a real interface doesn't get a real checksum done (also a good decision, they don't need one, they're using shared memory). Both of these things are design choices that by themselves aren't an issue but together people run into the aforementioned issue of freebsd dropping said inter-vm traffic, even though it was only ever in shared memory and couldn't have been corrupted over a physical link, it's still showing up on what pfsense sees as a physical interface with no checksum

All you need to do is the above mentioned tx offload=off command for the xen virtual interfaces on the pfsense VM only - that makes it generate a valid checksum for that inter-vm traffic so pf doesn't drop it, doesn't cause any noticeable increase in cpu usage from what I've noticed pushing gigabit traffic on an older intel cpu. The setting is a xen property of the pfsense VM and sticks across reboots and "should" stick with the VM across hardware migration like any other xe custom vm properties you have set. that's all you need to change, don't change or toggle anything in pfsense itself as the issue has nothing to do with actual hardware offloading

I outlined how to do this with 2.3 (as well as install guest utils) here: https://forum.pfsense.org/index.php?topic=109253.msg608562#msg608562

this also obviously causes issues on quite a few other nix distros that have the same default behavior (dropping packets that arrive with no/invalid checksum), and also other hypervisors that skip checksumming for inter-vm traffic that never leaves memory, and the solution is the same for all, granted I only know xen-specific commands

there are a couple bug reports about it filed with freebsd and maybe one day we might not have to do this, if the recommended workarounds are implemented https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197344 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=154428

corotte

That explain a lot of things !

I've check the bug report and it does not seem to catch the attention or get any priority at the time of writing.

Is there any way to put pressure or ask for a possible ETA for this bug ?

I think that if many people "push" the bug report, we could get a resolution soon.