Interface xn0 not usable for tagging

[ggzengel]

While upgrading from 2.1.5 to 2.2 interface changed from em0 to xn0.
After loosing all interface/vlan/fw settings I replaced the strings in backup.xml and restored it.

At "Interfaces: VLAN" I can't use xn0 for tagging. The still configured interfaces are working, but I can only choose igb0 if I open the vlan config.

xn0 is used untagged (default vlan) and tagged. ICMP and UDP is possible between untagged and tagged interfaces but TCP didn't work any more.
Between tagged interfaces TCP is working.
With 2.1.5 it worked perfectly.

Now I will go back to 2.1.5.

[phil.davis]

I think it will be helpful for the devs if you report exactly what hardware you have (NIC card…), because it would be really good to know if a device name like "em0" is FreeBSD 8.3 has changed to be "xn0" in FreeBSD 10.1 - then there could be some thought given to how to reliably detect what names change, and convert the config on upgrade.

[ggzengel]

I think they now it, because they use xen too.
The NIC changed from emulated to paravirtual.

[phil.davis]

I see that others have this in certain virtual environments. Search for "xn0" in this: http://bsd.slashdot.org/story/14/01/20/1731239/freebsd-100-released
It would be useful to think if there is some way to detect this during upgrades.
And also to let pfSense know what capabilities "xn" has.

[cmb]

We don't use much Xen at all (outside EC2 which is kind of a diff world), mostly VMware, some KVM and Hyper-V.

Why is Xen changing your NIC type just on an OS change? That's not a good behavior for any hypervisor. If you're on e1000, you should stay on e1000 unless you change the type at the hypervisor level. What it's doing there would break every OS in existence until you reconfigured the NICs.

If tagging doesn't work on xn0 that's an issue. The fact Xen changes it out on you like that is a POLA violation on its part.

[ggzengel]

I really want it to change from e1000 to PV NICs. It needs less resources and is faster. Now my pfsense will shutdown if my xen shuts down. Until 2.2 my XEN will shot (!) down my VM.
The block devices changed too, so I had to edit fstab (now I'm mounting the UID device). But it's easy to handle it and I like this.
I think it's possible to disable PV on HVMs but who wants?

The 2 issues are still there:

I don't know if it depends on xn0 or if it's always a problem about TCP routing between tagged and untagged subnets on same interface.

xn0 is not selectable for VLANs (perhaps it is still used as untagged interface)

That's the working config in 2.1.5:

	 <interfaces><wan><if>pppoe0</if>

			 <spoofmac><enable><ipaddr>pppoe</ipaddr></enable></spoofmac></wan> 
		 <lan><if>em0</if>
			 <enable><spoofmac><ipaddr>192.168.180.1</ipaddr>
			<subnet>24</subnet></spoofmac></enable></lan> 
		 <opt1><if>ovpnc1</if>
			 <enable><spoofmac></spoofmac></enable></opt1> 
		 <opt2><if>ovpnc2</if>
			 <enable><spoofmac></spoofmac></enable></opt2> 
		 <opt3><if>em0_vlan12</if>
			 <enable><spoofmac><ipaddr>10.32.12.1</ipaddr>
			<subnet>24</subnet></spoofmac></enable></opt3> 
		 <opt4><if>em0_vlan13</if>
			 <enable><spoofmac><ipaddr>10.32.13.1</ipaddr>
			<subnet>24</subnet></spoofmac></enable></opt4> 
		 <opt5><if>em0_vlan14</if>
			 <enable><spoofmac><ipaddr>10.32.14.1</ipaddr>
			<subnet>24</subnet></spoofmac></enable></opt5> 
		 <opt6><if>em0_vlan190</if>
			 <enable><spoofmac><ipaddr>192.168.19.1</ipaddr>
			<subnet>24</subnet></spoofmac></enable></opt6> 
		 <opt7><if>em0_vlan2134</if>
			 <enable><spoofmac><ipaddr>10.33.21.1</ipaddr>
			<subnet>24</subnet></spoofmac></enable></opt7> 
		 <opt8><if>em0_vlan2135</if>
			 <enable><spoofmac><ipaddr>10.33.22.1</ipaddr>
			<subnet>24</subnet></spoofmac></enable></opt8> 
		 <opt9><if>em0_vlan3456</if>
			 <enable><spoofmac><ipaddr>192.168.255.1</ipaddr>
			<subnet>24</subnet></spoofmac></enable></opt9> 
		 <opt10><if>ovpns4</if>
			 <enable><spoofmac></spoofmac></enable></opt10> 
		 <opt11><if>ovpns3</if>
			 <enable><spoofmac></spoofmac></enable></opt11> 
		 <opt12><if>igb0</if>
			 <spoofmac><enable></enable></spoofmac></opt12></interfaces> 
	 <vlans><vlan><if>em0</if>
			<tag>12</tag>

			<vlanif>em0_vlan12</vlanif></vlan> 
		 <vlan><if>em0</if>
			<tag>13</tag>

			<vlanif>em0_vlan13</vlanif></vlan> 
		 <vlan><if>em0</if>
			<tag>14</tag>

			<vlanif>em0_vlan14</vlanif></vlan> 
		 <vlan><if>em0</if>
			<tag>190</tag>

			<vlanif>em0_vlan190</vlanif></vlan> 
		 <vlan><if>em0</if>
			<tag>2134</tag>

			<vlanif>em0_vlan2134</vlanif></vlan> 
		 <vlan><if>em0</if>
			<tag>2135</tag>

			<vlanif>em0_vlan2135</vlanif></vlan> 
		 <vlan><if>em0</if>
			<tag>3456</tag>

			<vlanif>em0_vlan3456</vlanif></vlan></vlans> 

[xxbelocxx]

I can confirm that xn0 interfaces do not allow VLANs. I just did a fresh install on XenServer Cadence 6.5 and everything works great, except when trying to add VLANs. It dosent even show the interface as being able able to provide VLANs.

[cmb]

@ggzengel:

I really want it to change from e1000 to PV NICs.

Whether or not you want it to change isn't relevant to whether it should on its own, your system broke because Xen did something that a hypervisor should never do.

The xn driver reports it isn't VLAN-capable, which is why it doesn't show as an option for parent interface. For the time being if you're tagging within a VM, you'll need to make it use e1000 instead. I don't see a FreeBSD PR on that as an issue, should be reported upstream.

[ggzengel]

Starting from FreeBSD 10 the GENERIC kernel configuration on both i386 and amd64 contains full Xen PVHVM support, so there's no need to compile a specific kernel in order to get Xen drivers. Previous versions of FreeBSD (8.x and 9.x) required the user to compile a custom (XENHVM) kernel in order to make use of the PV optimizations when running inside of a HVM container.

What do you expect if PV is enabled at hypervisor and em0 was an fall back and suddenly PV is available in a new kernel?
It's like with all other drivers. If there is a new driver in a new kernel which fits better it will be loaded first.

But as I said: I really want the new xn0 because it needs less CPU time. But tagging must be possible. Please report it upstream.
If you patch the config.xml tagging was possibe for ICMP and UDP.

The next step after migrating to debian jessie (Xen 4.4) is to test PVH (without QEMU): https://wiki.freebsd.org/FreeBSD/XenNG
But I guess pygrub has to learn more about UFS to grab the kernel from the image.

Is it possible to boot pfsense disk less from nfs?