[solved] VLAN and pfsense as KVM guest (no switch)

[lbm_]

Hi,

Im having difficulties to get my VLANs to work in pfsense.
My lab setup is as following:

KVM host
KVM guest VM
KVM guest pfsense

The KVM host, has an bridged interface, which is used by both pfsense, and the guest VM, and its also used for non vlan traffic. This interface is not connected to any switch, since traffic is just parsing inside the KVM host + kvm guest, and also has an non vlan IP in a different subnet.

The guest VMs interface is configured as eth0.50.
PFSense, has an interface configured as vtnet1.50, + LAN and WAN (non vlans).

The problem is, I cannot connect to anything related to the vlan 50 network on pfsense, from this guest. If I try to tcpdump, I can see the traffic with vlan ids.

example:
From the guest VM, I try basic ping against the pfsense server, but no response:

ping 192.168.50.2
PING 192.168.50.2 (192.168.50.2) 56(84) bytes of data.

From the KVM host, if I do an tcpdump, I can see the vlan id 50..
vnet7 = virtual interface of the guestVM
vnet2 = virtual interface of pfSense

 tcpdump -nn -i vnet7 -e vlan
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet7, link-type EN10MB (Ethernet), capture size 262144 bytes
23:04:34.574633 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 171, length 64
23:04:35.598534 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 172, length 64
23:04:36.622708 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 173, length 64
23:04:37.646491 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 174, length 64

tcpdump -nn -i vnet2 -e vlan

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet2, link-type EN10MB (Ethernet), capture size 262144 bytes
23:05:19.628811 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 215, length 64
23:05:20.652802 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 216, length 64
23:05:21.676749 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 217, length 64
23:05:22.700682 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 218, length 64
23:05:23.724662 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 219, length 64
23:05:24.050950 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 85: vlan 50, p 0, ethertype IPv4, 192.168.50.13.49370 > 192.168.39.2.53: 46653+ A? 2.debian.pool.ntp.org. (39)
23:05:24.050974 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 85: vlan 50, p 0, ethertype IPv4, 192.168.50.13.49370 > 192.168.39.2.53: 56646+ AAAA? 2.debian.pool.ntp.org. (39)
23:05:24.748613 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 220, length 64
23:05:25.772583 52:54:00:4e:72:1c > 52:54:00:49:49:69, ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 802, seq 221, length 64

If I inside pfsense, tcpdump, again I can see the vlan.

tcpdump -i vtnet1 -e vlan
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144 bytes
23:08:35.175036 52:54:00:4e:72:1c (oui Unknown) > 52:54:00:49:49:69 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 813, seq 140, length 64
23:08:36.199005 52:54:00:4e:72:1c (oui Unknown) > 52:54:00:49:49:69 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 813, seq 141, length 64
23:08:37.223215 52:54:00:4e:72:1c (oui Unknown) > 52:54:00:49:49:69 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 813, seq 142, length 64
23:08:38.246896 52:54:00:4e:72:1c (oui Unknown) > 52:54:00:49:49:69 (oui Unknown), ethertype 802.1Q (0x8100), length 102: vlan 50, p 0, ethertype IPv4, 192.168.50.13 > 192.168.50.2: ICMP echo request, id 813, seq 143, length 64

I cannot see any blocks in the firewall log in pfsense.

Any suggestions ?

[lbm_]

Im not really sure exactly what was wrong...

I've started from scratch, and came to the same or new issues. Doing troubleshooting, i found that the broadcast address was way off, which I did not really understood. I then found that the VLAN interface was created as /32 CIDR, which it defaults to, so its highly important to remember to change this.

Changed it to /24 CIDR, and then it started working.