[Fixed] Interface stops receiving traffic

veloc1ty

Hi @all,

I'm trying to trace down an issue where interfaces stops receiving traffic.
My pfSense Version is 2.2.6-RELEASE (amd64). This machine is running in a QEMU/KVM virtual machine with some virtualized e1000 interfaces.

In this case the interface em3 has stopped working. On this interface there is the IP range 192.168.2.0/30 operating. The two possible IPs are used like this:
The pfSence interface has the IP 192.168.2.1 and acts as gateway.
The server is on IP 192.168.2.2.

The server is not able to send or receive traffic anymore. I've checked the host system and found no errors. I did some traces.

Here I'm tracing on the firewall. I see the ARP requests from the firewall to the server:

[2.2.6-RELEASE][admin@firewall.veloc1ty.lan]/root: tcpdump -n -i em3
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em3, link-type EN10MB (Ethernet), capture size 65535 bytes
capability mode sandbox enabled
10:17:43.721112 ARP, Request who-has 192.168.2.2 tell 192.168.2.1, length 28
10:17:44.972282 ARP, Request who-has 192.168.2.2 tell 192.168.2.1, length 28
10:17:45.663860 ARP, Request who-has 192.168.2.2 tell 192.168.2.1, length 28

These requests are transmitted to the vnet4 interface on the hostsystem which is the corresponding virtual NIC interface of pfSense:

root@home:~# tcpdump -n -i vnet4
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet4, link-type EN10MB (Ethernet), capture size 262144 bytes
10:20:05.934865 ARP, Request who-has 192.168.2.2 tell 192.168.2.1, length 28
10:20:05.935031 ARP, Reply 192.168.2.2 is-at 52:54:00:75:1f:3c, length 28

As you can see the server is replying. Communication on Layer 2 + 3 on the hostsystem is therefore working. But after giving the traffic back to pfSense itself it's somewhere dropped.

I assume it's a problem with pfSense, because after doing

[2.2.6-RELEASE][admin@firewall.veloc1ty.lan]/root: ifconfig em3 down
[2.2.6-RELEASE][admin@firewall.veloc1ty.lan]/root: ifconfig em3 up

the ARP request are showing up and traffic can flow again.

Does anybody of you have an idea on how to resolve this issue? Maybe you can provide me more details to dig in deeper and get more information.

Thanks in advance,
~velo

veloc1ty

Quite a while ago I figured out what's wrong:
While running pfSense in a KVM setup make sure you don't use rtl8139 as network interfaces but virtio instead.

Positive aspects:

10G instead of 1G
No broken traffic
Less overhead

Negative aspect: You have to remap your interfaces in pfSense.

I don't know the exact cause of of the problem but the workaround is pretty nice.