pfSense 2.4.3 HVM with PCI passthrough - no packets received

pietrushnic

Hi all,
I'm firmware maintainer of PC Engines apuX platforms and I'm trying to test PCI passthrough with pfSense. Unfortunately despite having Debian working without problem pfSense have some issues. I'm not experienced with FreeBSD, so maybe I'm missing something obvious.

I create HVM:

name = "pfSense-2.4.3"
builder = "hvm"
vcpus = 2
memory = 2048
pci = [ '02:00.0' ]
nographics = 1
serial = "pty"
# disk=[ '/root/pfSense-CE-memstick-serial-2.4.3-RELEASE-amd64.img,,hda,rw', '/dev/vg0/pfsense,,hdb,rw' ]                                                                                    
disk=[ '/dev/vg0/pfsense,,hdb,rw' ]

pfSense installs and boot without problem in logs I see it correctly detect 02:00.0 NIC:

igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6400bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>                                                                       
        ether 00:0d:b9:43:3f:bd
        hwaddr 00:0d:b9:43:3f:bd
        inet6 fe80::20d:b9ff:fe43:3fbd%igb0 prefixlen 64 scopeid 0x1
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

I tried both static and DHCP configuration of WAN port but non of that work. I see that DHCP request leaves network and reach the router, which reply with DHCP offer but nothing is received on pfSense side. Trying static IP also doesn't work, since I even can't ping router (192.168.3.1):

[2.4.3-RELEASE][root@pfSense.localdomain]/root: netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
localhost          link#2             UH          lo0
192.168.3.0/24     link#1             U          igb0
192.168.3.2        link#1             UHS         lo0

Internet6:
Destination        Gateway            Flags     Netif Expire
localhost          link#2             UH          lo0
fe80::%igb0/64     link#1             U          igb0
fe80::20d:b9ff:fe4 link#1             UHS         lo0
fe80::%lo0/64      link#2             U           lo0
fe80::1%lo0        link#2             UHS         lo0

I found something on FreeBSD bugzilla that can be related - here. I tried to add entry in /etc/default/rc.confg:

ifconfig_igb0="DHCP -rxcsum"

But there is not change in behavior.

Running tcpdump -i igb0 -vvv show only DHCP request sent no packet received.
If there is anything more I can do to debug this problem please let me know.

stephenw10

When you ping from a static IP are you also seeing packets leave but no replies?

I would definitely try disabling checksum offload. That should work fine with igb but with virtual igb NICs....

That needs to be added to the config usually via the GUI but if you can't access that you can edit the file directly /conf/config.xml.

In the <system> section at the top add:
<disablechecksumoffloading></disablechecksumoffloading>

Reboot to see that change.

Steve