TNSR Home+Lab ISO on KVM with Mellanox ConnectX-4 passthrough


  • Re: TNSR HomeLab ISO on KVM?

    Useful message for those tying to get Version: tnsr-v20.10.1-2
    working under KVM using SR-IOV with Mellanox Technologies MT27710 Family working with DPDK.
    The solution is to install Mellannox OFED_EN in host and most important also in KVM TNSR machine with the following commands:

    shell
    sudo -i
    ifconfig ens7 up # manually configure internet through VLAN
    ip link add link ens7 name ens7.100 type vlan id 100
    ifconfig ens7.100 192.168.5.109
    route add default gw 192.168.5.1
    echo nameserver 192.168.5.1 > /etc/resolv.conf
    yum update
    yum install wget
    wget http://www.mellanox.com/downloads/ofed/MLNX_EN-5.1-1.0.4.0/mlnx-en-5.1-1.0.4.0-rhel8.2-x86_64.iso
    mount -o loop mlnx-en-5.1-1.0.4.0-rhel8.2-x86_64.iso /mnt/
    cd /mnt
    yum search perl
    yum search perl-term
    yum install perl-Term-ANSIColor
    yum install lsof
    ./install --help
    ./install --dpdk
    reboot
    

    Although TNSR shows ConnectX-4 as Active interface DPDK configuration is now possible!

    TNSR tnsr(config)# dataplane dpdk dev ?
      0000:00:03.0           Ethernet controller: Red Hat, Inc. Virtio network device
      0000:00:07.0           Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] ( Active Interface ens7 )
      0000:00:08.0           Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] ( Active Interface ens8 )
      default
    

    The TNSR KVM instructions are quite outdated and not very useful for getting 10Gbit performance config. Therefore I am giving sample TNSR machine creation command used for my dual homed passthrough KVM:

    virt-install --name TNSR --vcpus=2,sockets=1,cores=2,threads=1 \
    --os-type linux --os-variant centos8 --cpu host-passthrough --ram 4096 \
    --noautoconsole --graphics vnc \
    --cdrom=/var/lib/libvirt/boot/TNSR_HomeLab_CentOSx86_64-20.10.1-2.iso \
    --disk path=/var/lib/libvirt/images/tnsr.qcow2,size=5,bus=virtio,format=qcow2 \
    --host-device=pci_0000_17_00_0 --host-device=pci_0000_17_00_1