Failing to add interfaces to dataplane.


  • Hello,

    I've installed TNSR in a new dell R440 chassi with broadcom internal cards.

    Installation went fine, but now I can't add interface to dataplane.

    Already tried the 3 available drivers, and show interfaces is always empty.

    How can I debug this?

    Thanks in advance.

    Best regards,
    Joao

  • LAYER 8 Netgate

    Broadcom NICs are not supported by dpdk. Your best bet is Intel or perhaps Mellanox NICs.

    https://docs.netgate.com/tnsr/en/latest/platforms/


  • Thanks for your reply.

    I´ve moved to a old R320 to try to continue testing since I can´t install a card yet in R440 (waiting for riser to come), but now I can´t even install. It doesn´t see the harddrive.
    The server controller is a PERC H310 mini, but I don´t find anything related to unsupported storage in docs.
    Do you´ve any clue about any storage controllers restrictions?

    Thanks again.

  • LAYER 8 Netgate

    @joaoa Not really. Anything that works in CentOS8 should work fine.


  • @Derelict
    Ok, will check if I can install CentOS8 on this server, if not, that explains all.

    Thanks again.

  • LAYER 8 Netgate

    The tnsr installer is a CentOS 8 installer that also installs the tnsr packages. But researching what does and does not work with CentOS 8 will get you more results than searching for what does and does not work for tnsr.


  • Hello again.

    So R320 raid card is not recognized even by centos.
    Confirming this, moved to VMWARE.

    Installed the machine, all good, until I want to add the card to dataplane, again.

    Machine is set with 2 cores 4G Ram, network cards are with VMXNET3 driver like stated here:

    "
    Virtual Machines
    A TNSR software Bare Metal Image is available and will run in most virtual x86 environments.

    VMware - vSphere/ESXi 6.0 and later, running VMXNET3 virtual network adapters.
    "

    machine is set as
    Guest OS: CentOS 8 (64-bit)
    Compatibility: ESXi 6.7 and later (VM version 14)
    VMware Tools: Not running, not installed

    In host OS, I´ve set ens224 to onboot=no.

    In clixon_cli, I try to set the dataplane, it accepts, but nothing is returned in show interfaces.

    Again, tried the 3 drivers, and setting the vfio-pci to pa, and nothing.

    What am I missing here?

    Please help.

  • LAYER 8 Netgate

    Please post exactly what you are doing and the output.


  • With default driver:

    labtnsr1 tnsr# conf
    labtnsr1 tnsr(config)# dataplane dpdk dev
      0000:0b:00.0           Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01) ( Active Interface ens192 )
      0000:13:00.0           Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
      default
    labtnsr1 tnsr(config)# dataplane dpdk dev
    labtnsr1 tnsr(config)# dataplane dpdk dev 0000:13:00.0 network
    labtnsr1 tnsr(config)# configuration candidate commit
    labtnsr1 tnsr(config)# service dataplane restart
    labtnsr1 tnsr(config)# exit
    labtnsr1 tnsr# show interface
    labtnsr1 tnsr#
    labtnsr1 tnsr#
    
    labtnsr1 tnsr# show configuration
    <dataplane-config xmlns="urn:netgate:xml:yang:netgate-dataplane">
       <dpdk>
          <dev>
             <id>0000:13:00.0</id>
             <device-type>network</device-type>
          </dev>
          <uio-driver>igb_uio</uio-driver>
       </dpdk>
    </dataplane-config>
    <nacm xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-acm">
       <enable-nacm>true</enable-nacm>
       <read-default>deny</read-default>
       <write-default>deny</write-default>
       <exec-default>deny</exec-default>
       <enable-external-groups>true</enable-external-groups>
       <groups>
          <group>
             <name>admin</name>
             <user-name>root</user-name>
             <user-name>tnsr</user-name>
          </group>
       </groups>
       <rule-list>
          <name>admin-rules</name>
          <group>admin</group>
          <rule>
             <name>permit-all</name>
             <module-name>*</module-name>
             <access-operations>*</access-operations>
             <action>permit</action>
          </rule>
       </rule-list>
    </nacm>
    labtnsr1 tnsr#
    

    trying vfio-pci

    labtnsr1 tnsr(config)# dataplane dpdk uio-driver vfio-pci
    labtnsr1 tnsr(config)# configuration candidate commit
    labtnsr1 tnsr(config)# service dataplane restart
    labtnsr1 tnsr(config)#
    labtnsr1 tnsr(config)#
    labtnsr1 tnsr(config)# exit
    labtnsr1 tnsr# show interface
    labtnsr1 tnsr# configure
    labtnsr1 tnsr(config)# dataplane dpdk iova-mode pa
    labtnsr1 tnsr(config)# configuration candidate commit
    labtnsr1 tnsr(config)# service dataplane restart
    labtnsr1 tnsr(config)#
    labtnsr1 tnsr(config)#
    labtnsr1 tnsr(config)# exit
    labtnsr1 tnsr# sh
     shell                 show
    labtnsr1 tnsr# show interface
    labtnsr1 tnsr#
    

    trying generic.

    labtnsr1 tnsr# conf t
    labtnsr1 tnsr(config)# dataplane dpdk uio-driver uio_pci_generic
    labtnsr1 tnsr(config)# configuration candidate commit
    labtnsr1 tnsr(config)# service dataplane restart
    labtnsr1 tnsr(config)# exit
    labtnsr1 tnsr# show interface
    labtnsr1 tnsr#
    labtnsr1 tnsr#
    labtnsr1 tnsr#
    

    in host os

    [toor@labtnsr1 ~]$ ip lin
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
        link/ether 00:50:56:ba:02:05 brd ff:ff:ff:ff:ff:ff
    [toor@labtnsr1 ~]$
    

    Thanks

  • LAYER 8 Netgate

    Be sure the host has:

    ONBOOT=no
    NM_CONTROLLED=no

    Are present in the host configuration files in /etc/sysconfig/network-scripts for the interfaces you want on the dataplane and reboot and try again.

    https://docs.netgate.com/tnsr/en/latest/setup/setup-host-interfaces.html


  • Hi, it was already when i took all that.

    fiule is like this:

    [toor@labtnsr1 ~]$ sudo cat /etc/sysconfig/network-scripts/ifcfg-ens224
    [sudo] password for toor:
    ONBOOT=no
    NM_CONTROLLED=no
    [toor@labtnsr1 ~]$
    
  • LAYER 8 Netgate

    How many vmxnet interfaces are you trying to assign to this? What version of ESXi? What VM version?


  • I´m only trying to assing this toi dataplane.

    The goal is to have has trunk and use vlans.

    regarding vmware:

    machine is set as
    Guest OS: CentOS 8 (64-bit)
    Compatibility: ESXi 6.7 and later (VM version 14)
    VMware Tools: Not running, not installed

  • LAYER 8 Netgate

    Try ESXi compatibility version 6.5. You will probably have to reinstall.

    I highly recommend a management interface on the host plus any dataplane interfaces you need.


  • ok, I'll try.

    The management interface is already set, is the ens192 in previous prints, it's how I'm reaching the machine now.

    I thought you were asking only about dataplane.


  • Hi,

    So, that does work, it was even set automatically to dataplane with the default driver.

    labtnsr1 tnsr# show configuration
    <dataplane-config xmlns="urn:netgate:xml:yang:netgate-dataplane">
       <dpdk>
          <uio-driver>igb_uio</uio-driver>
       </dpdk>
    </dataplane-config>
    <nacm xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-acm">
       <enable-nacm>true</enable-nacm>
       <read-default>deny</read-default>
       <write-default>deny</write-default>
       <exec-default>deny</exec-default>
       <enable-external-groups>true</enable-external-groups>
       <groups>
          <group>
             <name>admin</name>
             <user-name>root</user-name>
             <user-name>tnsr</user-name>
          </group>
       </groups>
       <rule-list>
          <name>admin-rules</name>
          <group>admin</group>
          <rule>
             <name>permit-all</name>
             <module-name>*</module-name>
             <access-operations>*</access-operations>
             <action>permit</action>
          </rule>
       </rule-list>
    </nacm>
    labtnsr1 tnsr# show interface
    Interface: GigabitEthernetb/0/0
        Admin status: down
        Link down, link-speed 10 Gbps, full duplex
        Link MTU: 9000 bytes
        MAC address: 00:50:56:ba:ed:31
        IPv4 MTU: 0 bytes
        IPv4 Route Table: ipv4-VRF:0
        IPv6 MTU: 0 bytes
        IPv6 Route Table: ipv6-VRF:0
        VLAN tag rewrite: disable
        Rx-queues
            queue-id 0 : cpu-id 1
        counters:
          received: 0 bytes, 0 packets, 0 errors
          transmitted: 0 bytes, 0 packets, 0 errors
          protocols: 0 IPv4, 0 IPv6
          0 drops, 0 punts, 0 rx miss, 0 rx no buffer
    
    labtnsr1 tnsr#
    

    Since there was a difference in the way I did the install, I retried with 6.7 compatibility again, and fails again.

    Basically, this time I started the installation already with both network cards set in vmware, and this wasn't the case in the previous problematic installation.

    But it's confirmed that was not this the issue, it failed again now with 6.7 and both cards available to the installer.

    So, I don't know if this is a bug or not, but for centOS8 based OS, would make sense to support 6.7 already.

    Thank you for your help, will continue the tests now.