[PARTIALLY SOLVED] wan/lan is 100baseTX on gbit nic



  • Hello,

    I have a virtualized pfsense with xen used only for the captiveportal. What's strange is that the nics in my xenserver are all gbit, but pfsense is using them in 100baseTX.

    Firstly I've change the cables and tested them. All right. Cat 5e with 4 meters.

    Xenserver ifconfig and ethtool command:

    # ifconfig
    eth0      Link encap:Ethernet  Endereço de HW 44:A8:42:39:CF:01  
              UP BROADCASTRUNNING MULTICAST  MTU:1500  Métrica:1
              RX packets:3149605921 errors:0 dropped:5 overruns:0 frame:0
              TX packets:3614802746 errors:0 dropped:0 overruns:0 carrier:0
              colisões:0 txqueuelen:1000 
              RX bytes:1510542831349 (1.3 TiB)  TX bytes:2903173185798 (2.6 TiB)
              IRQ:41 
    
    eth1      Link encap:Ethernet  Endereço de HW 44:A8:42:39:CF:02  
              UP BROADCASTRUNNING MULTICAST  MTU:1500  Métrica:1
              RX packets:543314975 errors:0 dropped:1 overruns:0 frame:0
              TX packets:787038642 errors:0 dropped:0 overruns:0 carrier:0
              colisões:0 txqueuelen:1000 
              RX bytes:143019364651 (133.1 GiB)  TX bytes:953993869726 (888.4 GiB)
              IRQ:45 
    
    eth2      Link encap:Ethernet  Endereço de HW 44:A8:42:39:CF:03  
              UP BROADCASTRUNNING  MTU:1500  Métrica:1
              RX packets:78873836 errors:0 dropped:0 overruns:0 frame:0
              TX packets:124480122 errors:0 dropped:0 overruns:0 carrier:0
              colisões:0 txqueuelen:1000 
              RX bytes:15020840900 (13.9 GiB)  TX bytes:159361978054 (148.4 GiB)
              IRQ:40 
    
    eth3      Link encap:Ethernet  Endereço de HW 44:A8:42:39:CF:04  
              UP BROADCASTRUNNING MULTICAST  MTU:1500  Métrica:1
              RX packets:302192890 errors:0 dropped:6 overruns:0 frame:0
              TX packets:59909174 errors:0 dropped:0 overruns:0 carrier:0
              colisões:0 txqueuelen:1000 
              RX bytes:110798832713 (103.1 GiB)  TX bytes:17436580065 (16.2 GiB)
              IRQ:44 
    
    
    # ethtool eth0
    Settings for eth0:
    	Supported ports: [ TP ]
    	Supported link modes:   10baseT/Half 10baseT/Full 
    	                        100baseT/Half 100baseT/Full 
    	                        1000baseT/Half 1000baseT/Full 
    	Supported pause frame use: No
    	Supports auto-negotiation: Yes
    	Advertised link modes:  10baseT/Half 10baseT/Full 
    	                        100baseT/Half 100baseT/Full 
    	                        1000baseT/Half 1000baseT/Full 
    	Advertised pause frame use: Symmetric
    	Advertised auto-negotiation: Yes
    	Speed: 1000Mb/s
    	Duplex: Full
    	Port: Twisted Pair
    	PHYAD: 1
    	Transceiver: internal
    	Auto-negotiation: on
    	MDI-X: Unknown
    	Supports Wake-on: g
    	Wake-on: d
    	Current message level: 0x000000ff (255)
    			       drv probe link timer ifdown ifup rx_err tx_err
    	Link detected: yes
    
    

    ifconfig from captiveportal vm on xen:

    [2.3.3-RELEASE][root@captiveportal]/root: ifconfig
    re0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether be:9b:aa:67:fc:e5
    	inet6 fe80::bc9b:aaff:fe67:fce5%re0 prefixlen 64 scopeid 0x1 
    	inet 200.x.x.x netmask 0xfffffe00 broadcast 200.x.x.x 
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (100baseTX <full-duplex>)
    	status: active
    re1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether 8a:73:46:d0:e7:bf
    	inet6 fe80::8873:46ff:fed0:e7bf%re1 prefixlen 64 scopeid 0x2 
    	inet 10.10.8.1 netmask 0xfffffc00 broadcast 10.10.11.255 
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (100baseTX <full-duplex>)
    	status: active</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,simplex,multicast> 
    

    dmesg:

    dmesg
    Copyright (c) 1992-2016 The FreeBSD Project.
    Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    	The Regents of the University of California. All rights reserved.
    FreeBSD is a registered trademark of The FreeBSD Foundation.
    FreeBSD 10.3-RELEASE-p17 #10 6da131e75c7(RELENG_2_3_3): Wed Mar  8 14:24:33 CST 2017
        root@ce23-amd64-builder:/builder/pfsense-233/tmp/obj/builder/pfsense-233/tmp/FreeBSD-src/sys/pfSense amd64
    FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
    XEN: Hypervisor version 4.4 detected.
    CPU: Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz (2600.05-MHz K8-class CPU)
      Origin="GenuineIntel"  Id=0x306f2  Family=0x6  Model=0x3f  Stepping=2
      Features=0x1783fbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,mmx,fxsr,sse,sse2,htt>Features2=0xe3fa3203 <sse3,pclmulqdq,ssse3,fma,cx16,pcid,sse4.1,sse4.2,x2apic,movbe,popcnt,tscdlt,aesni,f16c,rdrand,hv>AMD Features=0x28100800 <syscall,nx,rdtscp,lm>AMD Features2=0x21 <lahf,abm>Structured Extended Features=0x72a <tscadj,bmi1,avx2,bmi2,erms,invpcid>Hypervisor: Origin = "Microsoft Hv"
    real memory  = 4290772992 (4092 MB)
    avail memory = 4107960320 (3917 MB)
    Event timer "LAPIC" quality 400
    ACPI APIC Table: <xen hvm="">FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
    FreeBSD/SMP: 2 package(s) x 2 core(s)
     cpu0 (BSP): APIC ID:  0
     cpu1 (AP): APIC ID:  2
     cpu2 (AP): APIC ID:  4
     cpu3 (AP): APIC ID:  6
    random: <software, yarrow="">initialized
    ioapic0: Changing APIC ID to 1
    MADT: Forcing active-low polarity and level trigger for SCI
    ioapic0 <version 1.1="">irqs 0-47 on motherboard
    wlan: mac acl policy registered
    ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
    ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
    module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff80620930, 0) error 1
    ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
    ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
    module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff806209e0, 0) error 1
    ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
    ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
    module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff80620a90, 0) error 1
    iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi/.
    iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
    module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80647c30, 0) error 1
    iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi/.
    iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
    module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff80647ce0, 0) error 1
    iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi/.
    iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
    module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80647d90, 0) error 1
    netmap: loaded module
    kbd1 at kbdmux0
    xen_et0: <xen pv="" clock="">on motherboard
    Event timer "XENTIMER" frequency 1000000000 Hz quality 950
    Timecounter "XENTIMER" frequency 1000000000 Hz quality 950
    cryptosoft0: <software crypto="">on motherboard
    padlock0: No ACE support.
    acpi0: <xen>on motherboard
    acpi0: Power Button (fixed)
    acpi0: Sleep Button (fixed)
    cpu0: <acpi cpu="">on acpi0
    cpu1: <acpi cpu="">on acpi0
    cpu2: <acpi cpu="">on acpi0
    cpu3: <acpi cpu="">on acpi0
    hpet0: <high precision="" event="" timer="">iomem 0xfed00000-0xfed003ff on acpi0
    Timecounter "HPET" frequency 62500000 Hz quality 950
    attimer0: <at timer="">port 0x40-0x43 irq 0 on acpi0
    Timecounter "i8254" frequency 1193182 Hz quality 0
    Event timer "i8254" frequency 1193182 Hz quality 100
    atrtc0: <at realtime="" clock="">port 0x70-0x71 irq 8 on acpi0
    Event timer "RTC" frequency 32768 Hz quality 0
    Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
    acpi_timer0: <32-bit timer at 3.579545MHz> port 0xb008-0xb00b on acpi0
    pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0
    pci0: <acpi pci="" bus="">on pcib0
    isab0: <pci-isa bridge="">at device 1.0 on pci0
    isa0: <isa bus="">on isab0
    atapci0: <intel piix3="" wdma2="" controller="">port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc320-0xc32f at device 1.1 on pci0
    ata0: <ata channel="">at channel 0 on atapci0
    ata1: <ata channel="">at channel 1 on atapci0
    uhci0: <intel 82371sb="" (piix3)="" usb="" controller="">port 0xc300-0xc31f irq 23 at device 1.2 on pci0
    usbus0: controller did not stop
    usbus0 on uhci0
    pci0: <bridge>at device 1.3 (no driver attached)
    vgapci0: <vga-compatible display="">mem 0xf0000000-0xf1ffffff,0xf3000000-0xf3000fff irq 24 at device 2.0 on pci0
    vgapci0: Boot video device
    xenpci0: <xen platform="" device="">port 0xc000-0xc0ff mem 0xf2000000-0xf2ffffff irq 30 at device 3.0 on pci0
    xenstore0: <xenstore>on xenpci0
    re0: <realtek 10="" 8139c+="" 100basetx="">port 0xc100-0xc1ff mem 0xf3001000-0xf30010ff irq 32 at device 4.0 on pci0
    re0: Chip rev. 0x74800000
    re0: MAC rev. 0x00000000
    miibus0: <mii bus="">on re0
    rlphy0: <realtek internal="" media="" interface="">PHY 0 on miibus0
    rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow
    re0: Using defaults for TSO: 65518/35/2048
    re0: Ethernet address: be:9b:aa:67:fc:e5
    re0: netmap queues/slots: TX 1/64, RX 1/64
    re1: <realtek 10="" 8139c+="" 100basetx="">port 0xc200-0xc2ff mem 0xf3001100-0xf30011ff irq 36 at device 5.0 on pci0
    re1: Chip rev. 0x74800000
    re1: MAC rev. 0x00000000
    miibus1: <mii bus="">on re1
    rlphy1: <realtek internal="" media="" interface="">PHY 0 on miibus1
    rlphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow
    re1: Using defaults for TSO: 65518/35/2048
    re1: Ethernet address: 8a:73:46:d0:e7:bf
    re1: netmap queues/slots: TX 1/64, RX 1/64
    atkbdc0: <keyboard controller="" (i8042)="">port 0x60,0x64 irq 1 on acpi0
    atkbd0: <at keyboard="">irq 1 on atkbdc0
    kbd0 at atkbd0
    atkbd0: [GIANT-LOCKED]
    psm0: <ps 2="" mouse="">irq 12 on atkbdc0
    psm0: [GIANT-LOCKED]
    psm0: model IntelliMouse Explorer, device ID 4
    fdc0: <floppy drive="" controller="">port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
    fdc0: does not respond
    device_attach: fdc0 attach returned 6
    uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
    ppc0: <parallel port="">port 0x378-0x37f irq 7 on acpi0
    ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
    ppbus0: <parallel port="" bus="">on ppc0
    lpt0: <printer>on ppbus0
    lpt0: Interrupt-driven port
    ppi0: <parallel i="" o="">on ppbus0
    sc0: <system console="">at flags 0x100 on isa0
    sc0: VGA <16 virtual consoles, flags=0x300>
    vga0: <generic isa="" vga="">at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
    fdc0: No FDOUT register!
    Timecounters tick every 1.000 msec
    random: unblocking device.
    usbus0: 12Mbps Full Speed USB v1.0
    xenbusb_front0: <xen frontend="" devices="">on xenstore0
    ugen0.1: <intel>at usbus0
    uhub0: <intel 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usbus0
    cd0 at ata1 bus 0 scbus1 target 1 lun 0
    cd0: <qemu qemu="" dvd-rom="" 0.10="">Removable CD-ROM SCSI device
    cd0: Serial Number QM00004
    cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
    cd0: Attempt to query device size failed: NOT READY, Medium not present
    xenbusb_back0: <xen backend="" devices="">on xenstore0
    xctrl0: <xen control="" device="">on xenstore0
    xbd0: 15360MB <virtual block="" device="">at device/vbd/768 on xenbusb_front0
    xbd0: attaching as ada0
    xbd0: features: write_barrier
    xbd0: synchronize cache commands enabled.
    SMP: AP CPU #2 Launched!
    SMP: AP CPU #1 Launched!
    SMP: AP CPU #3 Launched!
    uhub0: 2 ports with 2 removable, self powered
    Root mount waiting for: usbus0
    Root mount waiting for: usbus0
    ugen0.2: <qemu 0.10.2="">at usbus0
    Trying to mount root from ufs:/dev/ufsid/5732257f4a8dc856 [rw]...
    re0: link state changed to UP
    re1: link state changed to UP
    pflog0: promiscuous mode enabled
    ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to accept, logging disabled
    DUMMYNET 0 with IPv6 initialized (100409)
    load_dn_sched dn_sched FIFO loaded
    load_dn_sched dn_sched QFQ loaded
    load_dn_sched dn_sched RR loaded
    load_dn_sched dn_sched WF2Q+ loaded
    load_dn_sched dn_sched PRIO loaded</qemu></virtual></xen></xen></qemu></intel></intel></xen></generic></system></parallel></printer></parallel></parallel></floppy></ps></at></keyboard></realtek></mii></realtek></realtek></mii></realtek></xenstore></xen></vga-compatible></bridge></intel></ata></ata></intel></isa></pci-isa></acpi></acpi></at></at></high></acpi></acpi></acpi></acpi></xen></software></xen></version></software,></xen></tscadj,bmi1,avx2,bmi2,erms,invpcid></lahf,abm></syscall,nx,rdtscp,lm></sse3,pclmulqdq,ssse3,fma,cx16,pcid,sse4.1,sse4.2,x2apic,movbe,popcnt,tscdlt,aesni,f16c,rdrand,hv></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,mmx,fxsr,sse,sse2,htt> 
    

    If I try to manually change the media option, but the error below occurs.

    ifconfig re1 media 1000baseTX
    ifconfig: SIOCSIFMEDIA (media): Device not configured
    
    

    Any sugestions or this is supposed to be the way it is? :o



    1. You appear to have enabled an option that disables the PV NICs.

    I know that hw.xen.disable_pv_nics=1 on /boot/loader.conf.local could be one of the reasons but there may be other ways to achieve the same thing. So you'll need to undo whatever is causing the disablement of the PV NICs.

    1. If you manage to get around #1 and see the xn NICs, you need to disable hardware checksum offloading on the VM VIF(s) via custom hypervisor settings: https://forum.pfsense.org/index.php?topic=88467.msg512128#msg512128

    2. Probably not really required, but I do it: XenTools: https://forum.pfsense.org/index.php?topic=109253.msg608562#msg608562



    1. You appear to have enabled an option that disables the PV NICs.

    Yes.

    1. If you manage to get around #1 and see the xn NICs, you need to disable hardware checksum offloading on the VM VIF(s) via custom hypervisor settings: https://forum.pfsense.org/index.php?topic=88467.msg512128#msg512128

    I had already been configured that way.

    1. Probably not really required, but I do it: XenTools: https://forum.pfsense.org/index.php?topic=109253.msg608562#msg608562

    I have xentools installed.

    So maybe it's Option 1 that's causing it. Just disable this option because we were having problems accessing our webservers internally.

    I will enable it again and perform some tests.

    Thank you very much for your help JorgeOliveira! :D



  • If you still face this issue after removing hw.xen.disable_pv_nics=1 from pfSense, my personal recommendation is for you to disable hardware checksum offloading on the internal webservers as well as pfSense. See the information in 2).

    Also be reminded that after changing this setting you need to shutdown the VMs and then start again. A reboot will not work, it needs to be a cold start.



  • Now it is appearing as "manual".  :o

    # ifconfig
    xn0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	ether be:9b:aa:67:fc:e5
    	inet6 fe80::bc9b:aaff:fe67:fce5%xn0 prefixlen 64 scopeid 0x5 
    	inet 200.x.x.x netmask 0xfffffe00 broadcast 200.x.x.x 
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet manual
    	status: active
    xn1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	ether 8a:73:46:d0:e7:bf
    	inet6 fe80::8873:46ff:fed0:e7bf%xn1 prefixlen 64 scopeid 0x6 
    	inet 10.10.8.1 netmask 0xfffffc00 broadcast 10.10.11.255 
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet manual
    	status: active</performnud,auto_linklocal></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></up,broadcast,running,simplex,multicast> 
    

    I tried again manually configuring for 1000baseTX, but the error persists.

    # ifconfig xn1 media 1000baseTX mediaopt full-duplex
    ifconfig: SIOCSIFMEDIA (media): Device not configured
    
    






  • I think you don't need to configure those interfaces… They'll work as is to their maximum speed.



  • I made some tests with Iperf and at first seems to be like gbit nic.

    Thanks for your help JorgeOliveira! :D


Log in to reply