Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Pfsesen iGB driver support - aka my box keeps crashing/network hangs

    2.0-RC Snapshot Feedback and Problems - RETIRED
    4
    18
    6443
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      richc last edited by

      Hi,

      Hopefully someone on here can help, I am attempting to install pfsense 2 BETA5 on a Quad core box, that has 2 x Intel Quad port 1GB NICs.

      However when I try7 and configure a interface the console just pumps out the message:

      igbX: Could not setup receive structures.

      I have done a bit of digging around and it looks to be a problem with the number of cores and the igb driver, however does anyone know any work arounds to get get this working?

      I saw some comments on tweaking the driver .c file, however as its on a pfsense box I am not sure if that's a good idea.

      1 Reply Last reply Reply Quote 0
      • E
        eri-- last edited by

        Look here http://redmine.pfsense.org/issues/1221

        Just put that tunable in loader.conf.local with value 4 or even 2

        1 Reply Last reply Reply Quote 0
        • R
          richc last edited by

          So just to confirm this, the fix is; Install Uniprocessor kernel, disable hyper threading and maxing the mbuf size?

          So a couple of questions;

          1. I take it I need to reinstall for the uniprocessor kernel?

          2. I need to disabled hyperthreading in the BIOS?

          3. I understand I need to edit /boot/loader.conf.local, is this the correct entry to add?

          hw.igb.num_queues=4

          1 Reply Last reply Reply Quote 0
          • E
            eri-- last edited by

            Just this.
            hw.igb.num_queues=4

            1 Reply Last reply Reply Quote 0
            • R
              richc last edited by

              Thanks ermal, I will give it ago now then

              1 Reply Last reply Reply Quote 0
              • B
                Brutal last edited by

                richc - have you had any success?

                I have the same issue.  The LiveCD sees the interfaces on the Quad Nic with the "receive structures" error.  After installing to the Hard disk, it doesn't see the the interfaces at all.  After install, I add the "loader.conf.local" file with "hw.igb.num_queues=4" and nothing changes.

                Question for me is why does the LiveCD handle it different?

                1 Reply Last reply Reply Quote 0
                • W
                  wallabybob last edited by

                  What snapshot build? (It MIGHT make a difference.)

                  The difference in behaviour MIGHT be that the Live CD kernel is a uniprocessor kernel while you have installed a SMP kernel. (There was some resource allocation problem previously reported for igb devices that got worse with increasing numbers of devices and increasing numbers of CPUs.) I this explains what you are seeing then  would expect the first boot after install to fail to attach some or all of the igbs (because the updated loader.conf.local wasn't read at boot time because you booted before updating the file) but they should all (or mostly) be seen on subsequent boots (because the updated loader.conf.local file was read at boot time).

                  If things don't come good on a reboot after updating the loader.conf.local file (it is /boot/loader.conf.local isn't it?) then please post the output of the pfSense shell command dmesg here so I can get some more clues as to what is going on.

                  1 Reply Last reply Reply Quote 0
                  • B
                    Brutal last edited by

                    Thank you Wall….

                    I wonder if Richc ever got his going.

                    On mine, I disabled Hyperthreading
                    boot the LiveCD  I was on the Feb 5th load.  Today I'm on the RC1 load (just downloaded). Same issue
                    LiveCD detects the quad card (I340-T4) and gives the "Receive Structures" error.  But it sees the ports.
                    I added the hw.igb.num_queues=2 to new loader.conf.local file
                    installed to hard disk with Uniprocessor load
                    after boot to hard disk, the igb driver doesn't get attached.

                    This makes me think there is more differences to the livecd vs hd install.

                    cpu0: <acpi cpu="">on acpi0
                    cpu1: <acpi cpu="">on acpi0
                    cpu2: <acpi cpu="">on acpi0
                    cpu3: <acpi cpu="">on acpi0
                    pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0
                    pci0: <acpi pci="" bus="">on pcib0
                    pcib1: <acpi pci-pci="" bridge="">irq 16 at device 3.0 on pci0
                    pci26: <acpi pci="" bus="">on pcib1
                    pcib2: <acpi pci-pci="" bridge="">irq 16 at device 5.0 on pci0
                    pci31: <acpi pci="" bus="">on pcib2
                    pci0: <base peripheral=""> at device 8.0 (no driver attached)
                    pci0: <base peripheral=""> at device 8.1 (no driver attached)
                    pci0: <base peripheral=""> at device 8.2 (no driver attached)
                    pci0: <base peripheral=""> at device 8.3 (no driver attached)
                    pci0: <base peripheral=""> at device 16.0 (no driver attached)
                    pci0: <base peripheral=""> at device 16.1 (no driver attached)

                    Why would the driver not be loaded with both LiveCD and HD install.
                    I'm not very familiar with Freebsd.  How would I check for the driver.  If there, do I just modprobe it?</acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi>

                    1 Reply Last reply Reply Quote 0
                    • B
                      Brutal last edited by

                      Here's the dmesg snippet from the LiveCD:

                      cpu0: <acpi cpu="">on acpi0
                      cpu1: <acpi cpu="">on acpi0
                      cpu2: <acpi cpu="">on acpi0
                      cpu3: <acpi cpu="">on acpi0
                      pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0
                      pci0: <acpi pci="" bus="">on pcib0
                      pcib1: <acpi pci-pci="" bridge="">irq 16 at device 3.0 on pci0
                      pci26: <acpi pci="" bus="">on pcib1
                      pcib2: <acpi pci-pci="" bridge="">irq 16 at device 5.0 on pci0
                      pci31: <acpi pci="" bus="">on pcib2
                      igb0: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91d80000-0x91dfffff,0x91e
                      0c000-0x91e0ffff irq 16 at device 0.0 on pci31
                      igb0: Using MSIX interrupts with 5 vectors
                      igb0: [ITHREAD]
                      igb0: [ITHREAD]
                      igb0: [ITHREAD]
                      igb0: [ITHREAD]
                      igb0: [ITHREAD]
                      igb1: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91d00000-0x91d7ffff,0x91e08000-0x91e0bfff irq 17 at device 0.1 on pci31
                      igb1: Using MSIX interrupts with 5 vectors
                      igb1: [ITHREAD]
                      igb1: [ITHREAD]
                      igb1: [ITHREAD]
                      igb1: [ITHREAD]
                      igb1: [ITHREAD]
                      igb2: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91c80000-0x91cfffff,0x91e04000-0x91e07fff irq 18 at device 0.2 on pci31
                      igb2: Using MSIX interrupts with 5 vectors
                      igb2: [ITHREAD]
                      igb2: [ITHREAD]
                      igb2: [ITHREAD]
                      igb2: [ITHREAD]
                      igb2: [ITHREAD]
                      igb3: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91c00000-0x91c7ffff,0x91e00000-0x91e03fff irq 19 at device 0.3 on pci31
                      igb3: Using MSIX interrupts with 5 vectors
                      igb3: [ITHREAD]
                      :</intel(r)></intel(r)></intel(r)></intel(r)></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi>

                      1 Reply Last reply Reply Quote 0
                      • W
                        wallabybob last edited by

                        Please post the full dmesg output from the HD boot. The snippet doesn't show what I'm looking for. In particular, comparing the two snippets:

                        @Brutal:

                        pci0: <base peripheral=""> at device 8.0 (no driver attached)
                        pci0: <base peripheral=""> at device 8.1 (no driver attached)
                        pci0: <base peripheral=""> at device 8.2 (no driver attached)
                        pci0: <base peripheral=""> at device 8.3 (no driver attached)
                        pci0: <base peripheral=""> at device 16.0 (no driver attached)
                        pci0: <base peripheral=""> at device 16.1 (no driver attached)

                        This shows device numbers 8 and 16 on pci bus 0 which doesn't match any of
                        @Brutal:

                        igb0: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91d80000-0x91dfffff,0x91e
                        0c000-0x91e0ffff irq 16 at device 0.0 on pci31

                        igb1: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91d00000-0x91d7ffff,0x91e08000-0x91e0bfff irq 17 at device 0.1 on pci31

                        igb2: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91c80000-0x91cfffff,0x91e04000-0x91e07fff irq 18 at device 0.2 on pci31

                        igb3: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91c00000-0x91c7ffff,0x91e00000-0x91e03fff irq 19 at device 0.3 on pci31</intel(r)></intel(r)></intel(r)></intel(r)>

                        which are all device number 0 on pci bus 31.

                        @Brutal:

                        How would I check for the driver.  If there, do I just modprobe it?

                        pfSense normally includes all the NIC drivers in the built kernel, rather than as separate loadable modules.

                        1 Reply Last reply Reply Quote 0
                        • B
                          Brutal last edited by

                          LiveCD full dmesg.  HD boot dmesg to follow

                          Copyright © 1992-2010 The FreeBSD Project.
                          Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
                                  The Regents of the University of California. All rights reserved.
                          FreeBSD is a registered trademark of The FreeBSD Foundation.
                          FreeBSD 8.1-RELEASE-p2 #0: Thu Feb 10 20:45:27 EST 2011
                              sullrich@FreeBSD_8.0_pfSense_2.0-snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_SMP.8 i386
                          Timecounter "i8254" frequency 1193182 Hz quality 0
                          CPU: Intel(R) Xeon(R) CPU          X3450  @ 2.67GHz (2660.00-MHz 686-class CPU)
                            Origin = "GenuineIntel"  Id = 0x106e5  Family = 6  Model = 1e  Stepping = 5
                            Features=0xbfebfbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>Features2=0x98e3fd <sse3,dtes64,mon,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,sse4.1,sse4.2,popcnt>AMD Features=0x28100000 <nx,rdtscp,lm>AMD Features2=0x1 <lahf>TSC: P-state invariant
                          real memory  = 4294967296 (4096 MB)
                          avail memory = 2072043520 (1976 MB)
                          ACPI APIC Table: <ibm   ="" foxhollw="">FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
                          FreeBSD/SMP: 1 package(s) x 4 core(s)
                          cpu0 (BSP): APIC ID:  0
                          cpu1 (AP): APIC ID:  2
                          cpu2 (AP): APIC ID:  4
                          cpu3 (AP): APIC ID:  6
                          ACPI Warning: Invalid length for Pm1aControlBlock: 32, using default 16 (20100331/tbfadt-707)
                          ioapic0 <version 2.0="">irqs 0-23 on motherboard
                          wpi: You need to read the LICENSE file in /usr/share/doc/legal/intel_wpi/.
                          wpi: If you agree with the license, set legal.intel_wpi.license_ack=1 in /boot/loader.conf.
                          module_register_init: MOD_LOAD (wpi_fw, 0xc095c280, 0) error 1
                          ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
                          ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
                          module_register_init: MOD_LOAD (ipw_bss_fw, 0xc07672e0, 0) error 1
                          ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
                          ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
                          module_register_init: MOD_LOAD (ipw_ibss_fw, 0xc07673a0, 0) error 1
                          ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
                          ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
                          module_register_init: MOD_LOAD (ipw_monitor_fw, 0xc0767460, 0) error 1
                          wlan: mac acl policy registered
                          kbd1 at kbdmux0
                          cryptosoft0: <software crypto="">on motherboard
                          padlock0: No ACE support.
                          acpi0: <ibm foxhollw="">on motherboard
                          acpi0: [ITHREAD]
                          acpi0: Power Button (fixed)
                          Timecounter "HPET" frequency 14318180 Hz quality 900
                          Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
                          acpi_timer0: <24-bit timer at 3.579545MHz> port 0x588-0x58b on acpi0
                          cpu0: <acpi cpu="">on acpi0
                          cpu1: <acpi cpu="">on acpi0
                          cpu2: <acpi cpu="">on acpi0
                          cpu3: <acpi cpu="">on acpi0
                          pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0
                          pci0: <acpi pci="" bus="">on pcib0
                          pcib1: <acpi pci-pci="" bridge="">irq 16 at device 3.0 on pci0
                          pci26: <acpi pci="" bus="">on pcib1
                          pcib2: <acpi pci-pci="" bridge="">irq 16 at device 5.0 on pci0
                          pci31: <acpi pci="" bus="">on pcib2
                          igb0: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91d80000-0x91dfffff,0x91e0c000-0x91e0ffff irq 16 at device 0.0 on pci31
                          igb0: Using MSIX interrupts with 5 vectors
                          igb0: [ITHREAD]
                          igb0: [ITHREAD]
                          igb0: [ITHREAD]
                          igb0: [ITHREAD]
                          igb0: [ITHREAD]
                          igb1: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91d00000-0x91d7ffff,0x91e08000-0x91e0bfff irq 17 at device 0.1 on pci31
                          igb1: Using MSIX interrupts with 5 vectors
                          igb1: [ITHREAD]
                          igb1: [ITHREAD]
                          igb1: [ITHREAD]
                          igb1: [ITHREAD]
                          igb1: [ITHREAD]
                          igb2: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91c80000-0x91cfffff,0x91e04000-0x91e07fff irq 18 at device 0.2 on pci31
                          igb2: Using MSIX interrupts with 5 vectors
                          igb2: [ITHREAD]
                          igb2: [ITHREAD]
                          igb2: [ITHREAD]
                          igb2: [ITHREAD]
                          igb2: [ITHREAD]
                          igb3: <intel(r) 1000="" pro="" network="" connection="" version="" -="" 2.0.7="">mem 0x91c00000-0x91c7ffff,0x91e00000-0x91e03fff irq 19 at device 0.3 on pci31
                          igb3: Using MSIX interrupts with 5 vectors
                          igb3: [ITHREAD]
                          igb3: [ITHREAD]
                          igb3: [ITHREAD]
                          igb3: [ITHREAD]
                          igb3: [ITHREAD]
                          pci0: <base peripheral=""> at device 8.0 (no driver attached)
                          pci0: <base peripheral=""> at device 8.1 (no driver attached)
                          pci0: <base peripheral=""> at device 8.2 (no driver attached)
                          pci0: <base peripheral=""> at device 8.3 (no driver attached)
                          pci0: <base peripheral=""> at device 16.0 (no driver attached)
                          pci0: <base peripheral=""> at device 16.1 (no driver attached)
                          ehci0: <intel pch="" usb="" 2.0="" controller="" usb-b="">mem 0x91f01000-0x91f013ff irq 19 at device 26.0 on pci0
                          ehci0: [ITHREAD]
                          usbus0: EHCI version 1.0
                          usbus0: <intel pch="" usb="" 2.0="" controller="" usb-b="">on ehci0
                          pcib3: <acpi pci-pci="" bridge="">irq 17 at device 28.0 on pci0
                          pci1: <acpi pci="" bus="">on pcib3
                          mpt0: <lsilogic sas="" sata="" adapter="">port 0x3000-0x30ff mem 0x91b10000-0x91b13fff,0x91b00000-0x91b0ffff irq 16 at device 0.0 on pci1
                          mpt0: [ITHREAD]
                          mpt0: MPI Version=1.5.20.0
                          mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 )
                          mpt0: 0 Active Volumes (2 Max)
                          mpt0: 0 Hidden Drive Members (14 Max)
                          pcib4: <acpi pci-pci="" bridge="">irq 17 at device 28.4 on pci0
                          pci6: <acpi pci="" bus="">on pcib4
                          pcib5: <pci-pci bridge="">irq 16 at device 0.0 on pci6
                          pci7: <pci bus="">on pcib5
                          vgapci0: <vga-compatible display="">mem 0x90000000-0x90ffffff,0x91800000-0x91803fff,0x91000000-0x917fffff at device 0.0 on pci7
                          pcib6: <acpi pci-pci="" bridge="">irq 16 at device 28.5 on pci0
                          pci11: <acpi pci="" bus="">on pcib6
                          em0: <intel(r) 1000="" pro="" network="" connection="" 7.1.9="">port 0x2000-0x201f mem 0x91a80000-0x91a9ffff,0x91a00000-0x91a7ffff,0x91aa0000-0x91aa3fff irq 17 at device 0.0 on pci11
                          em0: Using MSIX interrupts with 3 vectors
                          em0: [ITHREAD]
                          em0: [ITHREAD]
                          em0: [ITHREAD]
                          pcib7: <acpi pci-pci="" bridge="">irq 18 at device 28.6 on pci0
                          pci16: <acpi pci="" bus="">on pcib7
                          pcib8: <acpi pci-pci="" bridge="">irq 19 at device 28.7 on pci0
                          pci21: <acpi pci="" bus="">on pcib8
                          em1: <intel(r) 1000="" pro="" network="" connection="" 7.1.9="">port 0x1000-0x101f mem 0x91980000-0x9199ffff,0x91900000-0x9197ffff,0x919a0000-0x919a3fff irq 19 at device 0.0 on pci21
                          em1: Using MSIX interrupts with 3 vectors
                          em1: [ITHREAD]
                          em1: [ITHREAD]
                          em1: [ITHREAD]
                          ehci1: <intel pch="" usb="" 2.0="" controller="" usb-a="">mem 0x91f00000-0x91f003ff irq 19 at device 29.0 on pci0
                          ehci1: [ITHREAD]
                          usbus1: EHCI version 1.0
                          usbus1: <intel pch="" usb="" 2.0="" controller="" usb-a="">on ehci1
                          pcib9: <acpi pci-pci="" bridge="">at device 30.0 on pci0
                          pci36: <acpi pci="" bus="">on pcib9
                          isab0: <pci-isa bridge="">at device 31.0 on pci0
                          isa0: <isa bus="">on isab0
                          atapci0: <intel pch="" sata300="" controller="">port 0x4078-0x407f,0x408c-0x408f,0x4070-0x4077,0x4088-0x408b,0x4050-0x405f,0x4040-0x404f irq 19 at device 31.2 on pci0
                          atapci0: [ITHREAD]
                          ata2: <ata 0="" channel="">on atapci0
                          ata2: [ITHREAD]
                          ata3: <ata 1="" channel="">on atapci0
                          ata3: [ITHREAD]
                          pci0: <serial bus,="" smbus="">at device 31.3 (no driver attached)
                          atapci1: <intel pch="" sata300="" controller="">port 0x4068-0x406f,0x4084-0x4087,0x4060-0x4067,0x4080-0x4083,0x4030-0x403f,0x4020-0x402f irq 18 at device 31.5 on pci0
                          atapci1: [ITHREAD]
                          ata4: <ata 0="" channel="">on atapci1
                          ata4: [ITHREAD]
                          ata5: <ata 1="" channel="">on atapci1
                          ata5: [ITHREAD]
                          acpi_button0: <power button="">on acpi0
                          acpi_button0: enable wake failed
                          atrtc0: <at realtime="" clock="">port 0x70-0x77 irq 8 on acpi0
                          atrtc0: Warning: Couldn't map I/O.
                          uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
                          uart0: [FILTER]
                          uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
                          uart1: [FILTER]
                          pmtimer0 on isa0
                          orm0: <isa option="" roms="">at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff pnpid ORM0000 on isa0
                          sc0: <system console="">at flags 0x100 on isa0
                          sc0: VGA <16 virtual consoles, flags=0x300>
                          vga0: <generic isa="" vga="">at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
                          ata0 at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
                          ata0: [ITHREAD]
                          ata1 at port 0x170-0x177,0x376 irq 15 on isa0
                          ata1: [ITHREAD]
                          atkbdc0: <keyboard controller="" (i8042)="">at port 0x60,0x64 on isa0
                          atkbd0: <at keyboard="">irq 1 on atkbdc0
                          kbd0 at atkbd0
                          atkbd0: [GIANT-LOCKED]
                          atkbd0: [ITHREAD]
                          ppc0: parallel port not found.
                          est0: <enhanced speedstep="" frequency="" control="">on cpu0
                          p4tcc0: <cpu frequency="" thermal="" control="">on cpu0
                          est1: <enhanced speedstep="" frequency="" control="">on cpu1
                          p4tcc1: <cpu frequency="" thermal="" control="">on cpu1
                          est2: <enhanced speedstep="" frequency="" control="">on cpu2
                          p4tcc2: <cpu frequency="" thermal="" control="">on cpu2
                          est3: <enhanced speedstep="" frequency="" control="">on cpu3
                          p4tcc3: <cpu frequency="" thermal="" control="">on cpu3
                          RTC BIOS diagnostic error 80 <clock_battery>Timecounters tick every 1.000 msec
                          IPsec: Initialized Security Association Processing.
                          usbus0: 480Mbps High Speed USB v2.0
                          usbus1: 480Mbps High Speed USB v2.0
                          ugen0.1: <intel>at usbus0
                          uhub0: <intel 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usbus0
                          ugen1.1: <intel>at usbus1
                          uhub1: <intel 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usbus1
                          uhub0: 2 ports with 2 removable, self powered
                          uhub1: 2 ports with 2 removable, self powered
                          ugen0.2: <vendor 0x8087="">at usbus0
                          uhub2: <vendor 2="" 9="" 0x8087="" product="" 0x0020,="" class="" 0,="" rev="" 2.00="" 0.00,="" addr="">on usbus0
                          ugen1.2: <vendor 0x8087="">at usbus1
                          uhub3: <vendor 2="" 9="" 0x8087="" product="" 0x0020,="" class="" 0,="" rev="" 2.00="" 0.00,="" addr="">on usbus1
                          acd0: DVDR <tsstcorp cddvdw="" ts-l633b="" ib03="">at ata4-master UDMA100 SATA 1.5Gb/s
                          da0 at mpt0 bus 0 scbus0 target 1 lun 0
                          da0: <ibm-esxs vpca300900est1="" n="" a3c0="">Fixed Direct Access SCSI-6 device
                          da0: 300.000MB/s transfers
                          da0: Command Queueing enabled
                          da0: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
                          SMP: AP CPU #3 Launched!
                          SMP: AP CPU #1 Launched!
                          SMP: AP CPU #2 Launched!
                          uhub2: 6 ports with 6 removable, self powered
                          uhub3: 6 ports with 6 removable, self powered
                          ugen0.3: <ibm>at usbus0
                          ugen1.3: <logitech>at usbus1
                          ukbd0: <usb keyboard="">on usbus1
                          kbd2 at ukbd0
                          uhid0: <usb keyboard="">on usbus1
                          acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00
                          acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00
                          acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00
                          acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00
                          acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00
                          acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00
                          acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00
                          acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00
                          Trying to mount root from cd9660:/dev/iso9660/pfSense
                          md0.uzip: 2448 x 65536 blocks
                          pflog0: promiscuous mode enabled</usb></usb></logitech></ibm></ibm-esxs></tsstcorp></vendor></vendor></vendor></vendor></intel></intel></intel></intel></clock_battery></cpu></enhanced></cpu></enhanced></cpu></enhanced></cpu></enhanced></at></keyboard></generic></system></isa></at></power></ata></ata></intel></serial></ata></ata></intel></isa></pci-isa></acpi></acpi></intel></intel></intel(r)></acpi></acpi></acpi></acpi></intel(r)></acpi></acpi></vga-compatible></pci></pci-pci></acpi></acpi></lsilogic></acpi></acpi></intel></intel></intel(r)></intel(r)></intel(r)></intel(r)></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></ibm></software></version></ibm ></lahf></nx,rdtscp,lm></sse3,dtes64,mon,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,sse4.1,sse4.2,popcnt></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>

                          1 Reply Last reply Reply Quote 0
                          • B
                            Brutal last edited by

                            dmesg of HD install

                            Copyright © 1992-2010 The FreeBSD Project.
                            Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
                                    The Regents of the University of California. All rights reserved.
                            FreeBSD is a registered trademark of The FreeBSD Foundation.
                            FreeBSD 8.1-RELEASE-p2 #1: Mon Feb 14 02:00:18 EST 2011
                                root@FreeBSD_8.0_pfSense_2.0-snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense.8 i386
                            Timecounter "i8254" frequency 1193182 Hz quality 0
                            CPU: Intel(R) Xeon(R) CPU          X3450  @ 2.67GHz (2660.00-MHz 686-class CPU)
                              Origin = "GenuineIntel"  Id = 0x106e5  Family = 6  Model = 1e  Stepping = 5
                              Features=0xbfebfbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>Features2=0x98e3fd <sse3,dtes64,mon,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,sse4.1,sse4.2,popcnt>AMD Features=0x28100000 <nx,rdtscp,lm>AMD Features2=0x1 <lahf>TSC: P-state invariant
                            real memory  = 4294967296 (4096 MB)
                            avail memory = 2071941120 (1975 MB)
                            ACPI APIC Table: <ibm   ="" foxhollw="">FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
                            FreeBSD/SMP: 1 package(s) x 4 core(s)
                            cpu0 (BSP): APIC ID:  0
                            cpu1 (AP): APIC ID:  2
                            cpu2 (AP): APIC ID:  4
                            cpu3 (AP): APIC ID:  6
                            ACPI Warning: Invalid length for Pm1aControlBlock: 32, using default 16 (20100331/tbfadt-707)
                            ioapic0 <version 2.0="">irqs 0-23 on motherboard
                            wpi: You need to read the LICENSE file in /usr/share/doc/legal/intel_wpi/.
                            wpi: If you agree with the license, set legal.intel_wpi.license_ack=1 in /boot/loader.conf.
                            module_register_init: MOD_LOAD (wpi_fw, 0xc0958c30, 0) error 1
                            ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
                            ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
                            module_register_init: MOD_LOAD (ipw_bss_fw, 0xc07643f0, 0) error 1
                            ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
                            ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
                            module_register_init: MOD_LOAD (ipw_ibss_fw, 0xc07644b0, 0) error 1
                            ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
                            ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
                            module_register_init: MOD_LOAD (ipw_monitor_fw, 0xc0764570, 0) error 1
                            wlan: mac acl policy registered
                            kbd1 at kbdmux0
                            cryptosoft0: <software crypto="">on motherboard
                            padlock0: No ACE support.
                            acpi0: <ibm foxhollw="">on motherboard
                            acpi0: [ITHREAD]
                            acpi0: Power Button (fixed)
                            Timecounter "HPET" frequency 14318180 Hz quality 900
                            Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
                            acpi_timer0: <24-bit timer at 3.579545MHz> port 0x588-0x58b on acpi0
                            cpu0: <acpi cpu="">on acpi0
                            cpu1: <acpi cpu="">on acpi0
                            cpu2: <acpi cpu="">on acpi0
                            cpu3: <acpi cpu="">on acpi0
                            pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0
                            pci0: <acpi pci="" bus="">on pcib0
                            pcib1: <acpi pci-pci="" bridge="">irq 16 at device 3.0 on pci0
                            pci26: <acpi pci="" bus="">on pcib1
                            pcib2: <acpi pci-pci="" bridge="">irq 16 at device 5.0 on pci0
                            pci31: <acpi pci="" bus="">on pcib2
                            pci0: <base peripheral=""> at device 8.0 (no driver attached)
                            pci0: <base peripheral=""> at device 8.1 (no driver attached)
                            pci0: <base peripheral=""> at device 8.2 (no driver attached)
                            pci0: <base peripheral=""> at device 8.3 (no driver attached)
                            pci0: <base peripheral=""> at device 16.0 (no driver attached)
                            pci0: <base peripheral=""> at device 16.1 (no driver attached)
                            ehci0: <intel pch="" usb="" 2.0="" controller="" usb-b="">mem 0x91c01000-0x91c013ff irq 19 at device 26.0 on pci0
                            ehci0: [ITHREAD]
                            usbus0: EHCI version 1.0
                            usbus0: <intel pch="" usb="" 2.0="" controller="" usb-b="">on ehci0
                            pcib3: <acpi pci-pci="" bridge="">irq 17 at device 28.0 on pci0
                            pci1: <acpi pci="" bus="">on pcib3
                            mpt0: <lsilogic sas="" sata="" adapter="">port 0x3000-0x30ff mem 0x91b10000-0x91b13fff,0x91b00000-0x91b0ffff irq 16 at device 0.0 on pci1
                            mpt0: [ITHREAD]
                            mpt0: MPI Version=1.5.20.0
                            mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 )
                            mpt0: 0 Active Volumes (2 Max)
                            mpt0: 0 Hidden Drive Members (14 Max)
                            pcib4: <acpi pci-pci="" bridge="">irq 17 at device 28.4 on pci0
                            pci6: <acpi pci="" bus="">on pcib4
                            pcib5: <pci-pci bridge="">irq 16 at device 0.0 on pci6
                            pci7: <pci bus="">on pcib5
                            vgapci0: <vga-compatible display="">mem 0x90000000-0x90ffffff,0x91800000-0x91803fff,0x91000000-0x917fffff at device 0.0 on pci7
                            pcib6: <acpi pci-pci="" bridge="">irq 16 at device 28.5 on pci0
                            pci11: <acpi pci="" bus="">on pcib6
                            em0: <intel(r) 1000="" pro="" network="" connection="" 7.1.9="">port 0x2000-0x201f mem 0x91a80000-0x91a9ffff,0x91a00000-0x91a7ffff,0x91aa0000-0x91aa3fff irq 17 at device 0.0 on pci11
                            em0: Using MSIX interrupts with 3 vectors
                            em0: [ITHREAD]
                            em0: [ITHREAD]
                            em0: [ITHREAD]
                            pcib7: <acpi pci-pci="" bridge="">irq 18 at device 28.6 on pci0
                            pci16: <acpi pci="" bus="">on pcib7
                            pcib8: <acpi pci-pci="" bridge="">irq 19 at device 28.7 on pci0
                            pci21: <acpi pci="" bus="">on pcib8
                            em1: <intel(r) 1000="" pro="" network="" connection="" 7.1.9="">port 0x1000-0x101f mem 0x91980000-0x9199ffff,0x91900000-0x9197ffff,0x919a0000-0x919a3fff irq 19 at device 0.0 on pci21
                            em1: Using MSIX interrupts with 3 vectors
                            em1: [ITHREAD]
                            em1: [ITHREAD]
                            em1: [ITHREAD]
                            ehci1: <intel pch="" usb="" 2.0="" controller="" usb-a="">mem 0x91c00000-0x91c003ff irq 19 at device 29.0 on pci0
                            ehci1: [ITHREAD]
                            usbus1: EHCI version 1.0
                            usbus1: <intel pch="" usb="" 2.0="" controller="" usb-a="">on ehci1
                            pcib9: <acpi pci-pci="" bridge="">at device 30.0 on pci0
                            pci36: <acpi pci="" bus="">on pcib9
                            isab0: <pci-isa bridge="">at device 31.0 on pci0
                            isa0: <isa bus="">on isab0
                            atapci0: <intel pch="" sata300="" controller="">port 0x4078-0x407f,0x408c-0x408f,0x4070-0x4077,0x4088-0x408b,0x4050-0x405f,0x4040-0x404f irq 19 at device 31.2 on pci0
                            atapci0: [ITHREAD]
                            ata2: <ata 0="" channel="">on atapci0
                            ata2: [ITHREAD]
                            ata3: <ata 1="" channel="">on atapci0
                            ata3: [ITHREAD]
                            pci0: <serial bus,="" smbus="">at device 31.3 (no driver attached)
                            atapci1: <intel pch="" sata300="" controller="">port 0x4068-0x406f,0x4084-0x4087,0x4060-0x4067,0x4080-0x4083,0x4030-0x403f,0x4020-0x402f irq 18 at device 31.5 on pci0
                            atapci1: [ITHREAD]
                            ata4: <ata 0="" channel="">on atapci1
                            ata4: [ITHREAD]
                            ata5: <ata 1="" channel="">on atapci1
                            ata5: [ITHREAD]
                            acpi_button0: <power button="">on acpi0
                            acpi_button0: enable wake failed
                            atrtc0: <at realtime="" clock="">port 0x70-0x77 irq 8 on acpi0
                            atrtc0: Warning: Couldn't map I/O.
                            uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
                            uart0: [FILTER]
                            uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
                            uart1: [FILTER]
                            pmtimer0 on isa0
                            orm0: <isa option="" roms="">at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff pnpid ORM0000 on isa0
                            sc0: <system console="">at flags 0x100 on isa0
                            sc0: VGA <16 virtual consoles, flags=0x300>
                            vga0: <generic isa="" vga="">at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
                            ata0 at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
                            ata0: [ITHREAD]
                            ata1 at port 0x170-0x177,0x376 irq 15 on isa0
                            ata1: [ITHREAD]
                            atkbdc0: <keyboard controller="" (i8042)="">at port 0x60,0x64 on isa0
                            atkbd0: <at keyboard="">irq 1 on atkbdc0
                            kbd0 at atkbd0
                            atkbd0: [GIANT-LOCKED]
                            atkbd0: [ITHREAD]
                            ppc0: parallel port not found.
                            est0: <enhanced speedstep="" frequency="" control="">on cpu0
                            p4tcc0: <cpu frequency="" thermal="" control="">on cpu0
                            est1: <enhanced speedstep="" frequency="" control="">on cpu1
                            p4tcc1: <cpu frequency="" thermal="" control="">on cpu1
                            est2: <enhanced speedstep="" frequency="" control="">on cpu2
                            p4tcc2: <cpu frequency="" thermal="" control="">on cpu2
                            est3: <enhanced speedstep="" frequency="" control="">on cpu3
                            p4tcc3: <cpu frequency="" thermal="" control="">on cpu3
                            RTC BIOS diagnostic error 80 <clock_battery>Timecounters tick every 1.000 msec
                            IPsec: Initialized Security Association Processing.
                            usbus0: 480Mbps High Speed USB v2.0
                            usbus1: 480Mbps High Speed USB v2.0
                            ugen0.1: <intel>at usbus0
                            uhub0: <intel 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usbus0
                            ugen1.1: <intel>at usbus1
                            uhub1: <intel 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usbus1
                            uhub0: 2 ports with 2 removable, self powered
                            uhub1: 2 ports with 2 removable, self powered
                            ugen0.2: <vendor 0x8087="">at usbus0
                            uhub2: <vendor 2="" 9="" 0x8087="" product="" 0x0020,="" class="" 0,="" rev="" 2.00="" 0.00,="" addr="">on usbus0
                            ugen1.2: <vendor 0x8087="">at usbus1
                            uhub3: <vendor 2="" 9="" 0x8087="" product="" 0x0020,="" class="" 0,="" rev="" 2.00="" 0.00,="" addr="">on usbus1
                            da0 at mpt0 bus 0 scbus0 target 1 lun 0
                            da0: <ibm-esxs vpca300900est1="" n="" a3c0="">Fixed Direct Access SCSI-6 device
                            da0: 300.000MB/s transfers
                            da0: Command Queueing enabled
                            da0: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
                            acd0: DVDR <tsstcorp cddvdw="" ts-l633b="" ib03="">at ata4-master UDMA100 SATA 1.5Gb/s
                            SMP: AP CPU #3 Launched!
                            SMP: AP CPU #1 Launched!
                            SMP: AP CPU #2 Launched!
                            Root mount waiting for: usbus1 usbus0
                            uhub2: 6 ports with 6 removable, self powered
                            uhub3: 8 ports with 8 removable, self powered
                            Root mount waiting for: usbus1 usbus0
                            ugen0.3: <ibm>at usbus0
                            ugen1.3: <logitech>at usbus1
                            ukbd0: <usb keyboard="">on usbus1
                            kbd2 at ukbd0
                            uhid0: <usb keyboard="">on usbus1
                            Trying to mount root from ufs:/dev/da0s1a
                            WARNING: / was not properly dismounted
                            WARNING: R/W mount of / denied.  Filesystem is not clean - run fsck
                            pflog0: promiscuous mode enabled
                            em1: link state changed to UP
                            em0: link state changed to UP</usb></usb></logitech></ibm></tsstcorp></ibm-esxs></vendor></vendor></vendor></vendor></intel></intel></intel></intel></clock_battery></cpu></enhanced></cpu></enhanced></cpu></enhanced></cpu></enhanced></at></keyboard></generic></system></isa></at></power></ata></ata></intel></serial></ata></ata></intel></isa></pci-isa></acpi></acpi></intel></intel></intel(r)></acpi></acpi></acpi></acpi></intel(r)></acpi></acpi></vga-compatible></pci></pci-pci></acpi></acpi></lsilogic></acpi></acpi></intel></intel></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></acpi></ibm></software></version></ibm ></lahf></nx,rdtscp,lm></sse3,dtes64,mon,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,sse4.1,sse4.2,popcnt></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>

                            1 Reply Last reply Reply Quote 0
                            • W
                              wallabybob last edited by

                              It looks as if the devices on pci bus 31 (igb0 through igb3) are not visible in the pci device tree at startup in the HD boot you have logged. If the devices had been visible but unrecognised by any driver I would have expected the startup to report something like:

                              pci31: <network, ethernet="">at device 0.0 (no driver attached)</network,>

                              I have no explanation for why the devices might be visible in the PCI device tree on booting from CD but not on from booting from HD.

                              Could you provide the output of pfSense shell command # pciconf -l -v after booting from HD?

                              Do you see any different behaviour if you

                              • move the igb devices to another slot

                              • turn off the power, wait at least 20 seconds then boot from HD

                              1 Reply Last reply Reply Quote 0
                              • R
                                richc last edited by

                                Sorry for the long delay in getting back, I reinstalled with uniprocessor kernel and fixed the kernel  parameter to 4 and it seems to be OK now.

                                How I am just starting testing with LAGGS and trunked VLANs

                                1 Reply Last reply Reply Quote 0
                                • B
                                  Brutal last edited by

                                  Richc - That's good news.  I didn't intend to hijack your thread, I thought our issues may be similar dealing with the quad cards.

                                  Wallybybob  - I pulled the server from our co-lo facility and brought it back to the office.  Upon boot, the HD install saw the card.  Thinking about your last advice about pulling power and how that's in effect what I had just done, I proceeded to do several boots, by pulling power and also by software restarts.

                                  Each time the server was booted cold, it saw the card.  Each time the server was soft restarted, it didn't.

                                  I went back through the BIOS settings and found a setting for the PCI.  Settings were for Gen1 or Gen2 device.  It was set for Gen1.  I set it for Gen2.  Seems to be working now.  I should have seen that before.

                                  I'm going to do some changes to the hyperthreading and SMP install to see what the effects are and I'll post the results.  Thanks again Wally.

                                  1 Reply Last reply Reply Quote 0
                                  • W
                                    wallabybob last edited by

                                    @Brutal:

                                    I went back through the BIOS settings and found a setting for the PCI.  Settings were for Gen1 or Gen2 device.  It was set for Gen1.  I set it for Gen2.  Seems to be working now.  I should have seen that before.

                                    First time I've heard of anything like that: a BIOS switch to hide PCI devices on restart, And so aptly named  :)

                                    Well done.

                                    1 Reply Last reply Reply Quote 0
                                    • B
                                      Brutal last edited by

                                      After the bios "Gen2" change, it doesn't matter whether hyperthreading is active, or if the SMP kernel is installed, or both.  Works fine.

                                      It also doesn't matter whether the hw.igb.num_queues is set for 2 or 4.

                                      In case anyone is interested, this is on an IBM x3250 M3 server with a I340-4t quad card.  It is a bit finicky about having power pulled and reapplied.  Tends to lock up internals if you don't wait a bit.  It's also a very long boot process.  This runs the UEFI bios change.

                                      I'll be installing a twin for CARP soon.  Just love pf.

                                      Thanks again for your assistance.

                                      1 Reply Last reply Reply Quote 0
                                      • W
                                        wallabybob last edited by

                                        @Brutal:

                                        It also doesn't matter whether the hw.igb.num_queues is set for 2 or 4.

                                        I suspect that it will matter if you have "enough" cores or "enough" cards.

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post