Firebox x1250e crash after upgrading



  • I did an auto upgrade on my x1250e and now I cant even get a console to see whats happening. I get a black screen after the BIOS does its thing. Anyone have a successful upgrade on any watchguard firebox x750e and up?



  • I did an auto upgrade on my x1250e and now I cant even get a console to see whats happening. I get a black screen after the BIOS does its thing. Anyone have a successful upgrade on any watchguard firebox x750e and up?

    2.2 has change default speed. Default serial speed is now 115200 #3715

    I have a same problem (well, almost). After upgrading my x750e, I couldn't boot anymore. When I connect with the serial cable, I can see that it gets stock at:

    random: unblocking device.
    ada0 at ata0 bus 0 scbus0 target 0 lun 0
    ada0: <runcore 20091009="" 32g-c="" ssd="">ATA-0 device
    ada0: Serial Number 20090410 02240168
    ada0: 66.700MB/s transfers (UDMA4, PIO 512bytes)
    ada0: 30535MB (62537328 512 byte sectors: 16H 63S/T 62041C)
    ada0: Previously was known as ad0
    Timecounter "TSC" frequency 1300072808 Hz quality 800
    (ada0:ata0:0:0:0): READ_DMA. ACB: c8 00 6e 3e ba 43 00 00 00 00 01 00
    (ada0:ata0:0:0:0): CAM status: Command timeout
    (ada0:ata0:0:0:0): Retrying command
    (ada0:ata0:0:0:0): READ_DMA. ACB: c8 00 6e 3e ba 43 00 00 00 00 01 00
    (ada0:ata0:0:0:0): CAM status: Command timeout
    (ada0:ata0:0:0:0): Retrying command</runcore> 
    

    I've been playing with BIOS options for hours now without any luck. Any ideas?

    Hardware:

    Watchguard x750e
    storage CF card
    BIOS: B8



  • Start here

    https://forum.pfsense.org/index.php?topic=87312.msg479454#msg479454

    Right at "Simply stop the boot"…



  • Ill give it a try, still nothing on the upgraded box but black even on 115200, i tried before posting 9600, 19200 (INITIAL BIOS EDIT BAUD), and 115200

    I am using a HDD and not a CF card.



  • @igorek24:

    I did an auto upgrade on my x1250e and now I cant even get a console to see whats happening. I get a black screen after the BIOS does its thing. Anyone have a successful upgrade on any watchguard firebox x750e and up?

    2.2 has change default speed. Default serial speed is now 115200 #3715

    I have a same problem (well, almost). After upgrading my x750e, I couldn't boot anymore. When I connect with the serial cable, I can see that it gets stock at:

    random: unblocking device.
    ada0 at ata0 bus 0 scbus0 target 0 lun 0
    ada0: <runcore 20091009="" 32g-c="" ssd="">ATA-0 device
    ada0: Serial Number 20090410 02240168
    ada0: 66.700MB/s transfers (UDMA4, PIO 512bytes)
    ada0: 30535MB (62537328 512 byte sectors: 16H 63S/T 62041C)
    ada0: Previously was known as ad0
    Timecounter "TSC" frequency 1300072808 Hz quality 800
    (ada0:ata0:0:0:0): READ_DMA. ACB: c8 00 6e 3e ba 43 00 00 00 00 01 00
    (ada0:ata0:0:0:0): CAM status: Command timeout
    (ada0:ata0:0:0:0): Retrying command
    (ada0:ata0:0:0:0): READ_DMA. ACB: c8 00 6e 3e ba 43 00 00 00 00 01 00
    (ada0:ata0:0:0:0): CAM status: Command timeout
    (ada0:ata0:0:0:0): Retrying command</runcore> 
    

    I've been playing with BIOS options for hours now without any luck. Any ideas?

    Hardware:

    Watchguard x750e
    storage CF card
    BIOS: B8

    Atleast you can see whats happening :)



  • Stuck on this:

    cpu0 on motherboard
    est0: <enhanced speedstep="" frequency="" control="">on cpu0
    est: CPU supports Enhanced Speedstep, but is not recognized.
    est: cpu_vendor GenuineIntel, msr 612102906001029
    device_attach: est0 attach returned 6
    p4tcc0: <cpu frequency="" thermal="" control="">on cpu0
    pmtimer0 on isa0
    gpioapu0: Environment returned (null)

    Fatal trap 12: page fault while in kernel mode
    cpuid = 0; apic id = 00
    fault virtual address  = 0x0
    fault code              = supervisor read, page not present
    instruction pointer    = 0x20:0xc0da8cf0
    stack pointer          = 0x28:0xc2420c28
    frame pointer          = 0x28:0xc2420c34
    code segment            = base 0x0, limit 0xfffff, type 0x1b
                            = DPL 0, pres 1, def32 1, gran 1
    processor eflags        = interrupt enabled, resume, IOPL = 0
    current process        = 0 (swapper)
    [ thread pid 0 tid 100000 ]
    Stopped at      strncmp+0x20:  movzbl  0(%esi),%ebx
    db></cpu></enhanced>



  • Tried switching the kernel at boot and that does not work at all



  • Tracing pid 0 tid 100000 td 0xc209d460
    strncmp(0,c13a8aed,3,c2420c54,8c,…) at strncmp+0x20/frame 0xc2420c34
    gpioapu_probe(c7966400,c77d9858,c1df84e4,c796643c,0,...) at gpioapu_probe+0x4e/frame 0xc2420c6c
    device_probe_child(c791be00,c7966400,0,0,0,...) at device_probe_child+0x181/frame 0xc2420ca4
    device_probe(c7966400,c7966400,c2420d24,c0c81c52,c7966400,...) at device_probe+0x41/frame 0xc2420cc8
    device_probe_and_attach(c7966400,c1e0020c,0,c1df8460,c75ea640,...) at device_probe_and_attach+0x12/frame 0xc2420cd8
    isa_probe_children(c791be00) at isa_probe_children+0x112/frame 0xc2420d24
    configure(0,0,2,0,c15f4784,...) at configure+0x1b/frame 0xc2420d30
    mi_startup() at mi_startup+0xe7/frame 0xc2420d58
    begin() at begin+0x2c
    db>



  • What bios are you running?


  • Netgate Administrator

    Have a read through this thread. Identical issue:
    https://forum.pfsense.org/index.php?topic=85930.msg479775#msg479775

    Steve



  • @stephenw10:

    Have a read through this thread. Identical issue:
    https://forum.pfsense.org/index.php?topic=85930.msg479775#msg479775

    Steve

    ill give it a try, now only if I could see what the upgrade did instead of having to fix this new install



  • @chpalmer:

    Start here

    https://forum.pfsense.org/index.php?topic=87312.msg479454#msg479454

    Right at "Simply stop the boot"…

    Worked for me. Thanks.



  • Tried this and no go, same error, same screen, same traceback info:

    set vfs.root.mountfrom=ufs:/dev/ad0s1a
    set hw.ata.ata_dma="1"

    My HDD is primary and never was slave. BIOS is award BIOS v6

    I do have the HW to flash BIOS chips directly and via CF card

    The one I am working on is a new install not an upgrade.



  • Hi firebox,

    I had the same problem (0xc0da8cf0) , can you post your boot.config.local please ?
    Stephen already posted the link  ;)

    it should be empty.
    Also what bios are you using B6 ?
    I suggest flashing it to B8 and set the HD to auto

    greetz



  • @TieT:

    Hi firebox,

    I had the same problem (0xc0da8cf0) , can you post your boot.config.local please ?
    Stephen already posted the link  ;)

    it should be empty.
    Also what bios are you using B6 ?
    I suggest flashing it to B8 and set the HD to auto

    greetz

    Ill get the boot conf tonight. I flashed the latest BIOS to the firebox, the one where you dont have to alter the HDD settings to have the system detect the HDD. I dont know what version it is



  • Found the newer BIOS, my file was called ESERM, ill flash XEBIOS_8.BIN tonight. Anyone try this BIOS yet? XEBIOS_81.BIN


  • Netgate Administrator

    @firebox:

    my file was called ESERM

    What file is that? Where did you get it from?

    Steve



  • @stephenw10:

    @firebox:

    my file was called ESERM

    What file is that? Where did you get it from?

    Steve

    Yeah, that doesn't sound right !
    anyway the XEBIOS_8.BIN is the correct one  ;)



  • I see a XEBIOS_8.BIN and XEBIOS_81.BIN file out there.

    Whats the difference?  I have x8500e and x5500e FBs.

    Thanks!



  • I think you can use the XEBIOS_81.bin bios, but stephen is the person who can tell you exactly what the difference is.


  • Netgate Administrator

    I forgot to enable the ACPI menu in the bios on version 8. Enabling it didn't seem to justify an entire version jump so I added a .1.
    Although in the boot screen I think it says 0.81 any way.  ::)

    Versions 1-7, many of which never made it beyond my test box, were based on the code extracted from the x750e (and similar) models. Verison 8 was from code extracted from the SSL model which is equivalent to the X550e in hardware terms but crucially boots from a HD rather than CF.

    XEBIOS_81.BIN is the file to use. I didn't link to it for a while because not that many people had tested it but I've yet to here any bad reports so it should be considered the default option at this point.

    Steve



  • Flashed XEBIOS_8.BIN,

    (E17HDD) MOD V0.8 console on com1 115200. Stephenw10 10/12/13

    hit ESC during boot to get the loader prompt and entered:

    set vfs.root.mountfrom=ufs:/dev/ad0s1a
    set hw.ata.ata_dma="1"

    Then entered boot and enter to start the boot up.

    I still get after the above:

    cpu0 on motherboard
    est0: <enhanced speedstep="" frequency="" control="">on cpu0
    est: CPU supports Enhanced Speedstep, but is not recognized.
    est: cpu_vendor GenuineIntel, msr 612102906001029
    device_attach: est0 attach returned 6
    p4tcc0: <cpu frequency="" thermal="" control="">on cpu0
    pmtimer0 on isa0
    gpioapu0: Environment returned (null)

    Fatal trap 12: page fault while in kernel mode
    cpuid = 0; apic id = 00
    fault virtual address  = 0x0
    fault code              = supervisor read, page not present
    instruction pointer    = 0x20:0xc0da8cf0
    stack pointer          = 0x28:0xc2420c28
    frame pointer          = 0x28:0xc2420c34
    code segment            = base 0x0, limit 0xfffff, type 0x1b
                            = DPL 0, pres 1, def32 1, gran 1
    processor eflags        = interrupt enabled, resume, IOPL = 0
    current process        = 0 (swapper)
    [ thread pid 0 tid 100000 ]
    Stopped at      strncmp+0x20:  movzbl  0(%esi),%ebx
    db></cpu></enhanced>



  • I am adding the 81 BIOS to my flash card now and will flash that then enter:

    set vfs.root.mountfrom=ufs:/dev/ad0s1a
    set hw.ata.ata_dma="1"

    Then enter boot and enter to start the boot up.

    What am I doing wrong here?



  • I found some other boot.config files, but while in the loader prompt i cant find a boot.config.local

    loader_conf_files="/boot/device.hints /boot/loader.conf /boot/loader.conf.local"


  • Netgate Administrator

    Did you reset the CMOS after flashing the BIOS? Are the IDE parameters now set to auto?
    I don't actually think that would make any difference since it's obviously find the drive and booting from it.
    One of the bigger changes going to 2.2 is the new drivers for ATA devices:
    https://doc.pfsense.org/index.php/UpgradeGuide#Disk_Driver_Changes
    I suspect your install is trying to use swap space in a location that no longer exists, on ad0 as opposed to ada0. For other mounted drives/slices there are aliases to link to the new names but perhaps not with the swap slice. Can you access the filesystem table at /etc/fstab? What are it's contents?

    Steve



  • Ill check tonight when I have the device in front of me



  • etc/fstab
    *** FILE etc/fstab BEGIN ***

    Device                Mountpoint      FStype  Options        Dump    Pass#

    /dev/ada0s1a              /                  ufs        rw                1          1
    /dev/ada0s1b            none            swap      sw                0          0
    *** FILE etc/fstab END ***



  • Are you sure your HD is mounted as ad0s1a
    because I see you entered this line:```
    set vfs.root.mountfrom=ufs:/dev/ad0s1a

    
    but your HD in the fstab file is ada0s1a so the mountpount should be:```
    set vfs.root.mountfrom=ufs:/dev/ada0s1a
    


  • If you type dmesg in console you can see the boot messages and find out the name of your HD.


  • Netgate Administrator

    Not if you never get to a console.  ;)
    There should be aliases in place to take care of anything that still points at ad0 as opposed to ada0. I speculated that perhaps that isn't working for swap.

    There is a thread that collates all the reports of this error, lets try and keep all info there:
    https://forum.pfsense.org/index.php?topic=87542.0

    Steve



  • Didn't know about the aliases  :)

    I just modified the fstab file


  • Netgate Administrator

    No reason to know, it should 'just work'.  ;)

    Did you update to BIOS 8_1 to make this load do you think?

    Steve



  • I got Bios_81 on mine yes, and the HD settings are set to auto.
    The only thing i had to change is the fstab file or when booting, testing it with set vfs.root.mountfrom=ufs:/dev/adxxxxxx


  • Netgate Administrator

    I think it's the swapper process trying to use DMA and failing. On my box I had to enable UDMA for the IDE channel I was using, did you have to make that change?
    Also:
    @unknown001:

    Also enable ACPI function and IDE primary slave UDMA is now "auto" mode. No longer crashing

    Steve



  • yes i also did those steps


  • Netgate Administrator

    Ok this is an ACPI problem, it must be enabled:
    https://forum.pfsense.org/index.php?topic=87542.msg484384#msg484384

    Steve



  • I am a little lost on what I should try next….



  • The HDD is being detected as stephenw10 stated since it starts the boot process but ends with that kernel panic info I posted with a trace after.


  • Netgate Administrator

    You need to enable ACPI in the BIOS or at least check it's enabled. It could potentially be something else but that will definitely cause the panic you're seeing.

    Steve



  • On it! Thanks stephenw10…Ill try it in about 30min. All I did the other night was pull the FSTAB, I did not reset/check the HDD BIOS settings so I will do that as well since I have to go there for ACPI. Oh I did flash the 8.1 BIOS you created after no change with 8.