Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PfSense 2.4.2 crashing on PC engines apu2 at random times

    Scheduled Pinned Locked Moved General pfSense Questions
    20 Posts 9 Posters 3.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      CS
      last edited by

      @acascianelli, it's running around 50-55 Celsius and it happens every 10 days or so.

      @jimp, thanks for the hint, I'll search for another power supply and give it a try. I also think it's hardware related, I hope it's the hard drive or the power supply that I can easily replace and not the main board.

      1 Reply Last reply Reply Quote 0
      • C
        CS
        last edited by

        I finally managed to get some error logs from my console, nothing written in system logs though.

        
ahcich0: Timeout on slot 10 port 0
ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffe7ff tfd 40 serr 00000000 cm d 00406a17
(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 01 e8 c7 00 40 00 00 00 00 00 00
(ada0:ahcich0:0:0:0): CAM status: Command timeout
(ada0:ahcich0:0:0:0): Retrying command
ahcich0: Timeout on slot 11 port 0
ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00000800 tfd 50 serr 00000000 cm d 00406b17
(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
(aprobe0:ahcich0:0:0:0): Retrying command
ahcich0: Timeout on slot 12 port 0
ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00001000 tfd 50 serr 00000000 cm d 00406c17
(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
(aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
ahcich0: Timeout on slot 13 port 0
ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00002000 tfd 50 serr 00000000 cm d 00406d17
(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
(aprobe0:ahcich0:0:0:0): Error 5, Retry was blocked
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <ts16gmsa310 20120703="">s/n 20121222A55XXXXXX detached
...
db:0:kdb.enter.default> textdump set
...
db:0:kdb.enter.default>  capture on
...
db:0:kdb.enter.default>  run lockinfo
...
db:0:kdb.enter.default>  show pcpu
...
db:0:kdb.enter.default>  bt
...
db:0:kdb.enter.default>  ps
...
db:0:kdb.enter.default>  alltrace
...
db:0:kdb.enter.default>  capture off
...
db:0:kdb.enter.default>  textdump dump
...
...
Tracing command kernel pid 0 tid 100099 td 0xfffff800144fe000
sched_switch() at sched_switch+0x4aa/frame 0xfffffe011fdb9960
mi_switch() at mi_switch+0xe5/frame 0xfffffe011fdb9990
sleepq_wait() at sleepq_wait+0x3a/frame 0xfffffe011fdb99c0
_sleep() at _sleep+0x255/frame 0xfffffe011fdb9a40
taskqueue_thread_loop() at taskqueue_thread_loop+0x121/frame 0xfffffe011fdb9a70
fork_exit() at fork_exit+0x85/frame 0xfffffe011fdb9ab0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe011fdb9ab0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
db:0:kdb.enter.default>  capture off
db:0:kdb.enter.default>  textdump dump
textdump_writeblock: offset 801111552, error 6
Textdump: Error 6 writing dump
db:0:kdb.enter.default>  reset
cpu_reset: Restarting BSP
cpu_reset_proxy: Stopped CPU 2
PC Engines apu2
coreboot build 07/24/2017
BIOS version v4.6.0</ts16gmsa310>

        This is where it gets stuck and doesn't boot until I manually reset it.

        1 Reply Last reply Reply Quote 0
        • C
          CS
          last edited by

          Could a moderator move this topic under "Hardware" please?

          1 Reply Last reply Reply Quote 0
          • S
            software
            last edited by

            Hi,

            Same issue on APU3. I noticed few days ago the same behaviour

            
ahcich0: Timeout on slot 10 port 0
ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffe7ff tfd 40 serr 00000000 cm d 00406a17
(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 01 e8 c7 00 40 00 00 00 00 00 00
(ada0:ahcich0:0:0:0): CAM status: Command timeout
(ada0:ahcich0:0:0:0): Retrying command
ahcich0: Timeout on slot 11 port 0

            Temperature says around  53 - 57 °C .

            Running coreboot v4.6.0

            1 Reply Last reply Reply Quote 0
            • C
              CS
              last edited by

              That's interesting. Let me also highlight that the crashes do NOT happen when the system is under load.

              PC Engines apu2
              Coreboot: build 07/24/2017
              BIOS: version v4.6.0
              pfSense: 2.4.2-RELEASE-p1 (amd64)
              OS: FreeBSD 11.1-RELEASE-p6
              mSATA SSD: Transcend TS16GMSA310 16 GB - https://www.amazon.co.uk/Transcend-TS16GMSA310-16-GB-Internal/dp/B007DIS8Y2

              @software, what kind of storage do you use?

              1 Reply Last reply Reply Quote 0
              • GilG
                Gil Rebel Alliance
                last edited by

                I run a number of APU2; Coreboot 4.0.7 & have not seen this issue occur.

                11 cheers for binary

                1 Reply Last reply Reply Quote 0
                • H
                  hda
                  last edited by

                  No problems:

                  System PC Engines APU2B2
                  BIOS Vendor: coreboot
                  Version: 88a4f96
                  Release Date: Mon Mar 7 2016
                  Version 2.4.2-RELEASE-p1 (amd64)
                  built on Tue Dec 12 13:45:26 CST 2017
                  FreeBSD 11.1-RELEASE-p6
                  &
                  mSATA SSD: Transcend TS16GMSA370 16 GB

                  1 Reply Last reply Reply Quote 0
                  • S
                    software
                    last edited by

                    @/CS:

                    That's interesting. Let me also highlight that the crashes do NOT happen when the system is under load.

                    PC Engines apu2
                    Coreboot: build 07/24/2017
                    BIOS: version v4.6.0
                    pfSense: 2.4.2-RELEASE-p1 (amd64)
                    OS: FreeBSD 11.1-RELEASE-p6
                    mSATA SSD: Transcend TS16GMSA310 16 GB - https://www.amazon.co.uk/Transcend-TS16GMSA310-16-GB-Internal/dp/B007DIS8Y2

                    @software, what kind of storage do you use?

                    It's a no name chinese mSata SSD.  I think the cheap SSD bits me now in the ass.
                    Same here, it was during the night. Almost no load, except some openvpn ping traffic and some home automation.

                    
Jan 23 23:33:46	kernel		(aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
Jan 23 23:33:46	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:33:46	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:33:46	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00008000 tfd 50 serr 00000000 cmd 00406f17
Jan 23 23:33:46	kernel		ahcich0: Timeout on slot 15 port 0
Jan 23 23:33:16	kernel		(aprobe0:ahcich0:0:0:0): Retrying command
Jan 23 23:33:16	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:33:16	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:33:16	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00004000 tfd 50 serr 00000000 cmd 00406e17
Jan 23 23:33:16	kernel		ahcich0: Timeout on slot 14 port 0
Jan 23 23:32:46	kernel		(aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
Jan 23 23:32:46	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:32:46	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:32:46	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00000800 tfd 50 serr 00000000 cmd 00406b17
Jan 23 23:32:46	kernel		ahcich0: Timeout on slot 11 port 0
Jan 23 23:32:16	kernel		(aprobe0:ahcich0:0:0:0): Retrying command
Jan 23 23:32:16	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:32:16	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:32:16	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00000400 tfd 50 serr 00000000 cmd 00406a17
Jan 23 23:32:16	kernel		ahcich0: Timeout on slot 10 port 0
Jan 23 23:31:46	kernel		(aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
Jan 23 23:31:46	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:31:46	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:31:46	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00000020 tfd 50 serr 00000000 cmd 00406517
Jan 23 23:31:46	kernel		ahcich0: Timeout on slot 5 port 0
Jan 23 23:31:16	kernel		(aprobe0:ahcich0:0:0:0): Retrying command
Jan 23 23:31:16	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:31:16	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:31:16	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00000010 tfd 50 serr 00000000 cmd 00406417
Jan 23 23:31:16	kernel		ahcich0: Timeout on slot 4 port 0
Jan 23 23:30:46	kernel		(aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
Jan 23 23:30:46	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:30:46	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:30:46	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00000002 tfd 50 serr 00000000 cmd 00406117
Jan 23 23:30:46	kernel		ahcich0: Timeout on slot 1 port 0
Jan 23 23:30:16	kernel		(aprobe0:ahcich0:0:0:0): Retrying command
Jan 23 23:30:16	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:30:16	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:30:16	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00000001 tfd 50 serr 00000000 cmd 00406017
Jan 23 23:30:16	kernel		ahcich0: Timeout on slot 0 port 0
Jan 23 23:29:46	kernel		(aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted
Jan 23 23:29:46	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:29:46	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:29:46	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 08000000 tfd 50 serr 00000000 cmd 00407b17
Jan 23 23:29:46	kernel		ahcich0: Timeout on slot 27 port 0
Jan 23 23:29:16	kernel		(aprobe0:ahcich0:0:0:0): Retrying command
Jan 23 23:29:16	kernel		(aprobe0:ahcich0:0:0:0): CAM status: Command timeout
Jan 23 23:29:16	kernel		(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
Jan 23 23:29:16	kernel		ahcich0: is 00000002 cs 00000000 ss 00000000 rs 04000000 tfd 50 serr 00000000 cmd 00407a17
                    1 Reply Last reply Reply Quote 0
                    • KOMK
                      KOM
                      last edited by

                      Looks like crappy disks to me.

                      1 Reply Last reply Reply Quote 0
                      • C
                        CS
                        last edited by

                        @KOM:

                        Looks like crappy disks to me.

                        I think so too.

                        I have ordered a new "Transcend 32GB mSATA SSD (TS32GMSA370)" and I'll keep you posted.

                        1 Reply Last reply Reply Quote 0
                        • S
                          silentcreek
                          last edited by

                          Did your system crash before you upgraded the BIOS to 4.6.0? Which BIOS version were you running then? 4.0.x or 4.5.x?

                          Because the version that you are currently running is not recommended to use with pfSense or FreeBSD. PCEngines has a warning on their BIOS/Howto page:

                          For FreeBSD based OS like OPNSense and pfSense please use the legacy versions.

                          Note: "legacy" versions are 4.0.x
                          See: http://pcengines.ch/howto.htm#bios

                          There have been several reports about issues with the newer coreboot releases (afaik not this issue though), so I wouldn't be surprised if this is caused by the firmware and not the disk itself.

                          That being said, if you have seen this on the older firmware 4.0x. as well, then it's likely a disk issue.

                          1 Reply Last reply Reply Quote 0
                          • C
                            CS
                            last edited by

                            @silentcreek:

                            Did your system crash before you upgraded the BIOS to 4.6.0? Which BIOS version were you running then? 4.0.x or 4.5.x?
                            …
                            That being said, if you have seen this on the older firmware 4.0x. as well, then it's likely a disk issue.

                            It actually happened when I was on older firmware, I don't recall the version. I just thought it was a good opportunity to upgrade to the latest one hoping that it could possibly help.
                            I'll find out soon.

                            1 Reply Last reply Reply Quote 0
                            • C
                              CS
                              last edited by

                              I bought a brand new SSD (Transcend 32GB SATA III 6Gb/s MSA370 mSATA SSD - TS32GMSA370) and tried to reinstall pfSense loading the installer from one of the USB ports, but I'm getting similar errors during the installation process and it eventually fails. I tried multiple times and it always fails during the extraction of distribution files. Any ideas what's going on? The board itself maybe? I hope not…

                              
 pfSense Installer
 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
ahcich0: Timeout on slot 15 port 0
ahcich0: is 00000008 cs 00000000 ss 00000000 rs 0000ffc0 tfd 40 serr 00000000 cmd 00406f17
(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 a8 60 62 40 03 00 00 00 00 00
(ada0:ahcich0:0:0:0): CAM status: Command timeout
(ada0:ahcich0:0:0:0): Retrying command

                      lqqqqqqqArchive Extractionqqqqqqqqqk
                      x Extracting distribution files... x
                      x                                  x
                      x                                  x
                      x   Overall Progress:              x
                      x  lqqqqqqqqqqqqqqqqqqqqqqqqqqqqk  x
                      x  x             23%            x  x
                      x  mqqqqqqqqqqqqqqqqqqqqqqqqqqqqj  x
                      mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

 pfSense Installer
 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
ahcich0: Timeout on slot 15 port 0
ahcich0: is 00000008 cs 00000000 ss 00000000 rs 0000ffc0 tfd 40 serr 00000000 cmd 00406f17
(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 a8 60 62 40 03 00 00 00 00 00
(ada0:ahcich0:0:0:0): CAM status: Command timeout
(ada0:ahcich0:0:0:0): Retrying command
ahcich0: Timeout on slot 16 port 0
ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00010000 tfd 50 serr 00000000 cmd 00407017
(aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
(aprobe0:ahcich0:0:0:0): CAM status: Command timeout     x
(aprobe0:ahcich0:0:0:0): Retrying command                x
                      x   Overall Progress:              x
                      x  lqqqqqqqqqqqqqqqqqqqqqqqqqqqqk  x
                      x  x             23%            x  x
                      x  mqqqqqqqqqqqqqqqqqqqqqqqqqqqqj  x
                      mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
                              1 Reply Last reply Reply Quote 0
                              • C
                                CS
                                last edited by

                                For FreeBSD based OS like OPNSense and pfSense please use the legacy versions.

                                Reference: http://pcengines.ch/howto.htm#bios

                                The comment above was the reason I went to https://github.com/pcengines/apu2-documentation#binary-releases and downloaded http://pcengines.ch/file/apu2_v4.0.12.rom.tar.gz which is the latest "Legacy" version. After downgrading to 4.0.12 the installation of pfSense on my new SSD completed smoothly and without any errors.

                                Lets wait now for a week or so to see if the system will crash again or not.

                                1 Reply Last reply Reply Quote 0
                                • C
                                  CS
                                  last edited by

                                  Uptime: 22 days

                                  I had no crach since I downgraded to the stable firmware and replaced my SSD.  ;)

                                  1 Reply Last reply Reply Quote 0
                                  • boukeB
                                    bouke
                                    last edited by

                                    @/CS:

                                    Uptime: 22 days

                                    I had no crach since I downgraded to the stable firmware and replaced my SSD.  ;)

                                    Thanks! I have a similar problem and I will try the downgrade and report back.

                                    Apr  8 11:12:10 pfsense kernel: ahcich0: Timeout on slot 14 port 0
                                    Apr  8 11:12:10 pfsense kernel: ahcich0: is 00000008 cs 00000000 ss 00000000 rs 00007800 tfd 40 serr 00000000 cmd 00406e17
                                    Apr  8 11:12:10 pfsense kernel: (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 01 0f c3 00 40 00 00 00 00 00 00
                                    Apr  8 11:12:10 pfsense kernel: (ada0:ahcich0:0:0:0): CAM status: Command timeout
                                    Apr  8 11:12:10 pfsense kernel: (ada0:ahcich0:0:0:0): Retrying command
                                    Apr  8 11:12:40 pfsense kernel: ahcich0: Timeout on slot 15 port 0
                                    Apr  8 11:12:40 pfsense kernel: ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00008000 tfd 50 serr 00000000 cmd 00406f17
                                    Apr  8 11:12:40 pfsense kernel: (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
                                    Apr  8 11:12:40 pfsense kernel: (aprobe0:ahcich0:0:0:0): CAM status: Command timeout
                                    Apr  8 11:12:40 pfsense kernel: (aprobe0:ahcich0:0:0:0): Retrying command
                                    Apr  8 11:13:10 pfsense kernel: ahcich0: Timeout on slot 16 port 0
                                    Apr  8 11:13:10 pfsense kernel: ahcich0: is 00000002 cs 00000000 ss 00000000 rs 00010000 tfd 50 serr 00000000 cmd 00407017
                                    Apr  8 11:13:10 pfsense kernel: (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
                                    Apr  8 11:13:10 pfsense kernel: (aprobe0:ahcich0:0:0:0): CAM status: Command timeout
                                    Apr  8 11:13:10 pfsense kernel: (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted

                                    1 Reply Last reply Reply Quote 0
                                    • boukeB
                                      bouke
                                      last edited by

                                      Everything works absolutely fine after the BIOS downgrade. I am running coreboot 4.0.7.1 now. Uptime 1 day and >6 hours.

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post
                                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.