Topton N100 Reporting 402 MHz

[stephenw10]

Mmm, I'm not sure C-states is correct there. P-states are what Speedstep uses, that's shown as 801/6000 800/6000 700/5160 for you. Speedshift just changes the levels sing hardware in the CPU rather than software controlled which means it's much faster.

[TheNarc]

@stephenw10 Alright, I have this Topton machine back in my possession now, just swapped the old apu2e4 back in for now. I just booted MX Linux from a thumb drive, and it has no problems at all. I can see from inxi that it boosts a single core up to 3400MHz for the openssl speed test, and that speed test is executing about 8X faster than it is under pfSense:

$ openssl speed -elapsed -evp aes-256-cbc
You have chosen to measure elapsed time instead of user CPU time.
Doing AES-256-CBC for 3s on 16 size blocks: 160803402 AES-256-CBC's in 3.00s
Doing AES-256-CBC for 3s on 64 size blocks: 52614619 AES-256-CBC's in 3.00s
Doing AES-256-CBC for 3s on 256 size blocks: 10652587 AES-256-CBC's in 3.00s
Doing AES-256-CBC for 3s on 1024 size blocks: 2021881 AES-256-CBC's in 3.00s
Doing AES-256-CBC for 3s on 8192 size blocks: 246344 AES-256-CBC's in 3.00s
Doing AES-256-CBC for 3s on 16384 size blocks: 121479 AES-256-CBC's in 3.00s
version: 3.0.11
built on: Mon Oct 23 17:52:22 2023 UTC
options: bn(64,64)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -fzero-call-used-regs=used-gpr -DOPENSSL_TLS_SECURITY_LEVEL=2 -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/reproducible-path/openssl-3.0.11=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
CPUINFO: OPENSSL_ia32cap=0x7ffaf3bfffebffff:0x98c007bc239ca7eb
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
AES-256-CBC     857618.14k  1122445.21k   909020.76k   690135.38k   672683.35k   663437.31k

So I think it's safe to say that whatever this is, it's not a hardware issue, and likely not a BIOS setting issue (unless there's some BIOS setting that makes FreeBSD very unhappy but Linux doesn't care about).

Now in pfSense I am of course restoring the config from the apu2e4, but there were really no exotic changes made to it. That said, barring other ideas I could try doing a reinstall of pfSense with no config restore and see whether I still get this behavior. But does this make any obvious sense? Certainly there are others using N100s with pfSense.

I'm trying to think of what to test next and open to any ideas Thanks again!

[stephenw10]

It could still be a BIOS issue. Many will pass different values depending on the running OS. Though historically that has been a Windows issue.

You can try running FreeBSD directly.

[TheNarc]

@stephenw10 Good thought. Yeah I'm thinking next I'll boot a fresh pfSense image from a thumb drive because if that does not exhibit the problem, it will implicate something about my config, but if it does I'll try straight FreeBSD.

[TheNarc]

@stephenw10 Well whatever this is, it seems to be either a FreeBSD thing or to your point, something that the BIOS is passing differently to FreeBSD, or FreeBSD is acting on differently.

I booted the pfSense installed from a thumb drive and dropped to the rescue prompt, ran the same openssl speed test and saw the same poor results. Next I grabbed the NanoBSD live system image and booted it, again ran the same openssl speed test, and again saw the same slow results. The current NanoBSD is based on FreeBSD 14.0-RELEASE-p4.

I can't believe that no one else is using an N100 with pfSense (or is and has not noticed this issue) so it leads me back to wondering about the BIOS settings, but there's really nothing obvious there, to me at least.

I'm going to try to get some pictures of the potentially interesting BIOS settings to post next . . . not sure really where to go from here.

[stephenw10]

There are a whole, bunch of people using N100 based devices and not seeing that sort of low performance. Which is another reason I'm leaning toward some odd BIOS behaviour. I pretty guaranty that BIOS was never tested with FreeBSD. You're lucky if it was tested with Linux.

Been a while since I tried this but you might try adding a loader value:
hw.acpi.install_interface='Linux'

[TheNarc]

@stephenw10 Ha that's a very fair point! I'll try the loader value soon, thank you. The following are (poor quality, sorry) pictures of the CPU Configuration BIOS options screens, which strike me as the most likely relevant, at least:

CPU Configuration:
login-to-view /home/john/Pictures/BIOS_Advanced_CPU_Configuration.jpeg

SMM Enhancement sub-menu:
login-to-view /home/john/Pictures/BIOS_Advanced_CPU_Configuration_SMM_Enhancement.jpeg

And this is everything available in the Advanced tab of the BIOS interface:
login-to-view /home/john/Pictures/BIOS_Advanced.jpeg

I realize that this is becoming something of a wild goose chase but if anything here catches your eye . . .

[stephenw10]

Hmm, I don't have an N100 (or similar) to play with so I'm not really sure. You might be able to set a higher flex ratio if you enable override there.

[TheNarc]

@stephenw10 Yeah that was something I tried but anything higher than 8 seems to just snap back to 8. I found some conflicting information about that setting, but the consensus seemed to be that it was the minimum multiplier allowed. It would take 7, but nothing above 8. Perhaps I'll begin the fraught task of seeing whether I can locate a different BIOS to try with this thing But going to try your loader value first for sure.

[Tzvia]

I had been pointed in the right direction in a couple of threads at the 'Serve the Home Forums' (STH Youtube channel's forums, under NETWORKING.). There are threads dealing with the CWWK 6 ethernet port (what I have) and 4 ethernet port models... I was just 'let's try this.. no no... try that...' and getting nowhere till I found that forum and hunkered down and read the two long threads on those two models and decided to try what was suggested (including a different BIOS with all the hidden settings exposed). Yea not a turnkey device but I was determined to make it work...

PFSense still doesn't display it quite right in the GUI
login-to-view
but I'm not complaining at this point.

Once tweaked, in PFS I was able to go to System/Advanced/Miscellaneous and enable SpeedShift with 'core level control' and after some testing, left it at 60%. PowerD set OFF.

[xavix]

@stephenw10 just wanted to chime in and share that I running pfSense on a Trigkey G5 mini PC with a N100 and have also wondered why my CPU reads a max of 806 Mhz. Watching this thread in hope to find answers!

[TheNarc]

@stephenw10 said in Topton N100 Reporting 402 MHz:

Been a while since I tried this but you might try adding a loader value:
hw.acpi.install_interface='Linux'

Unfortunately no change in behavior with that. I was momentarily optimistic when I found this post on unlocking hidden BIOS settings for the exact board that my machine has, although it does not seem to do anything for me.

Another post here purports to have a modified BIOS for it with more settings unlocked, which is clearly risky, but I may try because I'm out of ideas at this point . . .

[stephenw10]

Hmm, that's a looooong thread! If there are answers they're probably there.

[TheNarc]

@stephenw10 Ha, yeah and two separate very long threads unfortunately, and both quite meandering and not at all specific to one piece of hardware or problem. Lots of fun.

[TheNarc]

@stephenw10 I threw caution to the wind and flashed the modded BIOS from the one thread, and against all odds it didn't brick me. But it didn't open up new settings that seemed interesting to this issue either (e.g. CPU power related things). I booted up pfSense anyway and ran the openssl test a few times. The first time it was the same as always, but then it got a little better. Still about 1/4 to 1/3 of what I was seeing in Linux though. I checked the CPU freqs via sysctl -a | grep dev.cpu and saw they were going higher, but for whatever reason seem to top out at 1813MHz, which feels like a random limit.

This was with SpeedShift enabled in pfSense and set to 0 (max performance). Next I tried with SpeedShift disabled entirely, and I seem to be getting openssl speed results closer to 1/2 what I saw in Linux, but the CPU freqs as reported by sysctl never move from 700MHz, so they're pretty clearly inaccurate.

Running powerd -v while I run the openssl speed test shows the current freq going to 801MHz under load, which I have read in various places is the maximum reported value (as reported by something at least) when it's at a boosted frequency. However it's also curious that the "wanted freq" reported by powerd -v never goes above 1602MHz, whereas the cores on the N100 can boost to 3.0GHz (or a single core can burst to 3.4GHz).

What feels suspicious there is that 1.6GHz is ~50% of 3.0GHz (or 3.4GHz) and I am seeing ~50% the benchmarks for the openssl test that I saw with Linux. But then the question is: why would the "wanted" frequency top out at about half what the processor is capable of?

In any case, 1.6GHz is better than my apu2e4, but I do hate not understanding things :) Maybe I'll try randomly playing with some more BIOS settings if I get time, but since none of them seem obviously relevant it will literally just be a game of flipping a switch and seeing if it does anything, ha.

[TheNarc]

By way of an amusing point of comparison on my older N5105 machine, powerd -v reports this during the openssl test:
load 102%, current freq 2795 MHz ( 0), wanted freq 3992 MHz

And that's on a CPU with a maximum boost frequency of 2.9GHz. So I'm kind of lost with respect to attempting to establish a ground truth source of information. It seems that every place I look for a reported CPU frequency gives me something different . . .

[stephenw10]

Mmm, in my experience powerd will show 'wanted' values between the actual available steps too. So it showing wanting more than can be achieved is not entirely surprising.

[TheNarc]

@stephenw10 Well I loaded another modded BIOS from here that exposes power & performance options so now I can enable or disable SpeedStep and SpeedShift, C-states, change PL1 and PL2, etc. But nothing seems to meaningfully move the needle on this . . . incredibly frustrating. If anyone else who has been following this is running N100-based hardware, I'd be curious to know the results of:
openssl speed -elapsed -evp aes-256-cbc
for you. I can't help but wonder whether this is really highly specific to the hardware/BIOS combination I have, or if performance may be degraded generally for the N100 in FreeBSD, just not so much that it's generally noticeable when used for applications such as pfSense. That seems highly unlikely, but any points of comparison would be welcome. Thank you!

[TheNarc]

One other observation - likely not related, but just piling on the strangeness here - is that iperf tests to this machine exhibit strange behavior. If it's been sitting idle for a bit, usually the first test shows about the expected 1Gbps speed (950+Mbps). But if I re-run several times, it consistently drops until it hits ~400Mbps and seems to stay there. I realize that iperf tests to the firewall as opposed to through it are of limited value, but I'm again using the N5105 box I have as a point of comparison, which gets consistent 950+Mbps iperf results. That machine has i225 NICs as opposed to the i226 in my N100 machine, but are otherwise configured identically. Getting ready to cut my losses and toss this Topton into an industrial shredder, but I sure would love to gain some understanding of what's going on before doing so, ha.

[stephenw10]

Have you checked the heatsink on that device is actually attached in a useful way? Wouldn't be the first China sourced device people have seen with the wrong shim or no thermal paste that was just running into thermal throttling all the time.