New Alix board for 2013
-
What makes you think that it's the Realtek driver rather than something else? I'm fully prepare to believe it might be but just curious as to your thinking.
Also your window size is not consistent between those tests so the numbers are not directly comparable. Though the larger window in pfSense should be favourable.It's better to run Nano on flash media.
Steve
-
@ tmueko
Nice test! would be interesting to test with freebsd 10, hoping the troughput is increasing!
-
I just tracked down my performance-problem with the APU: It is a problem with the realtek-chipset on freebsd! That Device is unusable with pfSense yet:
The same APU, one time bootet with ipfire, one time with pfSense:
ipfire(linux):
[root@ipfire ~]# iperf -c 192.168.0.67 -p 1000
–----------------------------------------------------------
Client connecting to 192.168.0.67, TCP port 1000
TCP window size: 21.0 KByte (default)[ 3] local 192.168.0.7 port 32863 connected with 192.168.0.67 port 1000
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 667 MBytes 559 Mbits/secpfSense-2.1(FreeBSD-8.3)
[2.1-RELEASE][root@pfSense.localdomain]/root(14): iperf -c 192.168.0.67 -p 1000
–----------------------------------------------------------
Client connecting to 192.168.0.67, TCP port 1000
TCP window size: 65.0 KByte (default)[ 3] local 192.168.0.121 port 9760 connected with 192.168.0.67 port 1000
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 2.7 sec 8.62 MBytes 26.4 Mbits/secI'll try to build a NanoBSD-Image from 10.0 …
Hi I just did a quick test from my NAS running Debian Jessie and connected over a 1GB switch to the APU.
iperf -c pfsense -i ⏎ iperf: option requires an argument -- i ------------------------------------------------------------ Client connecting to pfsense, TCP port 5001 TCP window size: 85.0 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.30 port 46982 connected with 192.168.1.1 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 357 MBytes 299 Mbits/secPfsense running 2.1.1 Prerelease
-
I have somewhat different results:
10.0.8.3 10.0.8.1 (LAN - re0) ---------- ---------- | PC1 |--------------| APU | ---------- ---------- | 10.0.42.237 (WAN - re2) | | ----------- | PC2 | 10.0.42.2 ------------ pfSense 2.1
- portforward from WAN to PC1
- 50 concurrent iperf connections from PC1 to PC2
- 50 concurrent iperf connections from PC2 to PC1
- Quite constant 440~445 Mbit/s (see quotes below)
@PC1:
TCP window size: 85.3 KByte (default)
[SUM] 2230.0-2240.0 sec 259 MBytes 217 Mbits/sec
[SUM] 2240.0-2250.0 sec 262 MBytes 219 Mbits/sec
[SUM] 2250.0-2260.0 sec 257 MBytes 216 Mbits/sec
[SUM] 2260.0-2270.0 sec 259 MBytes 217 Mbits/sec
[SUM] 2270.0-2280.0 sec 257 MBytes 215 Mbits/sec
[SUM] 2280.0-2290.0 sec 260 MBytes 219 Mbits/sec
[SUM] 2290.0-2300.0 sec 256 MBytes 215 Mbits/sec
[SUM] 2300.0-2310.0 sec 258 MBytes 217 Mbits/sec
[SUM] 2310.0-2320.0 sec 259 MBytes 217 Mbits/sec
[SUM] 2320.0-2330.0 sec 259 MBytes 217 Mbits/sec
[SUM] 2330.0-2340.0 sec 260 MBytes 218 Mbits/sec
[SUM] 2340.0-2350.0 sec 260 MBytes 218 Mbits/sec
[SUM] 2350.0-2360.0 sec 259 MBytes 217 Mbits/sec
[SUM] 2360.0-2370.0 sec 262 MBytes 219 Mbits/sec
[SUM] 2370.0-2380.0 sec 257 MBytes 216 Mbits/sec
[SUM] 2380.0-2390.0 sec 260 MBytes 218 Mbits/sec
[SUM] 2390.0-2400.0 sec 260 MBytes 218 Mbits/sec
[SUM] 2400.0-2410.0 sec 258 MBytes 217 Mbits/sec
[SUM] 2410.0-2420.0 sec 260 MBytes 218 Mbits/sec
[SUM] 2420.0-2430.0 sec 263 MBytes 220 Mbits/sec@PC2:
TCP window size: 85.3 KByte (default)
[SUM] 2230.0-2240.0 sec 271 MBytes 227 Mbits/sec
[SUM] 2240.0-2250.0 sec 270 MBytes 226 Mbits/sec
[SUM] 2250.0-2260.0 sec 270 MBytes 227 Mbits/sec
[SUM] 2260.0-2270.0 sec 268 MBytes 225 Mbits/sec
[SUM] 2270.0-2280.0 sec 268 MBytes 225 Mbits/sec
[SUM] 2280.0-2290.0 sec 269 MBytes 226 Mbits/sec
[SUM] 2290.0-2300.0 sec 270 MBytes 226 Mbits/sec
[SUM] 2300.0-2310.0 sec 270 MBytes 226 Mbits/sec
[SUM] 2310.0-2320.0 sec 271 MBytes 227 Mbits/sec
[SUM] 2320.0-2330.0 sec 269 MBytes 226 Mbits/sec
[SUM] 2330.0-2340.0 sec 273 MBytes 229 Mbits/sec
[SUM] 2340.0-2350.0 sec 271 MBytes 227 Mbits/sec
[SUM] 2350.0-2360.0 sec 268 MBytes 225 Mbits/sec
[SUM] 2360.0-2370.0 sec 268 MBytes 225 Mbits/sec
[SUM] 2370.0-2380.0 sec 270 MBytes 227 Mbits/sec
[SUM] 2380.0-2390.0 sec 270 MBytes 226 Mbits/sec
[SUM] 2390.0-2400.0 sec 271 MBytes 227 Mbits/sec
[SUM] 2400.0-2410.0 sec 269 MBytes 226 Mbits/sec
[SUM] 2410.0-2420.0 sec 269 MBytes 225 Mbits/sec
[SUM] 2420.0-2430.0 sec 269 MBytes 226 Mbits/sec -
After pushing traffic for 3 hours i took a look at the board with our thermal camera.
IR000609: From top, closed case.
IR000611: From bottom, closed case.
IR000613: From top, open case.
-
Some great images there, cheers!
What colour case do you have BTW (I'm thinking about emissivity)?
-
It's the black case.
-
MiniPCI Express not just MiniPCI
Use Google
Oops :-[
Google tells me there are no Mini-PCIe crypto accelerators, so I guess it's moot anyway.
-
I have somewhat different results:
10.0.8.3 10.0.8.1 (LAN - em0) ---------- ---------- | PC1 |--------------| APU | ---------- ---------- | 10.0.42.237 (WAN - em2) | | ----------- | PC2 | 10.0.42.2 -----------You em0 .. em2 on your APU? Is that the 4GB Device? Can you post a dmesg from boot?
-
Nice, love the thermal images. :)
You surely mean re0-1 though rather than em?
Steve
-
Wooops.
Yes i meant re.
I have a dual miniPCIe gbit card which is (em).
Going to test the board with them tomorrow. -
I have received my board today :)
I will be using this board to upgrade an Alix system.
The fit of the board in the Alix case is no good :(
EDIT: Netgate told me that my case is an old Netgate made case (not PCEngine), and do not work with APU board :(The board is curved and it's impossible to align the leds an reset button with the case holes.
EDIT: Netgate told me that my case is an old Netgate made case (not PCEngine), and do not work with APU board :(I hope Netgate get the new cases soon.
(I just order 2 correct cases) -
One noteworthy point of the test i did.
During the test the board is not responsive anymore.
E.g:- serial console is dead
- webinterface doesn't load anymore
As soon as the test-traffic is disabled, everything works immediately again.
When i stopped the test i saw lots of these messages in the syslog:
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=21459663
ad4: WARNING - WRITE_DMA requeued due to channel reset LBA=21459599
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=21459515
ad4: WARNING - WRITE_DMA requeued due to channel reset LBA=21459663
ata2: FAILURE - already active DMA on this device
ata2: setting up DMA failed
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=21459523
g_vfs_done():ad4s1a[WRITE(offset=10987307008, length=32768)]error = 5
ad4: WARNING - WRITE_DMA requeued due to channel reset LBA=21459515
ata2: FAILURE - already active DMA on this device
ata2: setting up DMA failed
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=2145953g5
_vfs_done():ad4s1a[WRITE(offset=10987231232, length=2048)]error = 5
ad4: TIMEOUT - WRITE_DMA retrying (0 retries left) LBA=21459523
ad4: TIMEOUT - WRITE_DMA retrying (0 retries left) LBA=21459535
ad4: FAILURE - WRITE_DMA timed out LBA=21459535
g_vfs_done():ad4s1a[WRITE(offset=10987241472, length=32768)]error = 5
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=1152271
.
.
.
etc.i guess this are failed writes of the RRD graph (or is there anything else writing regularly to the disk?).
Without investigating this further for now, my bet is that the realtek cards generate an interrupt storm that stops the rest of the system from working correctly.
Let's see how everything behaves with the intel cards :) -
Ok tests done.
Setup same as before, but this time with em instead of re.
Used card: http://www.computex.biz/cervoz/default.aspx?com_id=6417&pdt_id=57237&PageType=ProductDetail&ContentTab=Introduction
(It seems this card is no longer available and there is a successor: http://www.cervoz.com/products/products_04.php?id=1815 )- Quite constant traffic around 660~680 Mbit/s (compared to the 440Mbit/s of the onboard interfaces)
- Same unresponsiveness of the board under load
- Same DMA errors
@PC1:
[SUM] 1400.0-1410.0 sec 404 MBytes 339 Mbits/sec
[SUM] 1410.0-1420.0 sec 399 MBytes 334 Mbits/sec
[SUM] 1420.0-1430.0 sec 405 MBytes 340 Mbits/sec
[SUM] 1430.0-1440.0 sec 400 MBytes 335 Mbits/sec
[SUM] 1440.0-1450.0 sec 409 MBytes 343 Mbits/sec
[SUM] 1450.0-1460.0 sec 394 MBytes 331 Mbits/sec
[SUM] 1460.0-1470.0 sec 397 MBytes 333 Mbits/sec
[SUM] 1470.0-1480.0 sec 399 MBytes 335 Mbits/sec
[SUM] 1480.0-1490.0 sec 397 MBytes 333 Mbits/sec
[SUM] 1490.0-1500.0 sec 406 MBytes 341 Mbits/sec
[SUM] 1500.0-1510.0 sec 416 MBytes 349 Mbits/sec
[SUM] 1510.0-1520.0 sec 416 MBytes 349 Mbits/sec
[SUM] 1520.0-1530.0 sec 419 MBytes 351 Mbits/sec
[SUM] 1530.0-1540.0 sec 419 MBytes 352 Mbits/sec
[SUM] 1540.0-1550.0 sec 418 MBytes 351 Mbits/sec
[SUM] 1550.0-1560.0 sec 397 MBytes 333 Mbits/sec
[SUM] 1560.0-1570.0 sec 403 MBytes 338 Mbits/sec
[SUM] 1570.0-1580.0 sec 399 MBytes 335 Mbits/sec
[SUM] 1580.0-1590.0 sec 405 MBytes 340 Mbits/sec
[SUM] 1590.0-1600.0 sec 407 MBytes 342 Mbits/sec@PC2:
[SUM] 1400.0-1410.0 sec 400 MBytes 336 Mbits/sec
[SUM] 1410.0-1420.0 sec 394 MBytes 330 Mbits/sec
[SUM] 1420.0-1430.0 sec 399 MBytes 335 Mbits/sec
[SUM] 1430.0-1440.0 sec 392 MBytes 329 Mbits/sec
[SUM] 1440.0-1450.0 sec 401 MBytes 336 Mbits/sec
[SUM] 1450.0-1460.0 sec 386 MBytes 324 Mbits/sec
[SUM] 1460.0-1470.0 sec 396 MBytes 332 Mbits/sec
[SUM] 1470.0-1480.0 sec 404 MBytes 339 Mbits/sec
[SUM] 1480.0-1490.0 sec 390 MBytes 327 Mbits/sec
[SUM] 1490.0-1500.0 sec 397 MBytes 333 Mbits/sec
[SUM] 1500.0-1510.0 sec 399 MBytes 335 Mbits/sec
[SUM] 1510.0-1520.0 sec 395 MBytes 331 Mbits/sec
[SUM] 1520.0-1530.0 sec 395 MBytes 331 Mbits/sec
[SUM] 1530.0-1540.0 sec 395 MBytes 331 Mbits/sec
[SUM] 1540.0-1550.0 sec 396 MBytes 332 Mbits/sec
[SUM] 1550.0-1560.0 sec 391 MBytes 328 Mbits/sec
[SUM] 1560.0-1570.0 sec 403 MBytes 338 Mbits/sec
[SUM] 1570.0-1580.0 sec 396 MBytes 332 Mbits/sec
[SUM] 1580.0-1590.0 sec 399 MBytes 334 Mbits/sec
[SUM] 1590.0-1600.0 sec 397 MBytes 333 Mbits/sec
-
hi
i tried to install the embedded version on a 4GB Kingston SD Card. but it doesn't fit, so i used the 2GB nanobsd edition.
but when i try to boot, i got errors about mounting the root parition, wrong geometry aborting startupcan i fix this by this
https://doc.pfsense.org/index.php/Soekris_6501_USB_Flash_installation#Root_Mount_erroror the USB Boot Problem article?
Yes, that are the steps, you have to go, but 'set kern.cam.boot_delay="5000"' will be enough.
Thanks for your reply. Will this maybe added to the nanobsd image as default? also the automatic mapping of the new APU interfaces re0 re1 re2?
I used now the normal installer from a usb stick. i installed the embedded kernel and set a ramdisk for /tmp and /var in the webconfigurator. is this enough, that my SD card will not die in a few weeks or have i better to reinstall the whole boxes with the embedded edition?
regards angelo
Can somebody give me an answer on this?
regards
-
When i stopped the test i saw lots of these messages in the syslog:
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=21459663
ad4: WARNING - WRITE_DMA requeued due to channel reset LBA=21459599
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=21459515
ad4: WARNING - WRITE_DMA requeued due to channel reset LBA=21459663
ata2: FAILURE - already active DMA on this device
ata2: setting up DMA failed
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=21459523
g_vfs_done():ad4s1a[WRITE(offset=10987307008, length=32768)]error = 5
ad4: WARNING - WRITE_DMA requeued due to channel reset LBA=21459515
ata2: FAILURE - already active DMA on this device
ata2: setting up DMA failed
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=2145953g5
_vfs_done():ad4s1a[WRITE(offset=10987231232, length=2048)]error = 5
ad4: TIMEOUT - WRITE_DMA retrying (0 retries left) LBA=21459523
ad4: TIMEOUT - WRITE_DMA retrying (0 retries left) LBA=21459535
ad4: FAILURE - WRITE_DMA timed out LBA=21459535
g_vfs_done():ad4s1a[WRITE(offset=10987241472, length=32768)]error = 5
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=1152271
.
.
.
etc.i guess this are failed writes of the RRD graph (or is there anything else writing regularly to the disk?).
Are you running a plain NanoBSD install on there? That really sounds like what happens when you need to make these changes:
https://doc.pfsense.org/index.php/Boot_Troubleshooting#NanoBSD_on_Newer_HardwareThanks for your reply. Will this maybe added to the nanobsd image as default? also the automatic mapping of the new APU interfaces re0 re1 re2?
Unlikely, as it isn't needed for most. Not sure on interface mappings, but the general default may eventually change, though I'm not sure if the APU interfaces would be the new default or em0/em1 as those are increasingly more common.
I used now the normal installer from a usb stick. i installed the embedded kernel and set a ramdisk for /tmp and /var in the webconfigurator. is this enough, that my SD card will not die in a few weeks or have i better to reinstall the whole boxes with the embedded edition?
If you have a RAM disk setup for /var and /tmp using a full install should be OK on there so long as you aren't going crazy with packages that would write to the disk.
I'm running a full install on an mSATA in my APU and it's good so far. Not sure I'll bother with NanoBSD.
When Netgate starts shipping them in assembled form, they'll likely be full installs on mSATA not SD.
-
When i stopped the test i saw lots of these messages in the syslog:
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=21459663
ad4: WARNING - WRITE_DMA requeued due to channel reset LBA=21459599
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=21459515
ad4: WARNING - WRITE_DMA requeued due to channel reset LBA=21459663
ata2: FAILURE - already active DMA on this device
ata2: setting up DMA failed
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=21459523
g_vfs_done():ad4s1a[WRITE(offset=10987307008, length=32768)]error = 5
ad4: WARNING - WRITE_DMA requeued due to channel reset LBA=21459515
ata2: FAILURE - already active DMA on this device
ata2: setting up DMA failed
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=2145953g5
_vfs_done():ad4s1a[WRITE(offset=10987231232, length=2048)]error = 5
ad4: TIMEOUT - WRITE_DMA retrying (0 retries left) LBA=21459523
ad4: TIMEOUT - WRITE_DMA retrying (0 retries left) LBA=21459535
ad4: FAILURE - WRITE_DMA timed out LBA=21459535
g_vfs_done():ad4s1a[WRITE(offset=10987241472, length=32768)]error = 5
ad4: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=1152271
.
.
.
etc.i guess this are failed writes of the RRD graph (or is there anything else writing regularly to the disk?).
Are you running a plain NanoBSD install on there? That really sounds like what happens when you need to make these changes:
https://doc.pfsense.org/index.php/Boot_Troubleshooting#NanoBSD_on_Newer_HardwareNo this is a memstick install on the SSD.
However with the embedded kernel to get the serial output.
I've added these settings. and am testing again.One thing i've noticed is that with the intel cards a don't get any "DMA failed" messages anymore.
Only the retrying messages. -
No this is a memstick install on the SSD.
However with the embedded kernel to get the serial output.
I've added these settings. and am testing again.One thing i've noticed is that with the intel cards a don't get any "DMA failed" messages anymore.
Only the retrying messages.Difference could be that I'm using AHCI (Netgate image) and not the old ATA stack. Seems to work fine for me with the embedded kernel that way.
-
Tested again with the onboard NICs.
Getting lots of DMA errors under load again.Which image for netgate do you mean?
I could try with this one too.For now i'm going to test 2.2 :)
-
Tested again with the onboard NICs.
Getting lots of DMA errors under load again.Which image for netgate do you mean?
I could try with this one too.For now i'm going to test 2.2 :)
It's not a public image, it's the one that will be loaded onto the APU when it's released by Netgate. It's Netgate's 2.1.1 version tailored for the hardware, though some things are still being tweaked for it yet.
2.2 would have AHCI by default (no old ATA stack) so it might not be a bad test, though the current 2.2 snap has a pf rule issue the next one should be better (or pick an older one)
