PC Engines apu2 experiences
-
@tman222 You are welcome - I was also interested. I went back to pfS 2.4.5p1 and I'm on 101Mbit/s throughput again with OpenVPN.
-
RAW DATA COMPARISION ON APU2 (apu2d4) BIOS v4.13.0.3 mainline
- psSense 2.5.0
- OPNSense 21.1.1
- IPFire 2.25 - Core Update 153
Two devices connected directly on WANs (same hardware, same software, same cables, same firmware, default installation tuning options)
WAN to WAN
- psSense 2.5.0
[2.5.0-RELEASE][root@pfSense.localdomain]/root: iperf3 -c 192.168.1.1 Connecting to host 192.168.1.1, port 5201 [ 5] local 192.168.1.2 port 23459 connected to 192.168.1.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 32.2 MBytes 270 Mbits/sec 0 143 KBytes [ 5] 1.00-2.00 sec 29.1 MBytes 244 Mbits/sec 0 143 KBytes [ 5] 2.00-3.00 sec 32.5 MBytes 272 Mbits/sec 0 144 KBytes [ 5] 3.00-4.00 sec 29.1 MBytes 244 Mbits/sec 0 160 KBytes [ 5] 4.00-5.00 sec 32.2 MBytes 270 Mbits/sec 0 161 KBytes [ 5] 5.00-6.00 sec 32.3 MBytes 271 Mbits/sec 0 161 KBytes [ 5] 6.00-7.00 sec 32.4 MBytes 271 Mbits/sec 0 161 KBytes [ 5] 7.00-8.00 sec 28.2 MBytes 237 Mbits/sec 0 161 KBytes [ 5] 8.00-9.00 sec 32.3 MBytes 271 Mbits/sec 0 161 KBytes [ 5] 9.00-10.00 sec 32.4 MBytes 271 Mbits/sec 0 161 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 313 MBytes 262 Mbits/sec 0 sender [ 5] 0.00-10.26 sec 313 MBytes 256 Mbits/sec receiver[2.5.0-RELEASE][root@pfSense.localdomain]/root: iperf3 -c 192.168.1.1 -P 10 I've lost detailed data for 10 connections, sorry, but I remember about 760 Mbits/sec in total- OPNSense 21.1.1
root@OPNsense:~ # iperf3 -c 192.168.1.1 -p 22628 Connecting to host 192.168.1.1, port 22628 [ 5] local 192.168.1.2 port 38078 connected to 192.168.1.1 port 22628 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 31.8 MBytes 266 Mbits/sec 0 96.2 KBytes [ 5] 1.00-2.00 sec 31.4 MBytes 263 Mbits/sec 0 96.2 KBytes [ 5] 2.00-3.00 sec 31.5 MBytes 264 Mbits/sec 0 96.2 KBytes [ 5] 3.00-4.00 sec 31.2 MBytes 262 Mbits/sec 0 96.2 KBytes [ 5] 4.00-5.00 sec 31.5 MBytes 265 Mbits/sec 0 96.2 KBytes [ 5] 5.00-6.00 sec 31.6 MBytes 265 Mbits/sec 0 96.2 KBytes [ 5] 6.00-7.00 sec 31.6 MBytes 265 Mbits/sec 0 96.2 KBytes [ 5] 7.00-8.00 sec 31.6 MBytes 265 Mbits/sec 0 113 KBytes [ 5] 8.00-9.00 sec 31.3 MBytes 263 Mbits/sec 0 113 KBytes [ 5] 9.00-10.00 sec 31.3 MBytes 263 Mbits/sec 0 113 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 315 MBytes 264 Mbits/sec 0 sender [ 5] 0.00-10.07 sec 315 MBytes 262 Mbits/sec receiver root@OPNsense:~ # iperf3 -c 192.168.1.1 -p 22628 -P 10 ^C- - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [SUM] 0.00-0.00 sec 0.00 Bytes 0.00 bits/sec 0 sender [SUM] 0.00-0.00 sec 0.00 Bytes 0.00 bits/sec receiver iperf3: interrupt - the client has terminated root@OPNsense:~ # iperf3 -c 192.168.1.1 -p 38044 -P 10 Connecting to host 192.168.1.1, port 38044 [ 5] local 192.168.1.2 port 17792 connected to 192.168.1.1 port 38044 [ 7] local 192.168.1.2 port 32377 connected to 192.168.1.1 port 38044 [ 9] local 192.168.1.2 port 8971 connected to 192.168.1.1 port 38044 [ 11] local 192.168.1.2 port 47517 connected to 192.168.1.1 port 38044 [ 13] local 192.168.1.2 port 37570 connected to 192.168.1.1 port 38044 [ 15] local 192.168.1.2 port 63767 connected to 192.168.1.1 port 38044 [ 17] local 192.168.1.2 port 23932 connected to 192.168.1.1 port 38044 [ 19] local 192.168.1.2 port 34884 connected to 192.168.1.1 port 38044 [ 21] local 192.168.1.2 port 64675 connected to 192.168.1.1 port 38044 [ 23] local 192.168.1.2 port 14313 connected to 192.168.1.1 port 38044 ... ... ... - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 69.0 MBytes 57.8 Mbits/sec 0 sender [ 5] 0.00-10.11 sec 69.0 MBytes 57.3 Mbits/sec receiver [ 7] 0.00-10.01 sec 65.1 MBytes 54.5 Mbits/sec 0 sender [ 7] 0.00-10.11 sec 65.1 MBytes 54.0 Mbits/sec receiver [ 9] 0.00-10.01 sec 101 MBytes 85.0 Mbits/sec 0 sender [ 9] 0.00-10.11 sec 101 MBytes 84.2 Mbits/sec receiver [ 11] 0.00-10.01 sec 65.2 MBytes 54.6 Mbits/sec 0 sender [ 11] 0.00-10.11 sec 65.2 MBytes 54.1 Mbits/sec receiver [ 13] 0.00-10.01 sec 91.5 MBytes 76.7 Mbits/sec 0 sender [ 13] 0.00-10.11 sec 91.5 MBytes 76.0 Mbits/sec receiver [ 15] 0.00-10.01 sec 65.3 MBytes 54.7 Mbits/sec 0 sender [ 15] 0.00-10.11 sec 65.3 MBytes 54.2 Mbits/sec receiver [ 17] 0.00-10.01 sec 85.8 MBytes 71.9 Mbits/sec 0 sender [ 17] 0.00-10.11 sec 85.8 MBytes 71.2 Mbits/sec receiver [ 19] 0.00-10.01 sec 78.1 MBytes 65.4 Mbits/sec 0 sender [ 19] 0.00-10.11 sec 78.1 MBytes 64.8 Mbits/sec receiver [ 21] 0.00-10.01 sec 71.0 MBytes 59.4 Mbits/sec 0 sender [ 21] 0.00-10.11 sec 71.0 MBytes 58.9 Mbits/sec receiver [ 23] 0.00-10.01 sec 49.4 MBytes 41.4 Mbits/sec 0 sender [ 23] 0.00-10.11 sec 49.3 MBytes 41.0 Mbits/sec receiver [SUM] 0.00-10.01 sec 742 MBytes 622 Mbits/sec 0 sender [SUM] 0.00-10.11 sec 742 MBytes 616 Mbits/sec receiver- IPFire 2.25 - Core Update 153
[root@ipfire ~]# iperf3 -c 192.168.1.1 Connecting to host 192.168.1.1, port 5201 [ 5] local 192.168.1.2 port 39252 connected to 192.168.1.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 114 MBytes 957 Mbits/sec 0 402 KBytes [ 5] 1.00-2.00 sec 112 MBytes 939 Mbits/sec 0 402 KBytes [ 5] 2.00-3.00 sec 112 MBytes 942 Mbits/sec 0 419 KBytes [ 5] 3.00-4.00 sec 112 MBytes 942 Mbits/sec 0 440 KBytes [ 5] 4.00-5.00 sec 112 MBytes 944 Mbits/sec 0 440 KBytes [ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 0 440 KBytes [ 5] 6.00-7.00 sec 112 MBytes 942 Mbits/sec 0 440 KBytes [ 5] 7.00-8.00 sec 112 MBytes 942 Mbits/sec 0 440 KBytes [ 5] 8.00-9.00 sec 112 MBytes 943 Mbits/sec 0 440 KBytes [ 5] 9.00-10.00 sec 111 MBytes 932 Mbits/sec 0 440 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.10 GBytes 942 Mbits/sec 0 sender [ 5] 0.00-10.03 sec 1.10 GBytes 938 Mbits/sec receiver [root@ipfire ~]# iperf3 -c 192.168.1.1 -P 10 Connecting to host 192.168.1.1, port 5201 [ 5] local 192.168.1.2 port 39256 connected to 192.168.1.1 port 5201 [ 7] local 192.168.1.2 port 39258 connected to 192.168.1.1 port 5201 [ 9] local 192.168.1.2 port 39260 connected to 192.168.1.1 port 5201 [ 11] local 192.168.1.2 port 39262 connected to 192.168.1.1 port 5201 [ 13] local 192.168.1.2 port 39264 connected to 192.168.1.1 port 5201 [ 15] local 192.168.1.2 port 39266 connected to 192.168.1.1 port 5201 [ 17] local 192.168.1.2 port 39268 connected to 192.168.1.1 port 5201 [ 19] local 192.168.1.2 port 39270 connected to 192.168.1.1 port 5201 [ 21] local 192.168.1.2 port 39272 connected to 192.168.1.1 port 5201 [ 23] local 192.168.1.2 port 39274 connected to 192.168.1.1 port ... ... ... - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 115 MBytes 96.7 Mbits/sec 0 sender [ 5] 0.00-10.01 sec 114 MBytes 95.8 Mbits/sec receiver [ 7] 0.00-10.00 sec 110 MBytes 91.9 Mbits/sec 0 sender [ 7] 0.00-10.01 sec 109 MBytes 91.3 Mbits/sec receiver [ 9] 0.00-10.00 sec 110 MBytes 92.1 Mbits/sec 0 sender [ 9] 0.00-10.01 sec 109 MBytes 91.3 Mbits/sec receiver [ 11] 0.00-10.00 sec 116 MBytes 97.2 Mbits/sec 0 sender [ 11] 0.00-10.01 sec 114 MBytes 95.9 Mbits/sec receiver [ 13] 0.00-10.00 sec 115 MBytes 96.7 Mbits/sec 0 sender [ 13] 0.00-10.01 sec 114 MBytes 95.9 Mbits/sec receiver [ 15] 0.00-10.00 sec 115 MBytes 96.5 Mbits/sec 0 sender [ 15] 0.00-10.01 sec 114 MBytes 95.8 Mbits/sec receiver [ 17] 0.00-10.00 sec 115 MBytes 96.7 Mbits/sec 0 sender [ 17] 0.00-10.01 sec 114 MBytes 95.9 Mbits/sec receiver [ 19] 0.00-10.00 sec 115 MBytes 96.4 Mbits/sec 0 sender [ 19] 0.00-10.01 sec 114 MBytes 95.6 Mbits/sec receiver [ 21] 0.00-10.00 sec 110 MBytes 92.0 Mbits/sec 0 sender [ 21] 0.00-10.01 sec 109 MBytes 91.2 Mbits/sec receiver [ 23] 0.00-10.00 sec 110 MBytes 92.6 Mbits/sec 0 sender [ 23] 0.00-10.01 sec 110 MBytes 91.9 Mbits/sec receiver [SUM] 0.00-10.00 sec 1.10 GBytes 949 Mbits/sec 0 sender [SUM] 0.00-10.01 sec 1.10 GBytes 940 Mbits/sec receiverOpenVPN (LAN2 to LAN1) AES128-CBC SHA1
- psSense 2.5.0
Connecting to host 192.168.101.1, port 5201 [ 5] local 10.10.11.2 port 36519 connected to 192.168.101.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 5.28 MBytes 44.3 Mbits/sec 0 63.0 KBytes [ 5] 1.00-2.00 sec 5.25 MBytes 44.1 Mbits/sec 0 63.0 KBytes [ 5] 2.00-3.00 sec 5.27 MBytes 44.2 Mbits/sec 0 63.0 KBytes [ 5] 3.00-4.00 sec 5.10 MBytes 42.8 Mbits/sec 0 63.0 KBytes [ 5] 4.00-5.00 sec 5.00 MBytes 42.0 Mbits/sec 0 63.0 KBytes [ 5] 5.00-6.00 sec 5.18 MBytes 43.4 Mbits/sec 0 63.0 KBytes [ 5] 6.00-7.00 sec 5.25 MBytes 44.1 Mbits/sec 0 63.0 KBytes [ 5] 7.00-8.00 sec 5.15 MBytes 43.2 Mbits/sec 0 63.0 KBytes [ 5] 8.00-9.00 sec 5.24 MBytes 43.9 Mbits/sec 0 63.0 KBytes [ 5] 9.00-10.00 sec 5.23 MBytes 43.9 Mbits/sec 0 63.0 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 52.0 MBytes 43.6 Mbits/sec 0 sender [ 5] 0.00-10.25 sec 51.9 MBytes 42.5 Mbits/sec receiver- OPNSense 21.1.1
Connecting to host 192.168.101.1, port 24683 [ 5] local 10.10.11.2 port 39164 connected to 192.168.101.1 port 24683 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 4.88 MBytes 41.0 Mbits/sec 0 64.4 KBytes [ 5] 1.00-2.00 sec 4.80 MBytes 40.2 Mbits/sec 0 64.4 KBytes [ 5] 2.00-3.00 sec 4.81 MBytes 40.4 Mbits/sec 0 65.7 KBytes [ 5] 3.00-4.00 sec 4.74 MBytes 39.8 Mbits/sec 0 65.7 KBytes [ 5] 4.00-5.00 sec 4.80 MBytes 40.2 Mbits/sec 0 65.7 KBytes [ 5] 5.00-6.00 sec 4.76 MBytes 39.9 Mbits/sec 0 65.7 KBytes [ 5] 6.00-7.00 sec 4.77 MBytes 40.0 Mbits/sec 0 65.7 KBytes [ 5] 7.00-8.00 sec 4.80 MBytes 40.3 Mbits/sec 0 65.7 KBytes [ 5] 8.00-9.00 sec 4.76 MBytes 40.0 Mbits/sec 0 65.7 KBytes [ 5] 9.00-10.00 sec 4.75 MBytes 39.8 Mbits/sec 0 65.7 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 47.9 MBytes 40.2 Mbits/sec 0 sender [ 5] 0.00-10.10 sec 47.8 MBytes 39.8 Mbits/sec receiver- IPFire 2.25 - Core Update 153
Connecting to host 192.168.101.1, port 5201 [ 5] local 10.10.11.2 port 38256 connected to 192.168.101.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 7.33 MBytes 61.5 Mbits/sec 29 134 KBytes [ 5] 1.00-2.00 sec 7.46 MBytes 62.6 Mbits/sec 0 163 KBytes [ 5] 2.00-3.00 sec 7.34 MBytes 61.5 Mbits/sec 0 190 KBytes [ 5] 3.00-4.00 sec 6.97 MBytes 58.5 Mbits/sec 0 213 KBytes [ 5] 4.00-5.00 sec 7.76 MBytes 65.1 Mbits/sec 5 176 KBytes [ 5] 5.00-6.00 sec 7.46 MBytes 62.6 Mbits/sec 0 199 KBytes [ 5] 6.00-7.00 sec 7.52 MBytes 63.1 Mbits/sec 0 223 KBytes [ 5] 7.00-8.00 sec 7.58 MBytes 63.6 Mbits/sec 2 183 KBytes [ 5] 8.00-9.00 sec 7.58 MBytes 63.6 Mbits/sec 0 206 KBytes [ 5] 9.00-10.00 sec 7.70 MBytes 64.6 Mbits/sec 0 226 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 74.7 MBytes 62.7 Mbits/sec 36 sender [ 5] 0.00-10.05 sec 74.3 MBytes 62.0 Mbits/sec receiverOpenVPN (LAN2 to LAN1) AES128-GCM
- psSense 2.5.0
Connecting to host 192.168.105.1, port 5201 [ 5] local 10.10.15.2 port 24242 connected to 192.168.105.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 6.26 MBytes 52.5 Mbits/sec 0 63.9 KBytes [ 5] 1.00-2.00 sec 6.14 MBytes 51.5 Mbits/sec 0 63.9 KBytes [ 5] 2.00-3.00 sec 6.16 MBytes 51.7 Mbits/sec 0 63.9 KBytes [ 5] 3.00-4.00 sec 5.97 MBytes 50.1 Mbits/sec 0 63.9 KBytes [ 5] 4.00-5.00 sec 6.28 MBytes 52.7 Mbits/sec 0 63.9 KBytes [ 5] 5.00-6.00 sec 6.12 MBytes 51.3 Mbits/sec 0 63.9 KBytes [ 5] 6.00-7.00 sec 6.15 MBytes 51.6 Mbits/sec 0 63.9 KBytes [ 5] 7.00-8.00 sec 6.02 MBytes 50.5 Mbits/sec 0 63.9 KBytes [ 5] 8.00-9.00 sec 6.24 MBytes 52.3 Mbits/sec 0 65.2 KBytes [ 5] 9.00-10.00 sec 6.24 MBytes 52.4 Mbits/sec 0 65.2 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 61.6 MBytes 51.7 Mbits/sec 0 sender [ 5] 0.00-10.30 sec 61.6 MBytes 50.1 Mbits/sec receiver- OPNSense 21.1.1
Connecting to host 192.168.101.1, port 24683 [ 5] local 10.10.11.2 port 39164 connected to 192.168.101.1 port 24683 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 4.88 MBytes 41.0 Mbits/sec 0 64.4 KBytes [ 5] 1.00-2.00 sec 4.80 MBytes 40.2 Mbits/sec 0 64.4 KBytes [ 5] 2.00-3.00 sec 4.81 MBytes 40.4 Mbits/sec 0 65.7 KBytes [ 5] 3.00-4.00 sec 4.74 MBytes 39.8 Mbits/sec 0 65.7 KBytes [ 5] 4.00-5.00 sec 4.80 MBytes 40.2 Mbits/sec 0 65.7 KBytes [ 5] 5.00-6.00 sec 4.76 MBytes 39.9 Mbits/sec 0 65.7 KBytes [ 5] 6.00-7.00 sec 4.77 MBytes 40.0 Mbits/sec 0 65.7 KBytes [ 5] 7.00-8.00 sec 4.80 MBytes 40.3 Mbits/sec 0 65.7 KBytes [ 5] 8.00-9.00 sec 4.76 MBytes 40.0 Mbits/sec 0 65.7 KBytes [ 5] 9.00-10.00 sec 4.75 MBytes 39.8 Mbits/sec 0 65.7 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 47.9 MBytes 40.2 Mbits/sec 0 sender [ 5] 0.00-10.10 sec 47.8 MBytes 39.8 Mbits/sec receiver- IPFire 2.25 - Core Update 153
Connecting to host 192.168.101.1, port 5201 [ 5] local 10.10.11.2 port 38388 connected to 192.168.101.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 8.19 MBytes 68.7 Mbits/sec 14 149 KBytes [ 5] 1.00-2.00 sec 8.55 MBytes 71.8 Mbits/sec 2 131 KBytes [ 5] 2.00-3.00 sec 9.11 MBytes 76.4 Mbits/sec 0 169 KBytes [ 5] 3.00-4.00 sec 9.05 MBytes 75.9 Mbits/sec 1 147 KBytes [ 5] 4.00-5.00 sec 9.05 MBytes 75.9 Mbits/sec 0 182 KBytes [ 5] 5.00-6.00 sec 8.86 MBytes 74.3 Mbits/sec 18 140 KBytes [ 5] 6.00-7.00 sec 8.68 MBytes 72.8 Mbits/sec 8 84.0 KBytes [ 5] 7.00-8.00 sec 8.31 MBytes 69.7 Mbits/sec 0 132 KBytes [ 5] 8.00-9.00 sec 8.92 MBytes 74.9 Mbits/sec 28 121 KBytes [ 5] 9.00-10.00 sec 8.62 MBytes 72.3 Mbits/sec 0 160 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 87.3 MBytes 73.3 Mbits/sec 71 sender [ 5] 0.00-10.05 sec 86.7 MBytes 72.4 Mbits/sec receiverDuring next days I'm planning to test the same scenario with recommended v4.11.0.3 mainline BIOS.
-
RAW DATA FOR APU2 (apu2d4 - 3 Intel I210 ethernet) BIOS v4.11.0.3 mainline
WAN to WAN
- psSense 2.5.0
Connecting to host 192.168.1.1, port 5201 [ 5] local 192.168.1.2 port 50193 connected to 192.168.1.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 34.7 MBytes 291 Mbits/sec 0 113 KBytes [ 5] 1.00-2.00 sec 33.4 MBytes 280 Mbits/sec 0 113 KBytes [ 5] 2.00-3.00 sec 33.4 MBytes 280 Mbits/sec 0 113 KBytes [ 5] 3.00-4.00 sec 24.9 MBytes 209 Mbits/sec 1 1.41 KBytes [ 5] 4.00-5.00 sec 19.3 MBytes 162 Mbits/sec 1 1.41 KBytes [ 5] 5.00-6.00 sec 32.7 MBytes 275 Mbits/sec 0 129 KBytes [ 5] 6.00-7.00 sec 17.8 MBytes 149 Mbits/sec 0 127 KBytes [ 5] 7.00-8.00 sec 33.5 MBytes 281 Mbits/sec 0 129 KBytes [ 5] 8.00-9.00 sec 33.4 MBytes 280 Mbits/sec 0 129 KBytes [ 5] 9.00-10.00 sec 33.2 MBytes 278 Mbits/sec 0 129 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 296 MBytes 249 Mbits/sec 2 sender [ 5] 0.00-10.02 sec 296 MBytes 248 Mbits/sec receiver[2.5.0-RELEASE][root@pfSense.localdomain]/root: iperf3 -c 192.168.1.1 -P 10 Connecting to host 192.168.1.1, port 5201 [ 5] local 192.168.1.2 port 65370 connected to 192.168.1.1 port 5201 [ 7] local 192.168.1.2 port 53598 connected to 192.168.1.1 port 5201 [ 9] local 192.168.1.2 port 6033 connected to 192.168.1.1 port 5201 [ 11] local 192.168.1.2 port 33247 connected to 192.168.1.1 port 5201 [ 13] local 192.168.1.2 port 30494 connected to 192.168.1.1 port 5201 [ 15] local 192.168.1.2 port 41591 connected to 192.168.1.1 port 5201 [ 17] local 192.168.1.2 port 36782 connected to 192.168.1.1 port 5201 [ 19] local 192.168.1.2 port 10612 connected to 192.168.1.1 port 5201 [ 21] local 192.168.1.2 port 16764 connected to 192.168.1.1 port 5201 [ 23] local 192.168.1.2 port 53734 connected to 192.168.1.1 port 5201 ... ... ... - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 107 MBytes 89.8 Mbits/sec 0 sender [ 5] 0.00-10.03 sec 107 MBytes 89.5 Mbits/sec receiver [ 7] 0.00-10.00 sec 92.1 MBytes 77.3 Mbits/sec 0 sender [ 7] 0.00-10.03 sec 92.1 MBytes 77.0 Mbits/sec receiver [ 9] 0.00-10.00 sec 110 MBytes 92.5 Mbits/sec 0 sender [ 9] 0.00-10.03 sec 110 MBytes 92.2 Mbits/sec receiver [ 11] 0.00-10.00 sec 79.7 MBytes 66.8 Mbits/sec 0 sender [ 11] 0.00-10.03 sec 79.6 MBytes 66.6 Mbits/sec receiver [ 13] 0.00-10.00 sec 96.0 MBytes 80.5 Mbits/sec 0 sender [ 13] 0.00-10.03 sec 95.9 MBytes 80.2 Mbits/sec receiver [ 15] 0.00-10.00 sec 78.9 MBytes 66.1 Mbits/sec 0 sender [ 15] 0.00-10.03 sec 78.9 MBytes 65.9 Mbits/sec receiver [ 17] 0.00-10.00 sec 97.8 MBytes 82.0 Mbits/sec 0 sender [ 17] 0.00-10.03 sec 97.7 MBytes 81.7 Mbits/sec receiver [ 19] 0.00-10.00 sec 73.8 MBytes 61.9 Mbits/sec 0 sender [ 19] 0.00-10.03 sec 73.8 MBytes 61.7 Mbits/sec receiver [ 21] 0.00-10.00 sec 94.0 MBytes 78.8 Mbits/sec 0 sender [ 21] 0.00-10.03 sec 94.0 MBytes 78.6 Mbits/sec receiver [ 23] 0.00-10.00 sec 94.5 MBytes 79.3 Mbits/sec 0 sender [ 23] 0.00-10.03 sec 94.4 MBytes 79.0 Mbits/sec receiver [SUM] 0.00-10.00 sec 924 MBytes 775 Mbits/sec 0 sender [SUM] 0.00-10.03 sec 924 MBytes 772 Mbits/sec receiver iperf Done.OpenVPN (LAN2 to LAN1) AES128-CBC SHA1
- psSense 2.5.0
[2.5.0-RELEASE][root@pfSense.localdomain]/root: iperf3 -c 192.168.101.1 Connecting to host 192.168.101.1, port 5201 [ 5] local 10.10.11.2 port 1521 connected to 192.168.101.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 6.08 MBytes 51.0 Mbits/sec 0 63.5 KBytes [ 5] 1.00-2.00 sec 6.00 MBytes 50.3 Mbits/sec 0 63.5 KBytes [ 5] 2.00-3.00 sec 5.97 MBytes 50.1 Mbits/sec 0 63.5 KBytes [ 5] 3.00-4.00 sec 5.97 MBytes 50.1 Mbits/sec 0 63.5 KBytes [ 5] 4.00-5.00 sec 5.98 MBytes 50.2 Mbits/sec 0 63.5 KBytes [ 5] 5.00-6.00 sec 5.97 MBytes 50.1 Mbits/sec 0 63.5 KBytes [ 5] 6.00-7.00 sec 5.94 MBytes 49.8 Mbits/sec 0 63.5 KBytes [ 5] 7.00-8.00 sec 6.02 MBytes 50.5 Mbits/sec 0 63.5 KBytes [ 5] 8.00-9.00 sec 5.98 MBytes 50.2 Mbits/sec 0 63.5 KBytes [ 5] 9.00-10.00 sec 6.03 MBytes 50.6 Mbits/sec 0 63.5 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 59.9 MBytes 50.3 Mbits/sec 0 sender [ 5] 0.00-10.04 sec 59.9 MBytes 50.1 Mbits/sec receiverOpenVPN (LAN2 to LAN1) AES256-CBC SHA256
- psSense 2.5.0
[2.5.0-RELEASE][root@pfSense.localdomain]/root: iperf3 -c 192.168.103.1 Connecting to host 192.168.103.1, port 5201 [ 5] local 10.10.13.2 port 34867 connected to 192.168.103.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 5.35 MBytes 44.9 Mbits/sec 0 63.8 KBytes [ 5] 1.00-2.00 sec 5.27 MBytes 44.2 Mbits/sec 0 63.8 KBytes [ 5] 2.00-3.00 sec 5.21 MBytes 43.7 Mbits/sec 0 63.8 KBytes [ 5] 3.00-4.00 sec 5.19 MBytes 43.5 Mbits/sec 0 63.8 KBytes [ 5] 4.00-5.00 sec 5.20 MBytes 43.6 Mbits/sec 0 63.8 KBytes [ 5] 5.00-6.00 sec 5.15 MBytes 43.2 Mbits/sec 0 63.8 KBytes [ 5] 6.00-7.00 sec 5.15 MBytes 43.2 Mbits/sec 0 63.8 KBytes [ 5] 7.00-8.00 sec 5.17 MBytes 43.4 Mbits/sec 0 63.8 KBytes [ 5] 8.00-9.00 sec 5.14 MBytes 43.1 Mbits/sec 0 63.8 KBytes [ 5] 9.00-10.00 sec 5.04 MBytes 42.2 Mbits/sec 0 63.8 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 51.9 MBytes 43.5 Mbits/sec 0 sender [ 5] 0.00-10.17 sec 51.8 MBytes 42.7 Mbits/sec receiverOpenVPN (LAN2 to LAN1) AES128-GCM
- psSense 2.5.0
[2.5.0-RELEASE][root@pfSense.localdomain]/root: iperf3 -c 192.168.105.1 Connecting to host 192.168.105.1, port 5201 [ 5] local 10.10.15.2 port 25682 connected to 192.168.105.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 6.37 MBytes 53.4 Mbits/sec 0 63.2 KBytes [ 5] 1.00-2.00 sec 6.30 MBytes 52.9 Mbits/sec 0 63.2 KBytes [ 5] 2.00-3.00 sec 6.30 MBytes 52.8 Mbits/sec 0 63.2 KBytes [ 5] 3.00-4.00 sec 6.15 MBytes 51.6 Mbits/sec 0 63.2 KBytes [ 5] 4.00-5.00 sec 6.20 MBytes 52.0 Mbits/sec 0 63.2 KBytes [ 5] 5.00-6.00 sec 6.09 MBytes 51.1 Mbits/sec 0 63.2 KBytes [ 5] 6.00-7.00 sec 6.07 MBytes 50.9 Mbits/sec 0 63.2 KBytes [ 5] 7.00-8.00 sec 6.27 MBytes 52.6 Mbits/sec 0 63.2 KBytes [ 5] 8.00-9.00 sec 6.19 MBytes 51.9 Mbits/sec 0 63.2 KBytes [ 5] 9.00-10.00 sec 6.27 MBytes 52.6 Mbits/sec 0 63.2 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 62.2 MBytes 52.2 Mbits/sec 0 sender [ 5] 0.00-10.04 sec 62.2 MBytes 51.9 Mbits/sec receiver -
RAW DATA FOR APU2 (apu2d4 - 3 Intel I210 ethernet) BIOS v4.11.0.3 mainline
WIREGUARD VPN
- pfSense 2.5.0
[2.5.0-RELEASE][root@pfSense.localdomain]/root: iperf3 -c 192.168.106.1 Connecting to host 192.168.106.1, port 5201 [ 5] local 10.10.17.0 port 48366 connected to 192.168.106.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 4.59 MBytes 38.5 Mbits/sec 0 64.4 KBytes [ 5] 1.00-2.00 sec 4.51 MBytes 37.9 Mbits/sec 0 64.4 KBytes [ 5] 2.00-3.00 sec 4.52 MBytes 37.9 Mbits/sec 0 64.4 KBytes [ 5] 3.00-4.00 sec 4.52 MBytes 37.9 Mbits/sec 0 64.4 KBytes [ 5] 4.00-5.00 sec 4.50 MBytes 37.7 Mbits/sec 0 64.4 KBytes [ 5] 5.00-6.00 sec 4.50 MBytes 37.7 Mbits/sec 0 64.4 KBytes [ 5] 6.00-7.00 sec 4.49 MBytes 37.7 Mbits/sec 0 64.4 KBytes [ 5] 7.00-8.00 sec 4.48 MBytes 37.6 Mbits/sec 0 64.4 KBytes [ 5] 8.00-9.00 sec 4.49 MBytes 37.7 Mbits/sec 0 64.4 KBytes [ 5] 9.00-10.00 sec 4.32 MBytes 36.2 Mbits/sec 0 64.4 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 44.9 MBytes 37.7 Mbits/sec 0 sender [ 5] 0.00-10.23 sec 44.9 MBytes 36.8 Mbits/sec receiverIPSEC AES128 (SHA1)
- pfSense 2.5.0
[2.5.0-RELEASE][root@pfSense.localdomain]/root: iperf3 -c 192.168.107.1 Connecting to host 192.168.107.1, port 5201 [ 5] local 192.168.207.1 port 18656 connected to 192.168.107.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 5.89 MBytes 49.2 Mbits/sec 0 65.0 KBytes [ 5] 1.00-2.00 sec 7.37 MBytes 62.0 Mbits/sec 0 65.0 KBytes [ 5] 2.00-3.00 sec 7.39 MBytes 62.0 Mbits/sec 0 65.0 KBytes [ 5] 3.00-4.00 sec 7.43 MBytes 62.1 Mbits/sec 0 65.0 KBytes [ 5] 4.00-5.01 sec 7.38 MBytes 61.8 Mbits/sec 0 65.0 KBytes [ 5] 5.01-6.00 sec 7.35 MBytes 62.0 Mbits/sec 0 65.0 KBytes [ 5] 6.00-7.00 sec 7.34 MBytes 61.5 Mbits/sec 0 65.0 KBytes [ 5] 7.00-8.00 sec 7.35 MBytes 61.5 Mbits/sec 0 65.0 KBytes [ 5] 8.00-9.01 sec 5.67 MBytes 47.1 Mbits/sec 2 65.0 KBytes [ 5] 9.01-10.00 sec 7.21 MBytes 61.0 Mbits/sec 0 65.0 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 70.4 MBytes 59.0 Mbits/sec 2 sender [ 5] 0.00-10.28 sec 70.4 MBytes 57.4 Mbits/sec receiverIPSEC AES128-GCM (64bit)
- pfSense 2.5.0
[2.5.0-RELEASE][root@pfSense.localdomain]/root: iperf3 -c 192.168.107.1 Connecting to host 192.168.107.1, port 5201 [ 5] local 192.168.207.1 port 44304 connected to 192.168.107.1 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.01 sec 7.50 MBytes 62.5 Mbits/sec 0 65.0 KBytes [ 5] 1.01-2.00 sec 7.31 MBytes 61.5 Mbits/sec 0 65.0 KBytes [ 5] 2.00-3.00 sec 7.39 MBytes 62.2 Mbits/sec 0 65.0 KBytes [ 5] 3.00-4.00 sec 7.50 MBytes 62.6 Mbits/sec 0 65.0 KBytes [ 5] 4.00-5.01 sec 7.46 MBytes 62.5 Mbits/sec 0 65.0 KBytes [ 5] 5.01-6.00 sec 7.34 MBytes 61.8 Mbits/sec 0 65.0 KBytes [ 5] 6.00-7.00 sec 7.36 MBytes 61.8 Mbits/sec 0 65.0 KBytes [ 5] 7.00-8.01 sec 7.44 MBytes 61.6 Mbits/sec 0 65.0 KBytes [ 5] 8.01-9.01 sec 6.33 MBytes 53.5 Mbits/sec 0 65.0 KBytes [ 5] 9.01-10.01 sec 5.49 MBytes 45.9 Mbits/sec 2 65.0 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 71.1 MBytes 59.6 Mbits/sec 2 sender [ 5] 0.00-10.01 sec 71.1 MBytes 59.6 Mbits/sec receiver -
Anyone noticed there is a new service pcscd installed in pfSense 2.5.0?
https://forum.netgate.com/topic/161321/pcscd-pc-sc-smart-card-daemon -
@qinn said in PC Engines apu2 experiences:
Anyone noticed there is a new service pcscd installed in pfSense 2.5.0?
https://forum.netgate.com/topic/161321/pcscd-pc-sc-smart-card-daemonYes, I noticed that as well but I don't know what it's for.
-
@kevindd992002 https://www.freebsd.org/cgi/man.cgi?query=pcscd&sektion=8&manpath=freebsd-release-ports
-
@kevindd992002 said in PC Engines apu2 experiences:
I don't know what it's for.
on pfSense this is the real goal with it...
and plus from @viktor_g " support for PKCS#11 authentication (e.g. hardware tokens such as Yubikey) for IPsec: https://redmine.pfsense.org/issues/9878"
original:
https://pcsclite.apdu.fr/ -
@qinn said in PC Engines apu2 experiences:
When you are still using UFS it is best to move over to the filesystem ZFS
Another big advantage of ZFS is the snapshot feature. If you did a snapshot of your working 2.4.5 system before upgrading to 2.5 you can easily roll back to a fully installed and working 2.4.5 system in seconds if the bugs/issues/performance are causing you headaches.
This morning I rolled back. I’ll wait a while longer until the inevitable patches come out before thinking of upgrading again.
-
@vollans Maybe explain in detail how you did that step-by-step snapshot and the rollback, many are not familiar with copy-on-write systems like zfs and btrfs
-
This post is deleted! -
-
@logan5247 said in PC Engines apu2 experiences:
@qinn @Vollans - I also would be interested in hearing about this. I was under the impression ZFS wasn't really useful on a single-disk setup.
I have all my pfsense's on ZFS with a single Disk.
One day my UPS was going crazy (before she died) and switched power on / off in a short interval - the pfsense restarted, after I get a spare UPS in place, without any complains - is this good enough to show the advantages of ZFS?
Regards,
fireodoKettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
pfsense 2.8.0 CE
Packages: Apcupsd, Cron, Iftop, Iperf, LCDproc, Nmap, pfBlockerNG, RRD_Summary, Shellcmd, Snort, Speedtest, System_Patches. -
@logan5247 I’ll get it typed up in a couple of hours. It’s not tricky.
ZFS is a lot more robust that UFS and is definitely worth the effort to use. As it says above, a power failure is far less likely to cause loss of data, and a hard reboot in case of lockups again is highly unlikely to result in loss of data. And if you do snapshots before and after major upgrades you’re in a good place to revert if things go nuts.
-
@fireodo said in PC Engines apu2 experiences:
@logan5247 said in PC Engines apu2 experiences:
@qinn @Vollans - I also would be interested in hearing about this. I was under the impression ZFS wasn't really useful on a single-disk setup.
I have all my pfsense's on ZFS with a single Disk.
One day my UPS was going crazy (before she died) and switched power on / off in a short interval - the pfsense restarted, after I get a spare UPS in place, without any complains - is this good enough to show the advantages of ZFS?Regards,
fireodo....of course RAID gives more redundancy, best is more disks using RAID. As the problem with a single disk and "copies" is the same as creating an mdadm raid-1 using two partitions of the same disk: you have data redundancy, but not disk redundancy, as disk failure will cause the loss of both data sets.
Then ZFS, like btrfs, is copy-on-write, so power surges are never a problem and ZFS requires a system with ECC memory (APU2 has this), otherwise you're still not 100% safeguarded against bit errors.. -
@vollans said in PC Engines apu2 experiences:
@logan5247 I’ll get it typed up in a couple of hours. It’s not tricky.
Sorry, ended up being a couple of days due to events.
Log in via SSH
Check that you're using ZFS and what it's called, usually zroot
zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 2.43G 9.68G 88K /zroot zroot/ROOT 1.69G 9.68G 88K none zroot/ROOT/default 1.69G 9.68G 1.40G / zroot/tmp 496K 9.68G 496K /tmp zroot/var 751M 9.68G 401M /varTurn on listing of snapshots just to make life easier:
zpool set listsnapshots=on zrootDo your first snapshot - the bit after the @ sign is your name for the snapshot. I usually do a base, then @date or installed version similar:
zfs snapshot zroot@21-03-05 zfs snapshot zroot/ROOT@21-03-05 zfs snapshot zroot/ROOT/default@21-03-05 zfs snapshot zroot/var@21-03-05There is no point in snapshotting the tmp directory. It is normal to get no feedback from those commands.
Check you have a snapshot
zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 2.43G 9.68G 88K /zroot zroot@21-03-05 0 - 88K - zroot/ROOT 1.69G 9.68G 88K none zroot/ROOT@21-03-05 0 - 88K - zroot/ROOT/default 1.69G 9.68G 1.40G / zroot/ROOT/default@21-03-05 0 - 1.40G - zroot/tmp 496K 9.68G 496K /tmp zroot/var 752M 9.68G 401M /var zroot/var@21-03-05 1.48M - 401M -Do a snapshot whenever you make major changes, such as going from 2.4.5 to 2.5. Normal config changes it would be overkill for as the config backup would be enough.
You can remove your most recent snapshot with the commands, where the bit after @ is the snapshot name:
zfs destroy zroot/var@21-03-05 zfs destroy zroot/ROOT/default@21-03-05 zfs destroy zroot/ROOT@21-03-05 zfs destroy zroot@21-03-05If disaster strikes, otherwise known as 2.5, you can roll back to 2.4.5p1 working state by restoring the snapshot:
zfs rollback zroot/var@21-03-05 zfs rollback zroot/ROOT/default@21-03-05 zfs rollback zroot/ROOT@21-03-05 zfs rollback zroot@21-03-05 shutdown -r nowThe final line is vital! You MUST immediately reboot after rolling back the whole OS otherwise Bad Things Happen (TM).
There is also method in my madness of doing the rollback with /var first. If the var rollback fails, it's not the end of the world, and you can work out what you did wrong without putting the whole system at risk.
-
@vollans Thank you for the writeup. It's good to know this works.
Now please try pool checkpointing and let us know how it goes.
You'll need to boot from the installer and use the rescue image to recover.From the zpool Manual Page:
Pool checkpoint
Before starting critical procedures that include destructive actions (e.g zfs destroy ), an administrator can checkpoint the pool's state and in the case of a mistake or failure, rewind the entire pool back to the checkpoint. Otherwise, the checkpoint can be discarded when the procedure has completed successfully.
A pool checkpoint can be thought of as a pool-wide snapshot and should be used with care as it contains every part of the pool's state, from properties to vdev configuration. Thus, while a pool has a checkpoint certain operations are not allowed. Specifically, vdev removal/attach/detach, mirror splitting, and changing the pool's guid. Adding a new vdev is supported but in the case of a rewind it will have to be added again. Finally, users of this feature should keep in mind that scrubs in a pool that has a checkpoint do not repair checkpointed data.
To create a checkpoint for a pool:
# zpool checkpoint pool
To later rewind to its checkpointed state, you need to first export it and then rewind it during import:
# zpool export pool
# zpool import --rewind-to-checkpoint poolTo discard the checkpoint from a pool:
# zpool checkpoint -d pool
-
@dem I'll give that a go when I've got some spare time next week or when an updated 2.5 comes out.
-
@vollans I did a quick test in a virtual machine to figure out what the commands would be. This appears to work:
On a running 2.4.5-p1 system:
zpool checkpoint zrootBooted from the 2.5.0 installer and in the Rescue Shell:
zpool import -f -N --rewind-to-checkpoint zroot zpool export zroot poweroff -
@vollans Thanks for this write up! I am installed on UFS but may go back and switch to ZFS now. I'm a Linux guy, so ZFS has always been out of my wheelhouse.
When I do the initial setup and pfSense is working, do I:
- Perform a snapshot then and leave it around for years and years? Is this safe? I'm thinking like a VM snapshot where you don't want to have a snapshot hang around for long periods of time.
- Only perform snapshots before an upgrade, do the upgrade, then remove the snapshot after it's working?
Thanks again!
