FreeBSD 11.1 - Compressed ARC
11.1 is released now. Just noticed it now supports the compressed ARC feature. this affects both ARC and L2ARC, if I remember correctly.
that is indeed interesting, but much more so for servers than pfsense, on my pfsense unit ARC usage is extremely low given its mostly routing traffic not storage i/o activity.
My understanding of compressed ARC is LZ4 is so freaking fast that memory bandwidth is the bottleneck more than the CPU. Even all-managed C# code can give several GiB/s. Optimized C code is many factors faster if not a magnitude.
I actually tested LZ4 for handling web-requests for a project of mine. For large requests, it was actually faster to stream in the web response to a memory stream compressed with LZ4 than it was to leave it uncompressed. I assume smaller requests fit in cache and do not benefit. The break even point was around 128KiB. Never made it to prod because most of our responses are less than 128KiB and the increased complexity was not worth the minor increase in performance for our 99.9th percentile. It drastically complicated seeking, which we rarely need, but great for forward only streaming.