Opinions about possibly overkill setup (Qotom i5 + 8GB + 500GB ssd)
-
This is similar to my wants too. no need for vpn. every hardware post on a new build a person wants full speed vpn. I couldn't figure the caching out, but didn't give it much time. I like the qotom systems, but want to go custom so i can put it in a rack and add the intel x540 10gb cards down the road to have internal 10gb. I'm waiting for the netgear prosafe 10gb switch to drop by a lot first…currently at like $650.
I was also wanting an independent Steam cache server.
I'll be following this thread.
-
If you don't need high speed VPN you'll likely be happy with an APU2 for a couple hundred all in. (Caveat: that assumes skipping squid, which is pointless on the modern internet and will slow you down regardless of your hardware. If you ignore that advice and decide to play with squid you'll probably end up throwing hardware at it before deciding it's pointless.)
edit: I didn't see pppoe, that dramtically increases the CPU requirement and rules out the apu2. side note: ISPs that do pppoe suck.
-
The i5 box is perfect just in case in the future you decide you do need VPN and as you ne d pppoe
Regarding caching I've found on my own 1gbps symmetric connection that it's actually slower to render webpages using a proxy as the files have to first be fetched and then served to your computer. That adds a tiny bit of latency and thto hit rates for normal web browsing in a home network are horrendous unless all your devices browse the same sites. Even then the browser itself will cache the most used files so for web browsing it's pointless, especially with a gigabit connection.
For large file transfers again I doubt its benefit. Your internal network will be 1 gig as your external so provided that your isp actually delivers gigabit throughout you will still have the same bottleneck, the 1gbps connection from the pfsense box to your lan. So pointless as well.
On that basis I'd recommend you go with the i5 box, 4GB ram and a 32 gig ssd. Anything more in the ram or drive is a waste of money with a gigabit connection.
If you want to remain on your 100 mbit connection the same aapplies regarding web browsing due to hit rates and added latency. however if you do download the same extra large files often you might benefit from a cache. If not it's a waste as by the time you cache and th cache expires you won't have downloaded it again. If you're just downloaded don't large files infrequently and want access to thes maybe a NAS is a better idea for a home use case.
-
On that basis I'd recommend you go with the i5 box, 4GB ram and a 32 gig ssd. Anything more in the ram or drive is a waste of money with a gigabit connection.
I have the Qotom i5 , and chose 8GB ram , mainlly because i might want to run Snort or Suricata at a later time.
I'd say go w. 8GB ram , else you'd end up with a 4GB "unusable" if you ever need 8'I installed a 240GB SATA SSD in my primary , that's prob. a serious overkill as i won't be using squid , and use an extrenal syslog server.
500GB … overkill *2 ;)
/Bingo
-
Well i just ordered mine today. I decided to go with the Qotom i5 + 8GB + 32GB SSD. I currently have 200/25 ISP and planning on moving to 1 GIG speed in coming months.
I would like to thank jgiannakes for his orginial post about this hardware and for his assistance with all my questions.
-Flamez
-
I know everyone is saying squid is useless, but what about in the case that his home has many users at once and no one device gets the full bandwidth of the 1gbit wan connection, does squid come in on top in that situation?
-
I know everyone is saying squid is useless, but what about in the case that his home has many users at once and no one device gets the full bandwidth of the 1gbit wan connection, does squid come in on top in that situation?
Squid is a local cache sitting on the PFSense box. Therefore the max speed it can communicate out to the LAN clients is at 1gbps aggregate, as that is the max speed of the ethernet LAN link. Hence if all the clients are hitting squid they will get a maximum shared bandwith of 1gbps, same as if they were all hitting the 1gbps WAN. So by adding the Squid proxy you just move the bottleneck from the WAN to the LAN (provided you have good latency and a decent ISP).
In my personal experience with Hyperoptic in London and their symmetric 1gbps service, I barely hit anything higher than 20-30mbps sustained. There are lots of spikes close to 50-100mbps especially when watching youtube videos etc but these die off in seconds (and these would not benefit from squid). Even when downloading a 10gigabyte game, its done within a few of minutes and does not materially disrupt anyone else on the network.
Finally when browsing the internet I hit spikes of 20-30mbps which die off in under half a second (and then you spend time reading the web page :)). So assuming a download time of 1 second at 20mbps, you can have 50 concurrent requests served at max speed. if you factor a "reading time" of say 30 seconds you can serve over 1000 clients before reaching the limits of the WAN. Similarly with a 100mbps connection the numbers would be 5 concurrent requests and 100 clients which is well above the scope of a home network.
However squid is useful when:
1. Your WAN speed is significantly slower than your LAN speed. Then Squid can serve more clients and go to the WAN less often, reducing the load on the WAN. A WAN of say 5-10mbps would see me implementing squid & traffic shaping just to relieve some pressure off it. But then I'd not be buying a core i5 bit an atom :)2. Your WAN has significantly higher latency than your LAN. On a 1gbps WAN or even 100mbps Fiber to Home you get latency of under 2-3msec to the WAN which if you reduce further will bring you no benefits on web browsing.
3. You are on a metered connection. Then I'd implement it in any case to reduce as much as possible the data usage on the WAN.
However, for 1 & 2 you will only see tangible benefit (i.e. high hit rates) if your LAN devices ask for the same resources from the WAN, i.e. browse a similar set of sites, download similar files etc over a short period of time (cache TTL). Browsers cache files (css, jpegs, png, etc) when downloaded for the first time, even across websites as long as they are a public shared resource (for example jquery files served from google's cache). Therefore they will not even go to Squid to ask for a file when a webpage is re-loaded as they can serve it from the local cache.
If you have a high bandwidth internet connection (anything over 100mbps) and at a home setting with 4-5 users and 10-15 internet devices, web browsing will not, in practice, benefit from squid as:
1. There is not enough overlap in requested resources between the devices/users
2. Where there is overlap, in web browsing, the resources are small enough to be downloaded near instantly. Most web pages are under 2-5 mbytes in size (resources and html) which would mean it would take under 0.5 secs to download them over a 100mbit connection and would usually take anything between 1-5 or more seconds to render depending on the speed of the device (mobiles take longer to render complex CSS). So even if the page downloaded instantly your overall speed increase would be practically imperceptible.
3. HTTPS content is by default not cached. As the internet moves more and more towards https encrypted websites Squids utility reduces (unless you implement MITM)
Where there is a tangible benefit is for large files that take time to download, are accessed frequently and your WAN connection is slow. Examples would be windows updates, game files etc. There you will see a good speed increase. However in all other cases you are best served by a NAS that can store large downloaded content for longer periods of time and you can just fetch it from there.
-
Thanks to everybody (and to jgiannakas in particular) for the supremely complete explanations in talking me out of squid.
Guess I'll go for a small and reliable ssd (any advice is welcome, by the way any reason to pick SATA 2.5" over msata? I guess re-usability and being able to choose a 2-bit MLC samsung 850pro…)
I see the qotom i5 broadwell (Q355G4) sometimes on amazon and other shops is pictured in two different shapes? Sometimes this broader one:
And sometimes in the narrower case like for the J1900 model.
I assume those are errors and the right format is the one above.The only thing I'm left wondering is why in 2017 this product is based on Broadwell 15W and not Kaby Lake 15W….maybe Broadwell chips are "leftovers" and this helps driving the price down? If that's the reason I'm ok with that...with 4 intel NICs, quality heatsink-case and the like it wouldn't be fair to compare it price-wise to cheap kabylake/apollolake boxes (aimed at HTPC) you can find these days...still...an Apollo Lake QOTOM would be interesting... (Apollo Lake is the first "Atom" with AES-NI)
-
(Apollo Lake is the first "Atom" with AES-NI)
That's not true. The J1900/Baytrail series was actually a stripped down implementation of the silvermont architecture (that is, the same architecture as the avoton/rangely atoms) which was distinguished largely by intel turning off the AES-NI functionality (presumably to segment the market). For airmont, they turned AES-NI back on in braswell, so the N3050/N3150/N3700 chips do have AES-NI and are basically J1900s with slightly more functionality. I have no idea why qotom never bothered to switch to braswell parts in the past 2 years. Apollo Lake is based on the new goldmont core, and also supports AES-NI. What's new is that it's a more efficient implementation (faster) and it adds RDSEED and SHA offloading instructions. I'd like to think qotom will eventually move to apollo lake across the board, it might take a while (both for cost reasons and because intel seems to just not be producing many of them).
-
I guess I'll have to invest in internal 10gbit nics sooner than expected then, to make squid worth while again. though in the mean time, i won't even be on gigabit wan speeds for a year or so.
-
(Apollo Lake is the first "Atom" with AES-NI)
That's not true. The J1900/Baytrail series was actually a stripped down implementation of the silvermont architecture..
I stand corrected, thanks.
I misinterpreted an Intel slide.Braswell and Apollo Lake fanless boxes with 2 Intel NICs would be interesting…I'd like to see a supercompact apollo lake box (barely bigger than a POE injector) with 4GB RAM and 32GB flash for like 149$....when both the modem and the WAP (and possibly a managed switch) are outsorced to different devices, no need for the router to be particularly big...
On the other hand gimme gimme gimme those sweet beefy 15W "real" big chip ULV CPUs...as long as they're fanless, sealed, dust proof and not that pricey anyway (thank you QOTOM)...
-
I ended up buying Q355G4, capable of 100Mbs over VPN, never got CPU more than 35% but runs at ~12 watts. Perhaps there is something out there that can achieve the same results with less power…. The box stays @ 44 Celsius open air and 47 Celsius in a closed encase with ambient temperature 24 Celsius.
link: https://forum.pfsense.org/index.php?topic=128206.msg732331#msg732331
Hardware: qotom Q355G4 with SSD
BIOS: hyper-threading disabled/
pfsense advanced: powerd enabled, AES-NI Enabled, Thermal = Intel Core
pfsense system tunable: sysctl dev.cpu.[0|1].cx_lowest=C3
Client: Gigabit connectivity
Packages: avahi installed
ISP Speed: 100 Mbs
VPN Provider: StrongVPN (AES 256 bit, MDS 128bit Auth, Adaptive compression)
Room Temperature: 24 Celsiusidle power consumption
powerd enabled, minimum, 10-11 watts, 52-53 Celsius, casing 44 Celsius
powerd enabled, adaptive, 10-11 watts, 52-53 Celsius, casing 44 Celsius
powerd enabled, maximum, 11-12 watts,54-57 Celsius, casing 45 Celsius
powerd disabled, 11-12 watts, 55-57 Celsius, casing 45 CelsiusConclusion: When idle the box seems to always need 10-12 watts regardless of the power mode. Is there better hardware out there that requires less wattage, support AES-NI, descent clock speed (Since OpenVPN is Single Core) and can provide 100Mbs output ? If you find one, share it with everyone in the forum !
Bandwidth test with ISP (no encryption) using Bandwidth Website
110 Mbs with powerd minimum, cpu <10%, 10-11 watts
110 Mbs with powerd adaptive, cpu <10 %, 11 watts,
110 Mbs with powerd maximum, cpu <10%, 11-15 watts peak
110 Mbs with powerd disabled, cpu <10%, 11-16 watts peakConclusion: My ISP seems to provide me with 100 Mbs download speed
Bandwidth test using ISP downloading 5-6 HUGE FILES simultaneously for a good period of time :)
110 Mbs with powerd minimum, cpu 20 %, 11 watts, CPU 52 Celsius, Router casing 44 Celsius
110 Mbs with powerd adaptive, cpu 20 %, 11 watts, CPU 52 Celsius, Router casing 44 Celsius
110 Mbs with powerd maximum, cpu 20%, 14 watts (weird), CPU 52 Celsius, Router casing 44 Celsius
110 Mbs with powerd disabled, cpu 20%, 11 watts, CPU 52 Celsius, Router casing 44 CelsiusConclusion: I can download 5-6 Huge files @ 110 Mbs regardless of the power saving mode because there is no encryption.
Bandwidth test using Internet through OpenVPN (encryption) using Bandwidth Website (tested a few times)
60 Mbs with powerd minimum, cpu <10 %, 10-11 watts,
110 Mbs with powerd adaptive, cpu <10 %, 11 watts,
110 Mbs with powerd maximum, cpu <10%, 11-15 watts peak
110 Mbs with powerd disabled, cpu <10%, 11-16 watts peakConclusion: powerd minimum doesn't seem to work well with OpenVPN….
Bandwidth test using Internet through OpenVPN (encryption) downloading 5-6 HUGE FILES simultaneously for a good period of time :)
50 Mbs with powerd minimum, cpu 25 %, 11 watts, CPU 52 Celsius, Router casing 44 Celsius
100 Mbs with powerd adaptive, cpu 35 %, 11-12 watts, CPU 54 Celsius, Router casing 45 Celsius
100 Mbs with powerd maximum, cpu 12%, 15 watts, CPU 63 Celsius, Router casing 46 Celsius
100 Mbs with powerd disabled, cpu 15%, 15.4 watts, CPU 64 Celsius, Router casing 47 CelsiusOverall Conclusion:Bandwidth is slightly affected by encryption (assuming good hardware & vpn provider). If you want to save energy & generate a bit less heat, you can perhaps consider using powerd adaptive mode. Perhaps the next generation of energy efficient Celeron might be a better choice for home if you don't plan to use pfsense packages that are CPU intensive.
-
I'm spying the same model, configured with 8GB and a 60GB MSATA. Looks to be about $340 shipped. I have 300/20, and am interested in having an always-on VPN for at least my laptops and phones. It would be my first venture into pfsense, and I honestly don't know all that much about it, but I figure that should hopefully get me about half my line speed with VPN… Plus I like to tinker anyway. Talk me out of it? Or pull the trigger?
Thanks!
-
I'm spying the same model, configured with 8GB and a 60GB MSATA. Looks to be about $340 shipped. I have 300/20, and am interested in having an always-on VPN for at least my laptops and phones. It would be my first venture into pfsense, and I honestly don't know all that much about it, but I figure that should hopefully get me about half my line speed with VPN… Plus I like to tinker anyway. Talk me out of it? Or pull the trigger?
Thanks!
I'd say tinker on an old system first, before dishing out 340 bucks.
-
I'm spying the same model, configured with 8GB and a 60GB MSATA. Looks to be about $340 shipped. I have 300/20, and am interested in having an always-on VPN for at least my laptops and phones. It would be my first venture into pfsense, and I honestly don't know all that much about it, but I figure that should hopefully get me about half my line speed with VPN… Plus I like to tinker anyway. Talk me out of it? Or pull the trigger?
Thanks!Depends about your knowledge in networks, your current home setup and the amount of time/effort you are willing to allocate to the project. Installing pfsense & configuring it for a simple home setup is fairly straight forward. After you are up & running, you need to be a bit more careful when you tinker with it ;)
For me, installing pfsense was a small pet project because my old router (DD-WRT) was barely capable of running OpenVPN. I decided to split the routing and the AP into separate devices. Today a high end wifi router is easily $200 with limited functionality/flexibility compared to pfsense router + AP.
Before you buy it, you can consider running it on a Virtual Machine on a desktop or laptop. If you only have laptops, buy a few USB network cards on aliexpress and you can easily build yourself a pfsense that you can tinker with. After you have it fully working & happy with it….. then order the real hardware :)
-
Another doubt I have: how hard_power_off-resilient would this pfsense box be?
I'll explain.
I have set up a panel/dashboard with on/off switches each able to cut the power to a device of the network, and each labeled accordingly (modem, switch, wifi zone controller, wifi base station 1, router, NAS, etc.). Actually I figured the NAS shouldn't be easily/accidentally powered down so I removed its switch and connected the NAS straight into the UPS. (all the devices are connected to the UPS eventually)
This is all meant for "brute force" troubleshooting by the house occupants when I'm away.
Now, with regular routers, 99% of the times you can do no damage by pulling the cord, they will just spring back to regular operation once powered again.
Would a pfsense router risk to be messed up (data corruption, permissions, read only, etc.) after a sudden (intentional or else) power loss? Maybe enterprise grade SSDs with real power loss protection (not the fake one found on some consumer SSDs) could be a solution?
Should I treat it like the NAS (no easy power off switch) or like the other "indestructible" embedded devices?
Being prone to power off data corruption (and consequent need for human re-configuring) would be quite a relevant minus compared to actual off the shelf enterprise-grade routers…
-
Another doubt I have: how hard_power_off-resilient would this pfsense box be?
I'll explain.
I have set up a panel/dashboard with on/off switches each able to cut the power to a device of the network, and each labeled accordingly (modem, switch, wifi zone controller, wifi base station 1, router, NAS, etc.). Actually I figured the NAS shouldn't be easily/accidentally powered down so I removed its switch and connected the NAS straight into the UPS. (all the devices are connected to the UPS eventually)
This is all meant for "brute force" troubleshooting by the house occupants when I'm away.
Now, with regular routers, 99% of the times you can do no damage by pulling the cord, they will just spring back to regular operation once powered again.
Would a pfsense router risk to be messed up (data corruption, permissions, read only, etc.) after a sudden (intentional or else) power loss? Maybe enterprise grade SSDs with real power loss protection (not the fake one found on some consumer SSDs) could be a solution?
Should I treat it like the NAS (no easy power off switch) or like the other "indestructible" embedded devices?
Being prone to power off data corruption (and consequent need for human re-configuring) would be quite a relevant minus compared to actual off the shelf enterprise-grade routers…
It is not as indestructible as a embedded router but its less prone to failure by brute force restarting than a NAS. I've tested it by pulling the plug a few times to check its robustness once I first installed pfsense on the quotum box and all was ok. However I now have it plugged in to my UPS alongside the access point, NAS and gigabit switch.
The reason why your occupants might try brute force troubleshooting is because a device hangs and it does not respond - that is quite common with traditional wifi-router combos. However with the PFSense box you shouldn't need to do this. I've seen multiple users here with uptimes of over 300+ days. The underlying OS is super stable and will not need rebooting due to crash/hang etc. Surprisingly also after converting my wifi-router to access point only, its stability has increased, to the extend that I have not had to reboot it in weeks!
So I'd recommend that you plug it in your UPS alongside your remaining networking equipment (NAS, switch, wifi access point) and maybe leave the modem on a normal plug in case that shows any signs of instability. If your modem is stable as well then plugging into the UPS as well will give you the added bonus of having internet available even in the event of a power cut ;)
-
Thanks.
Most of the times (like, once a month) the problem
is the modem. The modem needs to be power cycled, and the easiest way is to on/off it.
But then the router sometimes fails to "handshake" again with the rebooted modem and needs to be power cycled as well.So pfsense could end up needing more (tech savvy) human intervention than a regular off the shelf router…it would be interesting to use a server-grade SSD with actual power loss protection and do a "pull the cord 100 times" test and see if it survives with intact settings...I'd be surprised if somebody haven't already tried and studied all of this in depth given some stories I read of pfsense being deployed in delicate (even critical?) situations...
By the way during power outages in my neighborhood the fiber ONUs in the street cabinets are unpowered as well so I can't connect to the internet anyway :D
-
I wouldn't think the problem would be the hardware (SSD etc) but rather the software being interrupted while writing to disk and leaving it in an inconsistent state. Possibly the use of ram drive for var & tmp would help in your case. However before doing anything, I'd check if pfsense is having the same issue as your router in handshaking with the modem or whether that was some form of issue with the router - modem itself.
-
I use a Dell R210 II Xeon Quad with HT, 24 GB ram, 480 GB SSD
all used items I had laying around, can't even hear the R210 run in the rack
And I'm currently using a Broadcom NET EXTREME II 10GB SFP Network Card
WITH 1 GB fibre, ( WAN ) to a 24 Port 10GB SFP SWITCH ( LAN ) , House is wired with 10 Gb fiber
With my ISP I typically get anywhere from 980 - 1230 and it doesn't even break a sweat :-)Used Servers are always a great way to go… cheap and fast :-)
And if you're using your PF sense router with your ISP modem your ISP modem should be in bridge mode, as a reminder.
