how you setup Cache Proxy to a cache drive
-
ok i had issues i had a usb running pfsense the latest... and i had a hard drive to be the squid proxy cache sever thingy...
i followed the instructions off pfesnse forum but after a month or so the usb bricked.. no longer readable.. so im guessing it was caching off the usb instead.. so now i installed pfsense to one of my laptop 7200rpm hard drives so that wont happen.. but
how do i see the hard drive in pfsense and how do i know how to point to the cache drive... as i going to get a SSD as people talk tht you want a SSD for a cache drive..but anyone can tell me how i can set it properly that i know its caching to the right drive
-
Running across multiple drives is not directly supported in pfSense.
You can set the cache location in Squid so you can choose a different drive if it's mounted but you will need to ensure it's always mounted. You can modify /etc/fstab to do that but it may be overwritten at upgrade etc.
There are a number of other posts here details similar efforts though.
Steve
-
ah ok i just worried to run off USB again and then a hard drive as the cache and then brick the thing again.. reason i was making my 1Tb 7200rpm drive the pfsense boot.. and then i wanted a SSD for cache drive... i did read couple articles but i still couldnt make sure i pointing to the right spot there is no verficiation says your pointing to the drive.. but ill keep search again for another article
-
You can do it, but like stephenw10 was eluding to, you have to make some changes and prep/mount the drive from the pfsense machine's command line (or SSH). I wouldn't advise it though, I would just do a fresh reinstall on the SSD itself and everything will run great. You can copy your settings during install from your current HDD and you're system won't skip a beat. Ditch the spinning rust, and for the love of GOD do not run critical infrastructure from a USB flash drive. USB flash drives are the worst scraps of silicon ever, install pfsense on the SSD directly.
-
oh so you can just install right to a SSD now doesnt a SSD also have a short life span like the usbs as the data comes on and off it gets filled with electrons and bricks.. from videos i watched about the usb
so its safe then to install pfsense to SSD and no issues of caching to it.. i figured i had to have a 2nd drive for caching so it wouldnt slow your pfsense down??
reason i use my 7200rpm is when my old laptops break and i salvage what parts i can i trash so i used the 7200rpm 1tb drive.... i have no experience yet in SSDs at time i was looking there was SSHD which i think you might as well skip... and prices were too high and then i dont know what brand to go with.. and then these m2 ssds came out technology moving so fast..
but for a SSD i using an old computer a sata2 or sata3 port is there a brand or size you guys recommend to install pfsense with the cache... as my home internet is justa 5mbps so its not as fast as id like but i cant get much more in country
-
as i gotta goto the hospital tommorow for cancer follow ups ugh but ill be passing by the computer store so id pick up a SSD then
-
You can absolutely use an SSD, and yes it does have a "shorter" life span, but if you're using a newer SSD (like the Samsung 850/950 etc) then it shouldn't be an issue for the non-enterprise users. If you are a 50+ user network then consider getting the Intel Optane 900p, it offers significantly better endurance. You really don't want to use a physical hard drive for 2 main reasons, the first is LATENCY, hard drives are slow as f*** and have to spin up to seek the location of the cached data. This is an eternity when it comes to web browsing. The second is throughput, the PEAK data flow for most mechanical hard drives is about 100MBps which will BARELY saturate a 1Gbps network. A sata ahci ssd will hit 400-500MBs, and an NVMe drive will do even better. I'm caching a 1Gbs fiber wan line on a Samsung 960 evo NVMe SSD (which does 2.5GBs write/3.2GBs read) that is then fed to a 10Gbps X520 adapter to an enterprise switch etc etc.
EDIT: What are the specs for your pfsense box? I can make a recommendation if you'd like.
EDIT2: The speed of your internet doesnt matter as much, that's because once you have cached the data, the speed is entirely up to your network for whatever data is already cached. Perhaps throughput won't matter as much for your use case but LATENCY will, and hard drives are THE WORST for that. Even the smallest SSD is totaly worth it. You don't need much space, a Samsung 860 EVO is only $76 USD on amazon for 250GB which is WAY overkill for you. But the newer the SSD the better.
EDIT3 (lol): Another option is to use RAM for caching, Squid supports setting aside a chunk of RAM for caching and that is the BEST option for light web caching (it just won't persist through a reboot). If you have a lot of ram lying around and you can put at least 8GB in your pfsense box then set aside 4GB for caching instead, that will be the absolute fastest. Especially if basic web browsing cache is all you need.
-
ah ok ya i dont need anything fancy im just a home user few people of my family on the internet sharing 5mbps connection not that fast..
as i read you want a squid proxy for when downloading windows updates when 1 computer does it saves on the squid and then my other computer will just get it from there not bother getting from the internet... so i liked that idea..
how long do SSDs last for i always worried i have good stuff on there and one day is like on off switch works today gone tommorow and no chance in hell to retrieve...thats why i always wonder why do they use ssds in raid if it bricks good then its all screwed.. as i watched that on linus tech tips 3 Raid 5 on there one system... when 1 raid failed they all failed... and had to have special company get it.. its not like you can have warning realyl other then the smart.. i am trying unraid seems to be good and uses parity drive... but i going to have 2 computers 1 to back up the 2nd..but ya ill get a SSD is there a certain size i need do these have counters that is like a death clock you monitor well there is only 5 percent of life left so then i can learn to swap it out.. before the data becomes garbage
-
sorry damn dislexia you said 250gb
as for specs i had a Asrock board but everytime you reboot it looses the cmos and thats in warm reboot..
my sister got me a couple almost 10 yr old or older dell computers from college she works at... so i got pfsense running on it.. using old PCI not express network cards as it doesnt matter my internet slow lol
and the board only has i think 2 gig on it
i was going to scrap my Acer H340 home server and turn it into a Pfsense router its a low power cpu only draws 36 watts on the upsi was going to make a rocking server as i always turn my gaming machines into servers.. the Asus ROG boards...
-
Years and years of life for sure, most SSD's endurance is measured in Terabytes Written. Newer SSD's (samsung all the way) will be MORE than adequate for this level of use. Yes there is a lot of read/writes going on but for VERY small amounts of data. You will burn out a low end SSD by read/writing tons of BIG data, for example with Linus they are storing lots of video/audio on a NAS and read/writing from it constantly. Web files are tiny, and you should get A LOT of life out of a consumer SSD for your use case. With windows update caching, keep in mind that SSL items are not cached, you can set up a windows update cache but it does require extra steps (I have not done it personally though).
-
Any of those systems should be fine, put a new CMOS battery in that Asrock board and you wont have that issue anymore. That's like $5 USD at most. In an ideal world, just make sure the system is 64 bit, and the CPU ideally supports the AES-NI instruction set. (you can look that up on ark.intel.com, or cpu-world.com). You should make sure you are using a Gigabit NIC for your LAN, but your WAN could certainly be a 100Mbs NIC since your internet is so slow. In short, the "power" of the system isn't as important as how new it is. I would take a newer PC that was less powerful over an older more powerful machine for pfsense.
-
ah ok so ill see about a samsung i usually shop at
www.canadacomputers.com as its an 1.5 hours from me but has better options then staples and bestbuy..
and i planning on picking up a WD Gold 10TB drive finance over 6 months they offer since i want space and my older WD are starting to have errors least thats what Unraid is saying which i dont understand what they saying..
as for the extra network card in my dell comp its free and so is the PCI card was free had it kicking around so dotn need to build a computer but its a 100baseTX but i figure dont need 1000 since its just a DHCP server and internet.. as the files be accessing through the switch to my other computers and i planning to change my cabling to cat 6 or cat 7 sometimei did swap out the battery and it still didnt help i going to swap it out of my good machines as i think all the cmos batteries are the same least size wise and give it a shot
as the board isnt bad its defently newer id have to get u specs .. but im always using Asus Rog boards like Crosshair or Maxmiums or Rampage... i cant really afford supermicro or server grade boards to make my home NAS's it be nice but i dont think id benefit it as i just copy my photos and install programsso all my newer computers i built then turn into servers are all high power gaming ones basiclly quad 6 8 core cpus i know over kill for a pfsense
i was looking at getting a low power motherboard just for pfsense what do you use
-
Make sure the jumper to reset the CMOS is in the correct position, that could be the problem too and make sure the BIOS/UEFI is up to date. You don't need a server board, I have a pretty intense setup but even I am just using an ASUS H110M-E/M.2 . I want to upgrade it to a server board with ECC ram someday but realistically speaking it's not necessary for my use case, it's mostly just a quality of life thing and having a board with IMPI (remote KVM). My NAS on the otherhand DOES have a supermicro board with IPMI, C236 chipset and ECC ram, but that's because it needs to be reliable to protect my data etc.
Unraid is probably just reading SMART data that's saying your drive is starting to fail by producing read/write errors. I would replace it. Ideally I would go with several smaller drives for redundancy, I have six 2 TB drives in my NAS running in RAID-Z6. In theory I could have 2 drives fail and I wouldn't lose data.
Remember that Bits are not Bytes, 100 Mbps = about 10 MBps, and 1000 Mbps = about 100 MBps. Your 100BaseTX is fine for your internet since it is only getting 5 Mbps from the ISP, but as you're caching web content you can supply it back to your network at whatever speed your Cache source and NIC are capable of. The move from HDD to SSD for cache/OS install is really going to benefit you in terms of latency. Which is the real appreciable difference when it comes to general web browsing. The size of the files in websites is quite small, but the time it takes to process a request is the killer, with a HDD as a cache you may actually experience "slower" internet for general web browsing over no cache at all. The HDD would only really help on big downloads, but even then the HDD wouldn't be able to handle multiple requests very well as HDD's are very limited on IOPS on top of the latency itself. (IOPS = Input/output operations per second)
A great option is second hand "T" series intel CPU (i.e. i3-2120T vs i3-2120) they are usually only found in OEM systems and so you have to buy the CPU's from EBAY or 2nd hand stores/e-recyclers. But they are throttled down, lower TDP versions of the same CPU. The aforementioned i3-2120T is a 35 Watt cpu vs the standard i3-2120 is 65 watts. That allows you to grab ANY socket 1155 motherboard and run it at a MUCH lower power/heat setup which is perfect for pfsense.
EDIT: Not that it's super applicable but keep in mind that modern Intel Pentium/Celeron/i3's support ECC ram when used with a motherboard that supports it as well. It can be hit or miss on what non-server boards will let you run ECC ram with those CPU's but I have done it successfully before even with H-series chipsets etc. The downside to those CPU's is usually the lack of instruction set support like AES-NI & Vt-d. Among other performance limitations of course.
-
ah ok ill check later the motherboard jumper its just a pain in butt i tell ya but when you get for free cant complain lol.. and i seen this remote KVM how you do that i did read synology or synergery i forget.. Linus Tech tips showed it.. 4 computers 1 keyboard mouse on 1 screen and you can slide from each OS like it was 1 computer.. thats cool
but also lower drives start not easy to come by but i figured i slowly move to 10tb so id get another 10 and another 10.. right now i copying files from my freenas drive in unraid... and is RAID-Z6 thats just the data and 2 parity as i dont know tha Z6... right now in my server i using a Crosshair ROG Asus board.. and i have 1TB 2TB 2.5TB 6tbnas red pro.. mixed Green Black and Red Drives.. if you know much about the errors or reports i can shoot you an email comet424@msn.com maybe you can explain it better
this computer AES-NI CPU Crypto: No not sure what that does..
so i wantedto slowly upgrade my drives to bigger ones go with enterprise or Red pro as they basiclly the same..and when you need server grade memory and board.. how much that roughly cost.. i figuredi need that but even Acer Home Server H340 doesnt even use server grade ram..
i was looking at a Dell R710 computers from usedservers.ca but i also like those bays 24 hard drive bays.. and no point in going to SAS drives no one seems to carry them and they soo expensive too
i did see at canada computers mini itx board ll51 version but they dont sell the cpus for it or i have to goto a Gaming mini itx ll51 and go with the 8th gen version..
but i can also use my crosshair v mobo as the pfsense router.. i just try to keep hydro low since in ontario we pay more for hydro then anyone else.
so many options and for your data to be reliable what so special..
id like to make a server that.. but if you have 2 servers making copies is there no program to double check that the 2nd server has the file uncorrupted so you wouldnt need ecc ram and board -
I agree 100% with this.
Install on any decent SSD and you'll have no problems.
If it really concerns you you can remove SWAP and move /var and /tmp to RAM. But it shouldn't concern you. The write life of any reasonable quality SSD will far exceed whatever pfSense can do to it.
Steve
-
ah ok no i just asked because when i started to try it they talked about you install on a usb and then you install a ssd for cache... and ive never bought a ssd as they always been too expensive and i had laptop drives... so i have never even tried.. frig i have never tried water cooling cpus either...
but ill get a SSD 250 gig or 500gb i guess. i still figuring pfsense there is some things i like some other things i dont like.. like not being able to edit the dhcp leases names without having to add it at the bottom.. i wanna rename stuff like i can with my asus router...... plus i not sure what the AES-NI CPU Crypto but ill probably convert one of my asus gaming motherboard and cpus to it and forget this low power crap cpu ill just use like quad core... -
@squarecircle also for your important data you mentioned raidz6 which is that one and i read unraid not best for data checksum what you use to make sure it secure no corrupt files as i even seen flexraid as i want a secure data using pfsense want secure network from hackers or whoever and secure data or is windows server 2016 best way to go
-
AES-NI is an encryption cpu instruction set, instruction sets are like sets of computer code that are set in "stone" (silicon), this is more secure because the code cannot be altered or messed with as it is a physical structure of transistors. It's also faster to process things this way too. Pfsense community edition 2.5 (coming in the future) will require a CPU with AES-NI, so future proofing yourself by getting one now is a good idea. (See this article: https://www.netgate.com/blog/pfsense-2-5-and-aes-ni.htm )
So RaidZ6 is just the ZFS file system version of Raid 6, ZFS is just an awesome file system that has a lot built in to protect against bit rot and other rare issues and errors. It really protects data integrity, which is why I choose it with FreeNAS. Google it up and read up on ZFS, it's a next-gen file system like BTRFS which is also awesome but I've only used ZFS myself. This is really only relevant for long term data storage on a NAS or SAN.
Also Comet424, try and finish each sentence and organize your thoughts a little better, it can be hard to read and understand what you're trying to ask. I struggle with it a bit too as I have ADD, but just try work on it a bit.
-
@squarecircle
ah ok ya my ADD gets better of me
its because i read alot there is so much info.. frig i went to college in 2000 for network technician job or administrator graduated but there were no jobs it was so saturated. but since i let most of it slack not keeping up to date i find technology gone soooo fast beyond me lolso you dont use unraid at all... the reason i went to unraid i was getting annoyed with zfs i having issues i transfering files the data sets the i cant move files off my freenas i can copy to but cant move or delete as i dont have a unix user rights or something i forget the error windows cant delete or move its frustrating.. i think i may use it or windows server2016 or as a backup server of the more important data from unraid.. and i didnt like freenas if and when i moved files from the freenas to my windows machines the diskspace didnt get more in freenas it was taking up more space i was running out of diskspace yet i moved 500 gb off the freenas..
maybe its not setup right but i just got fed up and plex server doesnt update on its own and unraid does..
as for btrfs i read it talks about its bad to use not mature yet... and go with refs i dunno too many file systems when you look upbut ill try to look for a server grade least pfsense working on these old dell desktops and ill just use a gaming machine..
i picked up a ssd wd blue yesterday first SSD with a 12TB gold drive.. man those SSDs are soooooooooooooooo light i was like is there anything in there ... so i got a 250gb one to try man she light..
but if you can shoot me email too easier to talk sometimes i didnt even get notification you replied i was just checking -
@squarecircle
also if i wrote something you confused about above re ask what i ment.. to me it all makes sense but i know with dislexia it comes out wrong or i dont use enough periods..... etc it gets better of me.. for that ya i truly sorry (:
