Hardware for transparent proxy server

[LeetDonkey]

Hello

I have a friend with a limited internet connection (500GB a month for a family of 5 with various uses, Facebook, Netflix etc.)

We were thinking about building him a pfsense box with a transparent proxy to cache as much content as possible to conserve bandwidth.

The line speed will at most be 70 mbit (Wireless internet so it can fluctuate alot)

I was thinking about something along theese lines:

J1900 with at least 2 x  Ethernet ports onboard (1 WAN, 1 LAN)

8 GB RAM

?GB SSD - Can anyone tell me what size would be realistic to use if you want to cache as much as possible? I was thinking about 256 or 512GB

Is the above sufficient for my description?

Anything else I need to keep in mind?

[Guest]

All would be perfect matching for you, please search the forum for Qotom J1900 it might be something around ~$260 with
8 GB RAM and 120 GB mSATA.

[LeetDonkey]

Thanks

I'll probably end up getting the Qotom since it is quite cheap, I just need to figure out the optimal amount of free SSD storage for squid I would need.

[VAMike]

@LeetDonkey:

I have a friend with a limited internet connection (500GB a month for a family of 5 with various uses, Facebook, Netflix etc.)

We were thinking about building him a pfsense box with a transparent proxy to cache as much content as possible to conserve bandwidth.
[…]
Anything else I need to keep in mind?

Be aware going in that the amount cached will be an insignificant fraction of the bandwidth consumed–a couple of hours of netflix watching will probably exceed what you save through caching in the course of the month. It was always a tough thing to make work well, and with sites increasingly either dynamic or HTTPS, there's a good chance that the effort involved with setting up a transparent proxy isn't worth the return.

[LeetDonkey]

@VAMike:

Be aware going in that the amount cached will be an insignificant fraction of the bandwidth consumed–a couple of hours of netflix watching will probably exceed what you save through caching in the course of the month. It was always a tough thing to make work well, and with sites increasingly either dynamic or HTTPS, there's a good chance that the effort involved with setting up a transparent proxy isn't worth the return.

Thanks - I'll keep that in mind, we might need to try a different approach then…
When the 500GB is spent the line speed will be capped at 1 mbit - When this happens I would assume that a proxy server would speed up things a bit if the files are available in the cache?

[VAMike]

@LeetDonkey:

When the 500GB is spent the line speed will be capped at 1 mbit - When this happens I would assume that a proxy server would speed up things a bit if the files are available in the cache?

Not really–the stuff that's likely to be cached would probably already be in the browser cache, and you'll be waiting on the dynamic & encrypted content every time you hit a web site anyway.

That said, 500GB is a pretty good bit of data. Before doing anything else I'd try to pin down how much is actually being used and on what, to see if this is a real problem at all, or if there's an obvious thing to get under control.

[KOM]

I agree with VAMike.  Caching has limited returns these days.  I use it more for URL filtering than caching.

[Guest]

pfSense with Squid & SquidGuard & SARG you may be better of Setting up;

• user accounts with logging all activities
• http proxy so no direct connect to the internet
• HAVP scanning for perhaps malware inside of websites
• caching might be also able to be tuned that only some files or things get cached and not all files
• media streaming such Netflix, amazon and others might be able to sort out of using the http proxy (squid)

So all in all it might be nice to have something likes squid, but with 120 GB of storage size it would be enough space
to handle that network traffic all. But together with snort and pfBlockerNG it should be a nice firewall with nearly served
UTM capabilities that will cut off the most plastic made home routers available on the market. So 500 GB is to much for that.

[asterix]

@BlueKobold:

pfSense with Squid & SquidGuard & SARG you may be better of Setting up;

• user accounts with logging all activities
• http proxy so no direct connect to the internet
• HAVP scanning for perhaps malware inside of websites
• caching might be also able to be tuned that only some files or things get cached and not all files
• media streaming such Netflix, amazon and others might be able to sort out of using the http proxy (squid)

So all in all it might be nice to have something likes squid, but with 120 GB of storage size it would be enough space
to handle that network traffic all. But together with snort and pfBlockerNG it should be a nice firewall with nearly served
UTM capabilities that will cut off the most plastic made home routers available on the market. So 500 GB is to much for that.

A bit out of topic but I believe SARG is not available under latest version of amd64 pfsense.

[LeetDonkey]

Hello again

Thanks for the input, I have some ideas to work with now, it seems my initial idea of setting up squid probably isn't the best solution.

I asked around to see if I could figure out what was using up the montly bandwidth and it seems at least one of the family members is very fond of torrents, not only downloading but also seeding.
This is probably a pretty bad idea when you have a monthly limit on your bandwidth.
He's not particularly interested in stopping his activity and we talked about setting up a dedicated connection for torrents.

The connection will be alot slower (2-5 mbit *DSL) but it will have unlimited usage.