Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Hardware for transparent proxy server

    Scheduled Pinned Locked Moved Hardware
    10 Posts 5 Posters 2.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      LeetDonkey
      last edited by

      Hello

      I have a friend with a limited internet connection (500GB a month for a family of 5 with various uses, Facebook, Netflix etc.)

      We were thinking about building him a pfsense box with a transparent proxy to cache as much content as possible to conserve bandwidth.

      The line speed will at most be 70 mbit (Wireless internet so it can fluctuate alot)

      I was thinking about something along theese lines:

      J1900 with at least 2 x  Ethernet ports onboard (1 WAN, 1 LAN)

      8 GB RAM

      ?GB SSD - Can anyone tell me what size would be realistic to use if you want to cache as much as possible? I was thinking about 256 or 512GB

      Is the above sufficient for my description?

      Anything else I need to keep in mind?

      1 Reply Last reply Reply Quote 0
      • ?
        Guest
        last edited by

        All would be perfect matching for you, please search the forum for Qotom J1900 it might be something around ~$260 with
        8 GB RAM and 120 GB mSATA.

        1 Reply Last reply Reply Quote 0
        • L
          LeetDonkey
          last edited by

          Thanks

          I'll probably end up getting the Qotom since it is quite cheap, I just need to figure out the optimal amount of free SSD storage for squid I would need.

          1 Reply Last reply Reply Quote 0
          • V
            VAMike
            last edited by

            @LeetDonkey:

            I have a friend with a limited internet connection (500GB a month for a family of 5 with various uses, Facebook, Netflix etc.)

            We were thinking about building him a pfsense box with a transparent proxy to cache as much content as possible to conserve bandwidth.
            […]
            Anything else I need to keep in mind?

            Be aware going in that the amount cached will be an insignificant fraction of the bandwidth consumed–a couple of hours of netflix watching will probably exceed what you save through caching in the course of the month. It was always a tough thing to make work well, and with sites increasingly either dynamic or HTTPS, there's a good chance that the effort involved with setting up a transparent proxy isn't worth the return.

            1 Reply Last reply Reply Quote 0
            • L
              LeetDonkey
              last edited by

              @VAMike:

              Be aware going in that the amount cached will be an insignificant fraction of the bandwidth consumed–a couple of hours of netflix watching will probably exceed what you save through caching in the course of the month. It was always a tough thing to make work well, and with sites increasingly either dynamic or HTTPS, there's a good chance that the effort involved with setting up a transparent proxy isn't worth the return.

              Thanks - I'll keep that in mind, we might need to try a different approach then…
              When the 500GB is spent the line speed will be capped at 1 mbit - When this happens I would assume that a proxy server would speed up things a bit if the files are available in the cache?

              1 Reply Last reply Reply Quote 0
              • V
                VAMike
                last edited by

                @LeetDonkey:

                When the 500GB is spent the line speed will be capped at 1 mbit - When this happens I would assume that a proxy server would speed up things a bit if the files are available in the cache?

                Not really–the stuff that's likely to be cached would probably already be in the browser cache, and you'll be waiting on the dynamic & encrypted content every time you hit a web site anyway.

                That said, 500GB is a pretty good bit of data. Before doing anything else I'd try to pin down how much is actually being used and on what, to see if this is a real problem at all, or if there's an obvious thing to get under control.

                1 Reply Last reply Reply Quote 0
                • KOMK
                  KOM
                  last edited by

                  I agree with VAMike.  Caching has limited returns these days.  I use it more for URL filtering than caching.

                  1 Reply Last reply Reply Quote 0
                  • ?
                    Guest
                    last edited by

                    pfSense with Squid & SquidGuard & SARG you may be better of Setting up;

                    • user accounts with logging all activities
                    • http proxy so no direct connect to the internet
                    • HAVP scanning for perhaps malware inside of websites
                    • caching might be also able to be tuned that only some files or things get cached and not all files
                    • media streaming such Netflix, amazon and others might be able to sort out of using the http proxy (squid)

                    So all in all it might be nice to have something likes squid, but with 120 GB of storage size it would be enough space
                    to handle that network traffic all. But together with snort and pfBlockerNG it should be a nice firewall with nearly served
                    UTM capabilities that will cut off the most plastic made home routers available on the market. So 500 GB is to much for that.

                    1 Reply Last reply Reply Quote 0
                    • A
                      asterix
                      last edited by

                      @BlueKobold:

                      pfSense with Squid & SquidGuard & SARG you may be better of Setting up;

                      • user accounts with logging all activities
                      • http proxy so no direct connect to the internet
                      • HAVP scanning for perhaps malware inside of websites
                      • caching might be also able to be tuned that only some files or things get cached and not all files
                      • media streaming such Netflix, amazon and others might be able to sort out of using the http proxy (squid)

                      So all in all it might be nice to have something likes squid, but with 120 GB of storage size it would be enough space
                      to handle that network traffic all. But together with snort and pfBlockerNG it should be a nice firewall with nearly served
                      UTM capabilities that will cut off the most plastic made home routers available on the market. So 500 GB is to much for that.

                      A bit out of topic but I believe SARG is not available under latest version of amd64 pfsense.

                      1 Reply Last reply Reply Quote 0
                      • L
                        LeetDonkey
                        last edited by

                        Hello again

                        Thanks for the input, I have some ideas to work with now, it seems my initial idea of setting up squid probably isn't the best solution.

                        I asked around to see if I could figure out what was using up the montly bandwidth and it seems at least one of the family members is very fond of torrents, not only downloading but also seeding.
                        This is probably a pretty bad idea when you have a monthly limit on your bandwidth.
                        He's not particularly interested in stopping his activity and we talked about setting up a dedicated connection for torrents.

                        The connection will be alot slower (2-5 mbit *DSL) but it will have unlimited usage.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.