Squid video/gif/pics-cache for youtube/social media in 2019



  • there's plenty of posts on the forum but they're all old, referring to pre 3.5 squid versions, what's the situation now?

    I found a couple of github projects that made ID helpers for squid 3.x to handle this but those projects are also old.

    is it even possible to achieve this?
    generally speaking do you guys manage to get a decent amount of hits besides windows update?
    is it just enough to add some custom refresh_pattern rules or do we still need helpers and url_rewrite to make it work?

    I know social media dynamic content makes it pretty hard, still the same situation as years ago?

    specifically i was thinking about using this

    https://github.com/hscbrasil/hsc-dynamic-cache

    HSC Dynamic Cache delivers ready-to-use Squid-3.5 "Store ID" helper. To improve the cache keys, there is an URL patterns file which can be customized by the user.

    Here is the list of patterns included:

    IMDb trailers
    Sourceforge downloads
    R7 vídeos
    Metacafe
    MSN videos
    Terra TV
    Globo vídeos
    Facebook
    YouTube
    Google Maps
    

    Requirements

    Perl
    Squid >= 3.5
    squidGuard or equivalent url_rewrite_program (optional)


  • I gave up on squid caching HTTPS years ago. My hit rate was never better than 7%. I didn't have the time to immerse myself in squid to figure out how to handle dynamic requests, store_ids and all that other stuff. Besides, this isn't the 90's anymore and bandwidth is generally plentiful, so caching isn't an important requirement anymore. I use squid now as a base for squidguard so I can filter URLs.



  • i get that, it's also basically useless since the cdn are doing that job already, the data is served from the closest point, I also see youtube has some dynamic adaptation going on too and for sure squid would mess with it.

    still, i am getting a 0.03% hit rate with ssl inspection (full splice) enabled and working, I am a single user tho and it's been active for a day only.

    You use no custom_refresh at all? I browse reddit a lot, anything reddit related that you guys know of?



  • No, I don't use custom_refresh. I don't even have a disk cache defined anymore. Also, I'm using squid in explicit mode with WPAD, not transparent with SSL interception.



  • thanks, i suspected as much given what people post about Squid, it's use as a cache it's kinda limited nowadays.





  • @aGeekhere I'd be curious to hear what hit rates you're getting.



  • it really depends, but like 5 to 10%



  • Hmm, seems hardly worth the effort. I was getting 4-7% with a basic config. I suppose it depends a lot on where you're going, their content, etc.



  • It helps for OS updates, so if you have windows, mac os and linux. It would be really helpful if it could cache steam updates (but i have not been able to get that to work). My knowledge in refresh pattens is very limited, what would be an idea is have a community maintained custom refresh patten, where more skill users in the area could update the list and improve it. Maybe have an update button in squid to download the list when there is a new update.


Log in to reply