Cache/Proxy

• T

  Squid + https
  • techtester-m  

  52
  0
  Votes
  52
  Posts
  317
  Views

  

  @Abdou-Ahmed said in Squid + https: just add it in mikrotik well, please specify this, please what kind of Mikrotik??? I'm pretty prepared in the "picture" - Mikrotik... (all our CATV traffic is provided by Mikrotik devices)
• D

  Can't login to skype after using squid proxy
  • dealornodeal  

  8
  0
  Votes
  8
  Posts
  27
  Views

  

  @dealornodeal said in Can't login to skype after using squid proxy: ofcourse it may be hardware mailfunction or just MS Windows added some crappy things into BIOS management system.. pfSense / FreeBSD, cannot change the BIOS this is the fact... Under UEFI you can win, modification like EasyBCD... https://neosmart.net/EasyBCD/
• M

  SquidGuard Group ACL not working
  • mdalacu  

  23
  0
  Votes
  23
  Posts
  37
  Views

  M

  @coffeelover Hi ..in the morinig..same problem: For sure it is not a cron job because in WE, when nobody (but me :/ ) is at the office squid does not crash. I think some user is trying to access a site which crashes squid, but i can find who or what. Any ideeas? Thanks again. Aug 4 07:57:37 radiusd 28195 (48) Login OK: [DSP03] (from client T2600G-52TS_01 port 0 via TLS tunnel) Aug 4 07:57:37 radiusd 28195 (49) Login OK: [DSP03] (from client T2600G-52TS_01 port 3 cli f4-4d-30-6b-80-ce) Aug 4 08:41:47 php-fpm 338 /status_logs.php: Session timed out for user 'admin' from: 192.168.16.10 (Local Database) Aug 4 08:41:50 php-fpm 338 /status_logs.php: Successful login for user 'admin' from: 192.168.16.10 (Local Database) Aug 4 09:00:36 kernel pid 56671 (squid), jid 0, uid 100: exited on signal 6 Aug 4 09:00:37 kernel pid 84027 (squid), jid 0, uid 100: exited on signal 6 Aug 4 09:00:39 kernel pid 89031 (squid), jid 0, uid 100: exited on signal 6 Aug 4 09:00:40 kernel pid 92708 (squid), jid 0, uid 100: exited on signal 6 Aug 4 09:00:41 kernel pid 95868 (squid), jid 0, uid 100: exited on signal 6 Aug 4 09:00:42 kernel pid 98402 (squid), jid 0, uid 100: exited on signal 6 Aug 4 09:01:31 Squid_Alarm 21806 Squid has exited. Reconfiguring filter. Aug 4 09:01:31 Squid_Alarm 22203 Attempting restart... Aug 4 09:01:34 Squid_Alarm 24558 Reconfiguring filter... Aug 4 09:01:35 check_reload_status Reloading filter Aug 4 09:01:36 php-fpm 339 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'nat' rules. Aug 4 09:01:36 php-fpm 339 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'pfearly' rules. Aug 4 09:01:36 php-fpm 339 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'filter' rules. Aug 4 09:35:20 php-fpm 338 /pkg_edit.php: [squid] Clear disk cache forced via GUI. Clearing cache now... Aug 4 09:35:20 php-fpm 338 /pkg_edit.php: [squid] Stopping any running proxy monitors Aug 4 09:35:21 php-fpm 338 /pkg_edit.php: [squid] Creating cache dir '/var/squid/cache' ... Aug 4 09:35:21 php-fpm 338 /pkg_edit.php: [squid] Creating Squid cache subdirs in /var/squid/cache ... Aug 4 09:35:25 php-fpm 338 /pkg_edit.php: [squid] Starting service... Aug 4 09:35:26 php-fpm 338 /pkg_edit.php: [squid] Starting a proxy monitor script Aug 4 09:35:27 check_reload_status Syncing firewall Aug 4 09:35:27 php-fpm 338 /pkg_edit.php: [squid] - squid_resync function call pr:1 bp: rpc:no Aug 4 09:35:28 php-fpm 338 /pkg_edit.php: [squid] Adding cronjobs ... Aug 4 09:35:28 php-fpm 338 /pkg_edit.php: [squid] Antivirus features disabled. Aug 4 09:35:28 php-fpm 338 /pkg_edit.php: [squid] Removing freshclam cronjob. Aug 4 09:35:28 php-fpm 338 /pkg_edit.php: [squid] Stopping any running proxy monitors Aug 4 09:35:29 php-fpm 338 /pkg_edit.php: [squid] Reloading for configuration sync... Aug 4 09:35:29 php-fpm 338 /pkg_edit.php: [squid] Starting a proxy monitor script Aug 4 09:35:30 check_reload_status Reloading filter
• F

  how to Configure squidGuard
  • firefox  

  10
  0
  Votes
  10
  Posts
  32
  Views

  F

  I installed and config squid now squidguard is Works but It does not filter Are these settings correct ? this is proxy config # This file is automatically generated by pfSense # Do not edit manually ! http_port 192.168.1.1:3128 icp_port 0 digest_generation off dns_v4_first off pid_filename /var/run/squid/squid.pid cache_effective_user squid cache_effective_group proxy error_default_language en icon_directory /usr/local/etc/squid/icons visible_hostname localhost cache_mgr admin@localhost access_log /dev/null cache_log /var/squid/logs/cache.log cache_store_log none netdb_filename /var/squid/logs/netdb.state pinger_enable on pinger_program /usr/local/libexec/squid/pinger logfile_rotate 0 debug_options rotate=0 shutdown_lifetime 3 seconds # Allow local network(s) on interface(s) acl localnet src 192.168.1.0/24 forwarded_for on uri_whitespace strip acl dynamic urlpath_regex cgi-bin ? cache deny dynamic cache_mem 64 MB maximum_object_size_in_memory 256 KB memory_replacement_policy heap GDSF cache_replacement_policy heap LFUDA minimum_object_size 0 KB maximum_object_size 4 MB offline_mode off cache_swap_low 90 cache_swap_high 95 cache allow all # Add any of your own refresh_pattern entries above these. refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|?) 0 0% 0 refresh_pattern . 0 20% 4320 #Remote proxies # Setup some default acls # ACLs all, manager, localhost, and to_localhost are predefined. acl allsrc src all acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 3128 3129 1025-65535 acl sslports port 443 563 acl purge method PURGE acl connect method CONNECT # Define protocols used for redirects acl HTTP proto HTTP acl HTTPS proto HTTPS acl allowed_subnets src 192.168.1.0/24 http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access deny !safeports http_access deny CONNECT !sslports # Always allow localhost connections http_access allow localhost request_body_max_size 0 KB delay_pools 1 delay_class 1 2 delay_parameters 1 -1/-1 -1/-1 delay_initial_bucket_level 100 delay_access 1 allow allsrc # Reverse Proxy settings # Package Integration url_rewrite_program /usr/local/bin/squidGuard -c /usr/local/etc/squidGuard/squidGuard.conf url_rewrite_bypass off url_rewrite_children 16 startup=8 idle=4 concurrency=0 # Custom options before auth # Setup allowed ACLs # Allow local network(s) on interface(s) http_access allow allowed_subnets http_access allow localnet # Default block all to be sure http_access deny allsrc icap_enable on icap_send_client_ip on icap_send_client_username on icap_client_username_encode off icap_client_username_header X-Authenticated-User icap_preview_enable on icap_preview_size 1024 icap_service service_avi_req reqmod_precache icap://127.0.0.1:1344/squid_clamav bypass=off adaptation_access service_avi_req allow all icap_service service_avi_resp respmod_precache icap://127.0.0.1:1344/squid_clamav bypass=on adaptation_access service_avi_resp allow all
• A

  Guest VLAN and Haproxy/acme
  • archiBXL  

  4
  0
  Votes
  4
  Posts
  31
  Views

  A

  I was on summer break, hence the late reply. I guess I started with step 7. Thanks for your reply! I will give it a try.
• _

  Squidguard update fails after upgrading pfSense 2.4.5-RELEASE-p1
  • _igor_  

  49
  5
  Votes
  49
  Posts
  2881
  Views

  C

  I had the same issues. The cause seems to be the update process is waiting for input that is not expected. If i update via the command line and the updates hangs, i just press enter. The it continues.
• Y

  Squidguard-Target Category-Regular Expression broken?
  • yourname  

  1
  0
  Votes
  1
  Posts
  7
  Views

  No one has replied

• U

  Encypted Alert 21 (tls v1.2) - HAProxy issue
  • urbanovits  

  1
  0
  Votes
  1
  Posts
  3
  Views

  No one has replied

• 

  After upgrade to 2.4.5 HAproxy doesn't work by outside ( WAN) traffic
  • reza3sw  

  7
  0
  Votes
  7
  Posts
  96
  Views

  M

  i have the same problem, but doesnt seem to what reza3sw said. though seen i found it to stop working, i have had a change in internet service provider, and having a dual wan, i edited 1 of those for the new service. i did have a config backup a couple mpnths old that in a test i uploaded that on the new pfsense version, and it works.
• C

  [SOLVED ]Squid 0.4.44_25 / assertion failed: http.cc:1533: "!Comm::MonitorsRead(serverConnection->fd)
  • CaliPilot  

  8
  0
  Votes
  8
  Posts
  704
  Views

  M

  I have the same problem and it is driving me nuts. Every day when office hours begins, squid crashes with this error. On 2.4.4p3 suid was rock solid...:/ The only thing i could do from UI is to delete the cache and then squid starts, otherwise it will not start from services. I have no DNS issues. 2020-07-31 08:46:56 [45559] loading dbfile /var/db/squidGuard/Misc/domains.db 2020-07-31 08:46:56 [45559] logfile not allowed in acl other than default 2020/07/31 09:02:56 kid1| assertion failed: http.cc:1533: "!Comm::MonitorsRead(serverConnection->fd)" 2020/07/31 09:02:56 kid1| Starting Squid Cache version 4.10 for amd64-portbld-freebsd11.3... 2020/07/31 09:02:56 kid1| Service Name: squid 2020-07-31 09:02:56 [53246] (squidGuard): can't write to logfile /var/log/squidGuard/squidGuard.log 2020-07-31 09:02:56 [53246] New setting: logdir: /var/squidGuard/log 2020-07-31 09:02:56 [53246] New setting: dbhome: /var/db/squidGuard 2020-07-31 09:02:56 [53246] init domainlist /var/db/squidGuard/blk_blacklists_ads/domains 2020-07-31 09:02:56 [53246] loading dbfile /var/db/squidGuard/blk_blacklists_ads/domains.db 2020-07-31 09:02:56 [53246] init urllist /var/db/squidGuard/blk_blacklists_ads/urls Jul 31 09:02:56 kernel pid 43401 (squid), jid 0, uid 100: exited on signal 6 Jul 31 09:02:57 kernel pid 52412 (squid), jid 0, uid 100: exited on signal 6 Jul 31 09:02:58 kernel pid 55101 (squid), jid 0, uid 100: exited on signal 6 Jul 31 09:02:59 kernel pid 58638 (squid), jid 0, uid 100: exited on signal 6 Jul 31 09:03:00 kernel pid 61188 (squid), jid 0, uid 100: exited on signal 6 Jul 31 09:03:01 kernel pid 63750 (squid), jid 0, uid 100: exited on signal 6 Jul 31 09:03:17 Squid_Alarm 68674 Squid has exited. Reconfiguring filter. Jul 31 09:03:17 Squid_Alarm 68975 Attempting restart... Jul 31 09:03:20 Squid_Alarm 71372 Reconfiguring filter... Jul 31 09:03:20 check_reload_status Reloading filter Jul 31 09:03:22 php-fpm 28232 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'nat' rules. Jul 31 09:03:22 php-fpm 28232 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'pfearly' rules. Jul 31 09:03:22 php-fpm 28232 /rc.filter_configure_sync: [squid] Installed but not started. Not installing 'filter' rules. Help please..
• M

  The following error was encountered while trying to retrieve https://http/*
  • mzmrk  

  11
  0
  Votes
  11
  Posts
  7182
  Views

  C

  I put these in "Custom options (before auth)" And for complete filtering (URLs instead of domains) of SSL-Traffic via squidguard you have to set the mode to "Splice whitelist, bump otherwise". Splice: Do not break the SSL Connection Bump: Break the SSL Connection (Proxy CA on Clients needed)
• N

  Cannot operate with transparent option
  • nikpony  

  6
  0
  Votes
  6
  Posts
  14
  Views

  

  @nikpony said in Cannot operate with transparent option: WPAD functions are mainly for desktops and laptops, and pfBlocker for mobile apps etc, WPAD or PAC file for professional Squid setting. PfBlockerNG is suitable for everything, which is on your network and requests DNS from pfSense...ergo for everything use (like Pihole, I just think much better) Snort + OpenAppID can be perfect for restricting social sites BTW: Squid is "dying" due to the evolution of https and requires a lot of administration
• I

  How to configure squidGuard for HTTPS?
  • IamArobot  

  3
  0
  Votes
  3
  Posts
  127
  Views

  C

  I hope I can help you on some targets: For complete squidguard support even for ssl sites you have to use the mode 'Splice whitelist, Bump otherwise'. The sites in the whitelist will be spliced which means no MITM is happening. The filter (squidguard) only gets the domain and not the complete url. Other sites are bumped (MITM is happening), so you need the CA-certificate of the proxy ca installed on the clients. To set a real whitelist (don't try to filter the domains), you have to set url_rewrite_access deny whitelist in custom options. This means, that the a domain in the whitelist will not be checked by squidguard. Next topic: To get the redirect working even for SSL sites, you have to add the following settings to the custom options: url_rewrite_access deny CONNECT url_rewrite_access allow all This means, that the initial connect session will not go through the filter but the first page served will be filtered. (The browser only accepts a socket address like host:port in response to the connect. So a redirect url like http://pfsense/sgerror.php will just be parsed as the socket address http:80 (default port) and will throw an error.) So the complete block should be: url_rewrite_access deny whitelist url_rewrite_access deny CONNECT url_rewrite_access allow all I set these in the field 'Custom options (before auth)'. After applying these settings, the redirect should work. In our case it did not, because we only have 1 interface. We configured the redirect Mode to 'ext url move' using the sgerror.php: 'https://pfsense.domain/sgerror.php?url=403&a=%a&n=%n&i=%i&s=%s&t=%t&u=%u' HTH
• S

  Whitelist
  • Simbad  

  2
  0
  Votes
  2
  Posts
  31
  Views

  C

  The proxy config snippet generated from the whitelist uses the acltype dstdom_regex acl aclname dstdom_regex [-n] [-i] .foo.com ... # regex matching server [fast] # # For dstdomain and dstdom_regex a reverse lookup is tried if a IP # based URL is used and no match is found. The name "none" is used # if the reverse lookup fails. Even if it used dstdomain before, the asterisk character was invalid syntax and possibly ignored by the parser. Short answer: You have to adapt your whitelist or patch the code. For adapting your whitelist: The dot-Character has a special meaning in regex. If you want to include the '.' as in '.microsoft.com', you have to escape it: '.microsoft.com' (and: squid knows the end of the domain name, you don't have to append the '.' in the end) The correct migration would be: *.microsoft.com. → .microsoft.com
• B

  Haproxy - bind abstract namespaces addresses (abns@)
  • benoithcc  

  6
  0
  Votes
  6
  Posts
  16
  Views

  B

  Thanks for your help @PiBa everything is clear to me now! Benoit
• X

  Squid transparent proxy + SquidGuard error "transaction-end-before-headers"
  • xalex1977  

  17
  0
  Votes
  17
  Posts
  772
  Views

  C

  The error in your log seems to be a CONNECT issue. The Browser opens a CONNECT session to the target site and will only accept a socket address, not a URL. The Rewrite URL from squidguard https://site.com/sgerror.php is parsed as a socket address like host:port We have squid with SSL MITM, ClamAV and Squidguard with correct url redirect working with the following setting: squid mitm: splice whitelist, bump otherwise additional advanced options: url_rewrite_access deny CONNECT url_rewrite_access allow all This will deny CONNECT sessions for non-whitelisted sites and will let the redirect work. As redirect function in squidguard you need to set "ext url move", not redirect.
• M

  This topic is deleted!
  • manu77  

  2
  0
  Votes
  2
  Posts
  2
  Views

  No one has replied

• Y

  Allow facebook messenger but block facebook website
  • yanafig  

  1
  0
  Votes
  1
  Posts
  10
  Views

  No one has replied

• A

  Enable SSL filtering with mikrotik router
  • Abdou Ahmed  

  1
  0
  Votes
  1
  Posts
  8
  Views

  No one has replied

• W

  HAProxy backend over VPN
  • wickeren  

  1
  0
  Votes
  1
  Posts
  16
  Views

  No one has replied

• M

  Random problems accessing HTTPS pages error PR_END_OF_FILE_ERROR
  • moti4553  

  2
  0
  Votes
  2
  Posts
  38
  Views

  M

  @moti4553 help @stephenw10 Dear we still present the problem, I already put DNS to IP 127.0.0.1, but the problem there is that I do not see my internal applications, please could you give me a light
• R

  Ombi + Haproxy stuck on loading
  • rekd0514  

  19
  0
  Votes
  19
  Posts
  107
  Views

  R

  @valentinius You just use the IP and port of the device you are running Radarr on and the API key from Radarr is put into Ombi as well.
• S

  Old data
  • Simbad  

  2
  0
  Votes
  2
  Posts
  22
  Views

  

  @Simbad Hi, This is not a problem, in fact, they have been in this version for a long time. AV filtering on the firewall doesn't really have a reason for existence, especially since it only applies to http the daily.cld is updated and this is the essential momentum http://database.clamav.net/main.cvd http://database.clamav.net/daily.cvd http://database.clamav.net/bytecode.cvd https://www.freshports.org/www/squidclamav
• C

  Haproxy and smugling fix
  • cjbujold  

  1
  0
  Votes
  1
  Posts
  7
  Views

  No one has replied

• D

  Cache Poisoning Issue in HTTP Request processing
  • dreamworld  

  1
  0
  Votes
  1
  Posts
  29
  Views

  No one has replied

• H

  How to update ClamAV
  • hugoeyng  

  13
  0
  Votes
  13
  Posts
  114
  Views

  H

  @Bismarck Worked!
• A

  QoS3: Secure Caching in HTTPS Based on Fine-Grained Trust Delegation
  • aGeekhere  

  1
  0
  Votes
  1
  Posts
  14
  Views

  No one has replied

• 

  in an effort to better fix/set up squid and the github information for others to use, I need some help understanding stuff
  squid mitm explicit proxy ssl inspection transparent • • High_Voltage  

  4
  0
  Votes
  4
  Posts
  45
  Views

  

  As soon as you have access to the full, decrypted data stream it's most probably possible to cache everything. But : The, for example, ccs style sheet file, can have a unique name - and won't be re used ever again, so it will get reloaded anyway. The file creation date can be set to 'now' so the browser will request a fresh copy, even if the content didn't change at all. etc etc .
• T

  Squid certificates
  • techtester-m  

  2
  0
  Votes
  2
  Posts
  32
  Views

  

  See https://forum.netgate.com/topic/155280/squid-https/3
• 

  new fun and odd issue with squid/wpad on pfsense with android!
  • High_Voltage  

  4
  0
  Votes
  4
  Posts
  12
  Views

  

  @High_Voltage said in new fun and odd issue with squid/wpad on pfsense with android!: These .microsoft.com .windowsupdate.com .akamaitechnologies.com .akadns.net should not (never) be cached. Example : if the windows update isn't guaranteed to from "the source" then every windows install is at risk. Microsoft couldn't tolerate that situation, it could kill the company overnight. So this acl splice_it ssl::server_name .reddit.com handles everything going to / coming from is handled the same way. ( no need to read a a manual to understand that ^^ )
• Y

  Squid local users
  • yanafig  

  13
  0
  Votes
  13
  Posts
  140
  Views

  Y

  @viktor_g That solves my problem on Version 2.4.5-RELEASE-p1 (amd64) built on Tue Jun 02 17:51:17 EDT 2020 FreeBSD 11.3-STABLE I guess I need to upgrade the Version 2.3.5-RELEASE-p2 (i386) built on Thu May 10 15:03:18 CDT 2018 FreeBSD 10.3-RELEASE-p29 Thank you for you patience and support
• A

  Squid's new SslBump Peek and Splice for https caching?
  • aGeekhere  

  2
  0
  Votes
  2
  Posts
  40
  Views

  

  @aGeekhere said in Squid's new SslBump Peek and Splice for https caching?: SslBump Peek and Splice I had some time this morning, so I decides that it was time for trying to understand what this 'peek','splice' and 'bump' is all about. World's biggest search engine was wilkling to help me : what does bump or splice mean ? A second link was asking that very question pointing to the first link, which was your https://wiki.squid-cache.org/Features/SslPeekAndSplice ... I read, as explained, the article 3 times. I retained Pick your poison , something I did understand. Btw : caching web pages is something from the past. These days, a web page is unique, fabricated for you, being valid at that moment, and meaningless, useless afterwards. A browser doesn't cache https pages, a web server doesn't neither. Any one in between is having a hard time. edit : sorry, I'm not answering your question, was just trying to understand it.
• M

  Using HAproxy for internal web servers
  • m0nKeY  

  6
  0
  Votes
  6
  Posts
  65
  Views

  

  @Astraea For me, that requires I maintain the certificates on HAProxy and the web servers themselves. That's why I tell HAproxy to listen on an internal VIP and use that for my DNS host overrides. Inside and outside connections go to the same frontend but without crud like NAT reflection.
• A

  squid + captive portal authentication
  • AcidTech  

  2
  0
  Votes
  2
  Posts
  36
  Views

  

  Redmine issue created: https://redmine.pfsense.org/issues/10749
• L

  HAProxy Slow
  • Lars_LE  

  1
  0
  Votes
  1
  Posts
  11
  Views

  No one has replied

• C

  Does Squid support 2020 LDAP channel binding ?
  • CZvacko  

  22
  0
  Votes
  22
  Posts
  506
  Views

  C

  Hi, I tried to use mixed mode and it works as expected (including LightSquid).
• T

  How to clear & clean squid logs safe?
  • thanhlangso  

  6
  0
  Votes
  6
  Posts
  426
  Views

  

  I admit that this is one of the problems with Squid as a package with pfSense, it doesn't rely on automatic log rotation and clean-up. I still do it manually, but if there is a software that can do this automatically, I'm in.
• P

  Have haproxy pass a variety of request types
  • pfdog  

  4
  0
  Votes
  4
  Posts
  45
  Views

  P

  @dragoangel Thank you, you are correct. I am my way to becoming versed with HAProxy.
• S

  HAproxy slow on WAN jagged throughput
  • S_m  

  4
  0
  Votes
  4
  Posts
  74
  Views

  

  You have TCP retransmissions... Where you see retransmission's - between pfSense and backend? between pfSense and outside client or internal client? Why you see retransmission? This you need to explain, this isn't normal flow and I doesn't think they flow can be tuned as it not designed to drop packages . Why you speak about ISP? What service you try to proxy? Do you see errors in HAproxy logs or in haproxy socat status?
• 

  HAProxy Not Working With Zammad
  • yuljk  

  3
  0
  Votes
  3
  Posts
  44
  Views

  

  Layer 4 connection problem - connection refused is clearly says that port you mention doesn't open or listen. So: or you have firewall issues on your server or app not listen on server IP you are pointing.