PFSense Squid multiple appliances cache sharing

trinidadrancheria

We have 2 SG-4860 PFSense firewalls using carp failover with DNS round robin load balancing for outbound connections to the firewalls / internet.
We run standard proxy servers on both of them (regulatory reasons).
Each box has a primary WAN, Secondary WAN, LAN, SYNC interfaces.
What I am having issues with is how to get the proxy caches sharing / syncing data, so that when all is normal, either one can serve cached pages to the users. When one of them goes down, the other should take over.
I noticed that there is a nice tab on PFSense under Proxy server called Remote Cache, but for the life of me I cannot find any documentation on the tab or any tutorials on how to set it up in PFSense. There re TONS of comments on it on the web, but no how to.

Anyone have any guidance on this?

trinidadrancheria

Optimally I would like to mirror the cache on both devices however I would settle for just having caching being served from both both devices

aGeekhere

Squid General Settings ICP port

This is the port the Proxy Server will send and receive ICP queries to and from neighbor caches. Leave this blank if you don't want the proxy server to communicate with neighbor caches through ICP.

Would this help?

trinidadrancheria

here are the relevant settings from PFSense:
General:
Proxy port: 3128
ICP Port: 3130
Custom ACLs (before Auth)
external_acl_type mauth children-startup=2 ttl=600 grace=50 %SRC /root/mauth
acl PFProxy1 src xx.xx.xx.1 #(replaced True IP address for readability)
acl PFProxy2 src xx.xx.xx.2
acl mauth external mauth
http_access allow PFProxy1
http_access allow PFProxy2
http_access allow mauth localnet

The above is the same on Both PFSense boxes.
Remote Cache settings:
Hostname: xx.xx.xx.2 on box 1, xx.xx.xx.1 on box 2
tcp port: 3128
Allow Miss: Alow Miss

Hierchy: Sibling
Metod: Multicast-Sibling
ICP Port: 3130
ICP Options: Multicast Responder

In troubleshooting, I created a user called proxy1 and proxy2 to give access.
Authentication options: login=user:password (tried auth=off)

I see this in the squid logs:
TCP_MISS/403 http://pfsense:3128/squid-internal-dynamic/netdb from the IP address of box 2
and the same on box 2 with the address of box 1.

trinidadrancheria

Anyone?

darkfader

@trinidadrancheria:

Anyone?

I'm not yet even sure if i have it working. I can just see udp data going forth and back.
After what feels like an age my "master" now also shows an "ON" status. Last time I checked the other node did NOT show "ON", it just showed nthing.
One thing I have different is the select method (carp) and icp options (multicast-responder)

I've NOT set passwords.

I think the carp setting is questionable since I am using it with a loadbalancer / virtual server distributing the traffic to both proxies.
But considering how underdocumented + bug ridden this is, we're just testing our luck and this seems to be lucky.
(yes bug ridden, I'm not even getting logs after I set them to be stored outside /var because /var is a ramdisk. I feel noone tests anything)