Anyone actually got upstream proxy working on 2.0 using 2 pfSense boxes?

luke240778 · Sep 14, 2012, 12:26 AM

I have tried for ages to get this to work, and still not seeing results. has anyone actually managed to get upstream proxy working?

My setup is as follows, maybe someone can tell me where i am going wrong:

BOX1 - Main firewall (pfSense 2.0.1-RELEASE)
2 NIC's - WAN and LAN
Squid and Lightsquid installed and working in transparent proxy. Caching and reporting working.

BOX2 - Cache box (pfSense 2.0.1-RELEASE)
1 NIC -WAN
Squid and Lightsquid installed - Nothing working yet as i cant get the traffic from the first box to this one working.

Upstream Proxy settings on BOX1 i imput the IP of BOX2 and the Admin username and password just incase it was needed.

So, why am i not seeing any traffic making it through to BOX2?

marcelloc · Sep 14, 2012, 2:52 PM

What squid version are you using?

If you have only wan and lan on main firewall, how did you configured it to do not try to cache traffic from server 2?

With vlans you can configure a dmz without adding an extra interface to your firewall.

luke240778 · Sep 15, 2012, 4:42 AM

Hey marcelloc, i was actually going by some instructions that you told me in another thread a while ago. You said to just have 1 NIC (WAN) on the second box.

The second box is not used or connected to anything, its only as a cache server, trying to collect the cache from the first box.

Maybe i am trying to do it wrong.. i don't know? What i want is that all cache is stored onto the second box, as the first one does not have enough space on it, and i dont want to rebuild it as i am very happy with how it is currently working.

So, i am hoping that the second box can just store all the cache and logs, and Lightsquid.

I originally wanted to see how i can expand the size of my original pfSense install (VM on ESXi) but was told that expanding would be too hard, so better to rebuild.. which i dont want to do.

marcelloc · Sep 15, 2012, 7:11 PM

The cache server can have only wan. The problem is how to intercept all traffic but not from cache server.
with LAN Wan + DMZ on main firewall you can do this.

Second point is that sarg(I know you prefer lightsquid ;)) package now is able to compact log files to reduce disk usage as well protect reports display with pfsense auth.

luke240778 · Sep 16, 2012, 6:23 PM

@marcelloc:

The cache server can have only wan. The problem is how to intercept all traffic but not from cache server.
with LAN Wan + DMZ on main firewall you can do this.

Second point is that sarg(I know you prefer lightsquid ;)) package now is able to compact log files to reduce disk usage as well protect reports display with pfsense auth.

Ok good. then my cache server has only WAN which is correct.. the rest i don't really understand.. how do i get it so that the first box stores all cache on the second box properly?
I also don't have anything against Sarg, just haven't used it enough to know much about it.. Lightsquid at least gives me what i need.. just to see what IP as been using how much bandwidth, and what percentage has been taken from cache.

marcelloc · Sep 17, 2012, 2:27 PM

The setup I'm suggesting is this:

Your network –--->----LAN-----pfsense------WAN-------internet
||
||
OPT1
||
||
pfsense with squid

This way you can forward http data to second box with nat on first pfsense and squid in transparent mode on second box.

luke240778 · Sep 22, 2012, 4:32 AM

Ok i see.. so have a third virtual NIC (OPT1) that is just a connection from BOX1 to the cache box. so that would have to be another subnet i am guessing?

What settings do i need to set on both boxes? does squid and lightsquid need to be installed on both boxes? does transparent mode need to be activated on the first box also? or do i somehow get all data to pass through box 2?

marcelloc · Sep 25, 2012, 1:49 PM

@luke240778:

does squid and lightsquid need to be installed on both boxes?
does transparent mode need to be activated on the first box also?
or do i somehow get all data to pass through box 2?

You can use nat on main firewall to do the transparente forward or squid with parent proxy listening only on lan.
The second pfsense does not need transparent mode enabled as it will receive requests on 3128.

remx_james · Sep 27, 2012, 11:12 AM

Hi all,

with this SetUp architecture (like @marcelloc have explained) is it possible to activate authentication on the second proxy server (which is in DMZ) ?
In this manner, all web traffic incoming in the principal Pfsense router (set up with squid in transparent mode) will be forwarded to the second server (via upstream configuration) and this second proxy can apply restrictions on traffic based on user or group for example. (please stop me if i'm wrong).

My goal with this configuration architecture is to firstly offload the principal PFSense router particulary on cache management, and secondly avoid to configure any individual web browser (or use WPAC system), but still use authentication with Squid.

Thanks in advance!

marcelloc · Sep 27, 2012, 1:53 PM

With a transparent squid on first box you will not be able to authenticate. (ident may work but it's really easy to forge ident responses)

wpad, pac, client proxy settings are the way to use squid with auth.

remx_james · Sep 27, 2012, 5:38 PM

ah ook!!
i hav omitted this detail! ???

thanks

luke240778 · Sep 28, 2012, 2:35 AM

@marcelloc:

@luke240778:

does squid and lightsquid need to be installed on both boxes?
does transparent mode need to be activated on the first box also?
or do i somehow get all data to pass through box 2?

You can use nat on main firewall to do the transparente forward or squid with parent proxy listening only on lan.
The second pfsense does not need transparent mode enabled as it will receive requests on 3128.

If possible can you explain how to do this? like what settings and all that do i need? I dont really know how to do what you have mentioned.

marcelloc · Sep 28, 2012, 3:42 AM

On first Box, create a nat rule with
source LAN nat
destination any
Destination port 80
server ip second pfsense on dmz
Server port 3128

On second box enable squid.

luke240778 · Oct 1, 2012, 4:57 AM

@marcelloc:

On first Box, create a nat rule with
source LAN nat
destination any
Destination port 80
server ip second pfsense on dmz
Server port 3128

On second box enable squid.

So.. squid in transparent mode and all that setup on the first box? As in, all of the cache management settings have to be setup on the first box ?
Lightsquid setup on which box? If it has to be that cache and lightsquid need to be setup on the second box, how can i transfer all lightsquid logs from the first box to the second one? i would like that all of the data that i see now in my lightsquid reports, still be on the new setup, combined with all new data.

In the past i think you mentioned that in this setup squid needs to be installed on both boxes.. i just wonder where all the settings for cache management need to be setup, i would guess the second box, but i dont see how i can have squid enabled on the first box without any settings.