PfSense newbie configuration problem

podilarius

If you are going to get NAT going, you need to uncheck the option to disable all firewall filtering. This will turn NAT back on so that you can use Automatic NAT.

The thing is that every router behind your public IP (which is doing the main NAT), is going to have to know how to route 192.168.83 to your pfSense machine. Without that, you are not going to get this working. (IF you are not NATing)

okay, so the main confusion is if you are going to NAT or not, firewall or not. Once you let us know, then we can help further. Otherwise, we are going to talk in generalities to help you make up your mind on NATing or not. It can be done either way, its just that the config is very different.

vdecristofaro

@podilarius:

…
The thing is that every router behind your public IP (which is doing the main NAT), is going to have to know how to route 192.168.83 to your pfSense machine. Without that, you are not going to get this working. (IF you are not NATing)
...

Well… I thought to this very very long time, and at the end, came to the decision to use NAT (mainly because I cannot ask nobody to configure routers behind my WAN...)
Which parameters should I setup?

:)

podilarius

Okay … in advanced setup -> uncheck the option to disable firewalling. Save and apply.
Then head to firewall -> advanced outbound NAT and select auto. Save and apply.
After that, head to firewall -> rules -> LAN. Setup a rule to allow any protocol with source LAN subnet to Any/Any. Save and apply.
Then go to Services -> DHCP and enable that on LAN. give is a range like 192.168.83.50-250. save and apply.
reboot the FW.

Get on a machine behind the FW and trace route to www.google.com and see how far your get.

vdecristofaro

@podilarius:

Okay … in advanced setup -> uncheck the option to disable firewalling. Save and apply.

Done!

@podilarius:

Then head to firewall -> advanced outbound NAT and select auto. Save and apply.

Do you mean the tab Firewall/NAT on the Advanced Setup page?

I went to System - > Advanced Setup and then to the Firewall/NAT tab.
These are my settings now

@podilarius:

After that, head to firewall -> rules -> LAN. Setup a rule to allow any protocol with source LAN subnet to Any/Any. Save and apply.

Here it is!

@podilarius:

Then go to Services -> DHCP and enable that on LAN. give is a range like 192.168.83.50-250. save and apply.
reboot the FW.

Done!

@podilarius:

Get on a machine behind the FW and trace route to www.google.com and see how far your get.

I have tried. Now i am able to traceroute any public server. I can see that, respect to traceroute when done from the HOST, now the first hop is the pfSense router itself. Very good. Unfortunately anyway I am not able to navigate on the internet. I have used Fiddler to see what's going on and I can see that the request terminate with HTTP 502 - Connection Failed.

I suspect that is the proxy server…

podilarius

@vdecristofaro:

Do you mean the tab Firewall/NAT on the Advanced Setup page?

Yes I do. Not the Advanced in system setup.

Error 502 does seem to be a proxy stopping the connection (lists as bad gateway).

Check your DNS settings on the FW to make sure you websites are resolving correctly. Can you get to a page that other Clients can get to as well?

vdecristofaro

@podilarius:

@vdecristofaro:

Do you mean the tab Firewall/NAT on the Advanced Setup page?

Yes I do. Not the Advanced in system setup.

Error 502 does seem to be a proxy stopping the connection (lists as bad gateway).

Check your DNS settings on the FW to make sure you websites are resolving correctly. Can you get to a page that other Clients can get to as well?

For other reasons this morning I was trying to install wget.
So from the pfSense box I've choosen "8 - Shell" and then:

Went to /etc/csh.cshrc to setup http_proxy, https_proxy and ftp_proxy environment variables
restarted the box
Again to the shell I have tried to install wget with the command /usr/sbin/pkg_add -r wget

The result really surprised me.


Error: Unable to get to ftp://ftp.freebsd.org/......: No address record
pkg_add: Unable to fetch 'ftp://ftp.freebsd.org/.....' by URL

this seems to be a nameserver problem. Right? my resolv.conf file seems to have right values anyway

domain localdomain
nameserver 127.0.0.1
nameserver 10.182.209.132
nameserver 10.254.49.133
nameserver 8.8.8.8
nameserver 8.8.4.4

I dont understand what the "domain" entry and the "nameserver 127.0.0.1" are for, but I guess their presence shall not be a problem….Do I am right?

johnpoz

Ok here is the thing - you mention your in a large network right.

Large networks quite often do a few things for security reasons. 1 they normally only allow a proxy internet access, so to get to the internet you have to use that proxy. Direct internet access is blocked - only the proxy is allowed internet access.

2 they normally block dns, your local dns you point to - ie those 10.182 and 10.254 IPs more than likely don't even resolve public domains. quick enough to check, using nslookup or dig - do a query to them directly for outside something, www.google.com, ftp.freebsd.org, etc. Along with not allowing the local nameservers to resolve public domains they normally do not allow you to query outside dns, ie the those googledns you have there at 8.8.8.8

This is common practice for corp networks. Why you would need to run a router/firewall inside your corp network without support and details from corp IT is beyond me.

But unless you bounce off your corp proxy is more than likely your never getting off the corp network.

So does your workstation have internet access? If so look to see what proxy your browser is pointing to.

you state

"Now i am able to traceroute any public server"

Lets see this! And what are you using for your dns? You doing this from your PC or pfsense? Please post the details of where you doing this traceroute ip address, gateway and nameserves - along with the full trace to www.google.com – here is example


C:\Windows\system32>tracert www.google.com

Tracing route to www.google.com [74.125.225.210]
over a maximum of 30 hops:

  1     2 ms    <1 ms    <1 ms  pfsense.local.lan [192.168.1.253]
  2    28 ms    28 ms    29 ms  c-24-13-176-1.hsd1.il.comcast.net [24.13.176.1]
  3    13 ms    10 ms    11 ms  te-1-2-ur07.mtprospect.il.chicago.comcast.net [68.85.131.149]
  4    11 ms    11 ms     9 ms  te-8-4-ur08.mtprospect.il.chicago.comcast.net [68.86.187.202]
  5    16 ms    15 ms    15 ms  te-1-2-0-5-ar01.area4.il.chicago.comcast.net [68.87.230.53]
  6    15 ms    23 ms    23 ms  pos-3-10-0-0-cr01.350ecermak.il.ibone.comcast.net [68.86.93.181]
  7    12 ms    14 ms    16 ms  pos-1-8-0-0-pe01.350ecermak.il.ibone.comcast.net [68.86.87.166]
  8    21 ms    12 ms    13 ms  66.208.228.202
  9    13 ms    13 ms    28 ms  209.85.254.128
 10    13 ms    13 ms    27 ms  72.14.237.130
 11    25 ms    23 ms    24 ms  209.85.241.22
 12    66 ms    33 ms    34 ms  72.14.239.49
 13    32 ms    34 ms    37 ms  216.239.46.149
 14    33 ms    33 ms    34 ms  209.85.251.111
 15    34 ms    34 ms    33 ms  den03s06-in-f18.1e100.net [74.125.225.210]

Trace complete.

stephenw10

For the record you can just use fetch instead of wget and it's already installed.

Also the pkg source location is now out of date for 2.0.1 so you have to specify the full path to the file. E.g.

pkg_add -r ftp://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.1-release/Latest/wget.tbz

Also if ftp is blocked upstream you can use http instead:

pkg_add -r http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.1-release/Latest/wget.tbz

Steve

podilarius

what johnpoz says is very true of corporate networks. I would use a DNS server other than corporate if I was going to try to get around security. Most likely they have it so that only certain IP or users can even get to the internet. Can you get to local web servers? They may even have a captive portal or something.

vdecristofaro

@johnpoz:

Ok here is the thing - you mention your in a large network right.

Large networks quite often do a few things for security reasons. 1 they normally only allow a proxy internet access, so to get to the internet you have to use that proxy. Direct internet access is blocked - only the proxy is allowed internet access.

Right!

@johnpoz:

2 they normally block dns, your local dns you point to - ie those 10.182 and 10.254 IPs more than likely don't even resolve public domains. quick enough to check, using nslookup or dig - do a query to them directly for outside something, www.google.com, ftp.freebsd.org, etc. Along with not allowing the local nameservers to resolve public domains they normally do not allow you to query outside dns, ie the those googledns you have there at 8.8.8.8

I dont really know how to read dig results but here is the result of that command when executed inside the pfSense VM

@johnpoz:

This is common practice for corp networks. Why you would need to run a router/firewall inside your corp network without support and details from corp IT is beyond me.
But unless you bounce off your corp proxy is more than likely your never getting off the corp network.

The reason is quick explained. I dont mind about the firewall. I have several VM (5/6) that resemble networks and that I use to develop software solutions. Having a software router for those "virtual labs", which is the only thing I need, has a couple of advantages for me. First of all I have a network lab that is really very similar to the production environment. In addition having 5/6 VM running inside my laptop, every one with two NIC (1 for internal network and one for the internet, will completely jeopardize my host NIC…
For those reasons I just thought that having a "software router" would be a solution. Maybe this solution does not apply on the network where I am now and will work without any further configuration if I would have DIRECT access to the Internet.

@johnpoz:

So does your workstation have internet access? If so look to see what proxy your browser is pointing to.

Yes. I can browse the internet from my host using a proxy. I have statically (in /etc/csh.cshrc) setup the same proxy inside the pfSense.
but unfortunately, even if I follow stephenw10 suggestion I cannot get to the record because of the same "No address record" error.

@johnpoz:

you state

"Now i am able to traceroute any public server"

Lets see this! And what are you using for your dns? You doing this from your PC or pfsense? Please post the details of where you doing this traceroute ip address, gateway and nameserves - along with the full trace to www.google.com – here is example

Well I have just posted the result of doing dig in the pfSense VM. So let's see how the host and the VM in the Virtual network behaves.

When I execute nslookup from the host PC I get these results

C:\>nslookup www.google.com
Server:  usdnsl201-ficus.usinet.it
Address:  10.182.209.132

Non-authoritative answer:
Name:    www.google.com
Addresses:  2a00:1450:400c:c06::6a
          74.125.132.106
          74.125.132.147
          74.125.132.99
          74.125.132.103
          74.125.132.104
          74.125.132.105

Finally this is the result when I execute nslookup from a windows PC/server inside the virtual network

C:\>nslookup www.google.com
1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa
        primary name server = 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.
0.0.0.0.0.0.0.ip6.arpa
        responsible mail addr = (root)
        serial  = 0
        refresh = 28800 (8 hours)
        retry   = 7200 (2 hours)
        expire  = 604800 (7 days)
        default TTL = 86400 (1 day)
Server:  UnKnown
Address:  ::1

Non-authoritative answer:
Name:    www.google.com
Addresses:  2a00:1450:400c:c06::63
          74.125.132.106
          74.125.132.147
          74.125.132.99
          74.125.132.103
          74.125.132.104
          74.125.132.105

johnpoz

Well your vm there it asked itself for dns on the ipv6 loopback, so is then forwarding or looking up direct from roots?

Same thing with your pfsense - it asked its dns forwarding process for that address, you don't actually know which dns server it asked for that. I believe it asks all of them and uses the one that answers first? Or does it ask in order? There was a thread back a while back that went over this - but I don't recall the details on that.

So clearly you can do outside dns queries - but if your having to use a proxy to actually get outbound access, the ability to do dns doesn't help much ;)

Now in
System: Advanced: Miscellaneous

There is a way to point to an upstream proxy to allow pfsense access - but not sure on how that actually functions, does that mean that need to be using proxy on pfsense? Or does pfsense just route all that traffic to that proxy?

vdecristofaro

@johnpoz:

Well your vm there it asked itself for dns on the ipv6 loopback, so is then forwarding or looking up direct from roots?

Same thing with your pfsense - it asked its dns forwarding process for that address, you don't actually know which dns server it asked for that. I believe it asks all of them and uses the one that answers first? Or does it ask in order? There was a thread back a while back that went over this - but I don't recall the details on that.

So clearly you can do outside dns queries - but if your having to use a proxy to actually get outbound access, the ability to do dns doesn't help much ;)

Now in
System: Advanced: Miscellaneous

There is a way to point to an upstream proxy to allow pfsense access - but not sure on how that actually functions, does that mean that need to be using proxy on pfsense? Or does pfsense just route all that traffic to that proxy?

I did already setup proxy as you're suggesting. Me too, I dont have idea on how this works. In the next days I will try this setup in a network with direct access just to verify that in effect this can be a problem caused by the presence of the proxy.

For now, I would like to thank all the people that spent some of it's time trying to help.
:)

stephenw10

There is no need to add the proxy information to pfSense, you only need to do that as a convenience for machines behind pfSense and to enable the box itself to have web access. You can just add the proxy to the clients behind pfSense as would for boxes in front it. You won't be able to update from the webgui.
It maybe that the upstream proxy has a problem with NATed clients connecting to it perhaps by design. It would seem reasonable for your network admin to not won't people running their own routers.

Steve

johnpoz

How would proxy know they are natted? Shouldn't all the traffic look like it is coming from pfsense wan interface that is on the network just like a normal client?

stephenw10

Multiple clients logging in with the same credentials? Too many simultaneous sessions?
I'm speculating. I agree it seems unlikely.

Steve

vdecristofaro

@stephenw10:

There is no need to add the proxy information to pfSense, you only need to do that as a convenience for machines behind pfSense and to enable the box itself to have web access. You can just add the proxy to the clients behind pfSense as would for boxes in front it. You won't be able to update from the webgui.
It maybe that the upstream proxy has a problem with NATed clients connecting to it perhaps by design. It would seem reasonable for your network admin to not won't people running their own routers.

Steve

Well even if I remove the proxy informations I am not able to navigate the web from the pfSense machine.
This continue to appear very strange to me and I am still convinced of a configuration problem….

Look at this screenshot (taken from the pfSense machine) to understand why I am saying this...

How it is possible that I can do nslookup without any problem and getting "No address record" when using fetch??

podilarius

If you do a fetch with the IP, does it work? (fetch http://173.194.65.105)
I think generally you want to be file specific with fetch so you don't get to much. Perhaps:
fetch http://files.chi.pfsense.org/jimp/foo/shiny/ehrmagerd/pfSense-Full-Update-2.0.2-RELEASE-i386-20121004-1028.tgz

stephenw10

If you have removed the upstream proxy settings from pfSense then this won't work. Assuming the network admin has blocked non proxied http access.

Steve

johnpoz

So do you need a proxy to get out?? This has been verified right? Have you set this for fetch to use?

I believe you can view this with the echo of the $HTTP_PROXY variable

echo $HTTP_PROXY
HTTP_PROXY: Undefined variable.

also what is in your /etc/resolv.conf file

Do you have any limitation on dns in any of your rules or the dns server your trying to use?

This proxy on your network - how is it implemented. Do you have to set it explicit, is wccp used? Is a transparent proxy? Is there any sort of captive portal setup where you have to auth or agree to something before you get access using the proxy?

vdecristofaro

@podilarius:

If you do a fetch with the IP, does it work? (fetch http://173.194.65.105)
I think generally you want to be file specific with fetch so you don't get to much. Perhaps:
fetch http://files.chi.pfsense.org/jimp/foo/shiny/ehrmagerd/pfSense-Full-Update-2.0.2-RELEASE-i386-20121004-1028.tgz

@stephenw10:

If you have removed the upstream proxy settings from pfSense then this won't work. Assuming the network admin has blocked non proxied http access.

Steve

Yes I have removed the configuration. But I am just trying to get to the internet from the pfsense shell…

@johnpoz:

So do you need a proxy to get out?? This has been verified right? Have you set this for fetch to use?

I believe you can view this with the echo of the $HTTP_PROXY variable

echo $HTTP_PROXY
HTTP_PROXY: Undefined variable.

also what is in your /etc/resolv.conf file

@johnpoz:

Do you have any limitation on dns in any of your rules or the dns server your trying to use?

This proxy on your network - how is it implemented. Do you have to set it explicit, is wccp used? Is a transparent proxy? Is there any sort of captive portal setup where you have to auth or agree to something before you get access using the proxy?

There is an internal DNS server in a Windows 2008 VM which is running at 192.168.83.11 and which serves the clients of the virtual network 192.168.83.0.
On this server I also have configured to forward queries to 192.168.83.1 (the pfSense router). As you can see from the previous screenshot, the pfSense router has DNS of my host network in resolve.conf.
I dont really know how to verify if there are any limitations.

The proxy it is a Squid 2.7v9 with Basic Authentication. On our client pc's we can configure it directly using the sintax I've used above or even with a wpad autoconfiguration script which provides load balancing. It does'nt make any difference for the clients.
It is not a transparent proxy. We dont have wccp.