PfSense newbie configuration problem

vdecristofaro · Sep 28, 2012, 12:56 PM

First of all, thanks to anybody that would help on this.
I am very poor in Linux knowledge and networking, I am sorry if my question would be considered dumb for most of you.

Scenario:
I have 4 virtualbox VM's each with a single NIC attached to the Internal network.
Every server is configured to have a static IP Address (192.168.83.11 to 14) with subnet mask 255.255.255.0

Those servers reproduce a simple Windows 2008 Network with Active Directory and DNS installed to server 192.168.83.11.

I have tried to setup a new VM to create an WAN router (as described in the "Common Deployments" section of pfSense web site).
For this reason, using the resources found around, I have been able to create a new VM with the following settings:

BSD/FreeBSD OS, 256Mb Memory, 4 Gb HD
NIC 1: Bridged Networking. To be used as WAN
NIC 2: Internal Network. Same network name as other vm's

I have then installed pfSense using the pfSense-2.0.1-RELEASE-amd64.iso.gz and following the official instructions at http://doc.pfsense.org/index.php/InstallationGuide

Checks that I have done:

Every server can ping the WAN Router and vice versa
I can see that every interface is up and has an IP address assigned
I have configured the pfSense LAN IP (192.168.83.1) as Gateway for other VM
I have configured the pfSense LAN IP (192.168.83.1) as DNS Forwarder in the Internal network DNS

However I am not able to connect to the internet from inside the Internal network. I am looking for help on troubleshooting this setup.

podilarius · Sep 28, 2012, 3:07 PM

I had the same problem (and I just tested again), but on the WAN interface, you need to disable the block on private networks.

vdecristofaro · Sep 28, 2012, 3:33 PM

Hello podilarius,
thanks for your answer. I have already removed the check on "Block private networks".
I suspect that somewhere the firewall is blocking everything. Also I do not need the firewall, I would like just to setup a WAN router.

From the pfSense console I choosed option 10 - Filter Logs.
It continuously write text like:

rule 1/0(match): block in on em0: 10.169.121.X.137 > 10.169.121.255.137

where X is in turn a different number. What this means?

johnpoz · Sep 28, 2012, 3:52 PM

That looks like a directed broadcast to me – you would normally want those blocked.. I assume em0 is your wan interface.

vdecristofaro · Sep 28, 2012, 3:55 PM

Your assumption is correct: em0 is my WAN interface.

How can I block those directed broadcast? Also, do I need to setup other rules?

stephenw10 · Sep 28, 2012, 6:03 PM

Port 137 is NetBIOS traffic. It will be coming from windows machines on the WAN side of your pfSense VM. It's nothing to worry about.

Steve

vdecristofaro · Sep 28, 2012, 6:36 PM

Good to know :)
In your opinion, is there any method I should follow to troubleshoot My issue?
Could you please drive me on what I should do to make it working?

Thanks!

wallabybob · Sep 28, 2012, 8:11 PM

One check you haven't mentioned is ping to a public IP address from the pFsense console. What response do you get? (Posting the actual response will probably be more informative than posting something like "it doesn't work".)

podilarius · Sep 29, 2012, 1:02 AM

If you don't need firewall in and you have your routing setup correctly, you can go to setup -> advanced -> firewall and disable the firewall.

vdecristofaro · Oct 1, 2012, 8:11 AM

@wallabybob:

One check you haven't mentioned is ping to a public IP address from the pFsense console. What response do you get? (Posting the actual response will probably be more informative than posting something like "it doesn't work".)

Ok. Let me say that I am trying this setup in my office where I am in a very complex network environment that spreads around different countries.
Anyway, if I try to ping a public server (like google.com) I get the same behaviour that I get if I do the same from my host. That's it, ping does not work.
However, if I try to traceroute a public server I can see that somewhere it stops working, and the result is the same from the pfSense console or from my host console. Something like the following result:

C:\>ping www.google.com

Pinging www.google.com [173.194.35.146] with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 173.194.35.146:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

This is the traceroute result

C:\>tracert www.google.com

Tracing route to www.google.com [173.194.35.146]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  10.69.121.2
  2     1 ms     1 ms     1 ms  10.72.33.57
  3     2 ms     3 ms     2 ms  172.31.190.122
  4    12 ms    12 ms    12 ms  172.31.1.250
  5    12 ms    11 ms    13 ms  172.31.1.249
  6    12 ms    12 ms    12 ms  10.254.141.244
  7    11 ms    12 ms    12 ms  10.254.130.114
  8    13 ms    13 ms    12 ms  10.254.36.62
  9     *        *        *     Request timed out.
 10     *        *        *     Request timed out.
 11  ^C
C:\>

I have executed those commands from my host machine but, believe me, the results are the same if I do it from the pfSense console.

@podilarius:

If you don't need firewall in and you have your routing setup correctly, you can go to setup -> advanced -> firewall and disable the firewall.

I have tried to disable the firewall going into the webConfigurator, System, Advanced, Firewall/NAT and then I have selected the checbox that says "Disable all packet filtering". Is that correct? In any case it does not work either. Please let me know if you want to know any further detail. Thanks.

wallabybob · Oct 1, 2012, 11:05 AM

@vdecristofaro:

This is the traceroute result

I take it from the preceding text in your reply that the tracert output is taken from one of the VMs that can't reach the internet.

@vdecristofaro:

C:\>tracert www.google.com

Tracing route to www.google.com [173.194.35.146]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  10.69.121.2

This is allegedly on a machine that is using pfSense as its default gateway and gets it IP address from DHCP server running on pfSense LAN interface. Therefore why is the nexthop address on a completely different subnet from the pfSense LAN interface (192.168.83.1/24)?

In short, the information you have provided is horribly contradictory. Until you correct that I doubt I can help you.

stephenw10 · Oct 1, 2012, 11:21 AM

Traceroute in Windows (XP sp3 at least) gives the WAN gateway as the first hop:

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\Steve>tracert google.com

Tracing route to google.com [74.125.230.97]
over a maximum of 30 hops:

  1     5 ms     6 ms     7 ms  217.32.145.233
  2     6 ms     5 ms     6 ms  217.32.146.30
  3    10 ms    10 ms    10 ms  213.120.181.118
  4    10 ms    10 ms    10 ms  217.41.169.203
  5    10 ms    10 ms    10 ms  217.41.169.109
  6    10 ms    10 ms    10 ms  acc2-10GigE-9-2-0.sf.21cn-ipp.bt.net [109.159.251.221]
  7    19 ms    18 ms    19 ms  core1-te0-2-2-0.ilford.ukcore.bt.net [109.159.251.145]
  8    18 ms    18 ms    18 ms  peer1-xe3-1-0.telehouse.ukcore.bt.net [109.159.254.213]
  9    19 ms    19 ms    19 ms  195.99.125.21
 10    15 ms    16 ms    15 ms  209.85.252.188
 11    17 ms    17 ms    17 ms  209.85.251.62
 12    16 ms    16 ms    16 ms  lhr14s01-in-f1.1e100.net [74.125.230.97]

Trace complete.

C:\Documents and Settings\Steve>ipconfig

Windows IP Configuration

Ethernet adapter Local Area Connection:

        Connection-specific DNS Suffix  . : fire.box
        IP Address. . . . . . . . . . . . : 192.168.2.10
        Subnet Mask . . . . . . . . . . . : 255.255.255.0
        Default Gateway . . . . . . . . . : 192.168.2.1

@vdecristofaro:

rule 1/0(match): block in on em0: 10.169.121.X.137 > 10.169.121.255.137

This implies your WAN is in 10.169.121.* but that doesn't appear in the traceroute. However 10.69.121.* does, typo?

Steve

vdecristofaro · Oct 1, 2012, 12:44 PM

@wallabybob:

This is allegedly on a machine that is using pfSense as its default gateway and gets it IP address from DHCP server running on pfSense LAN interface. Therefore why is the nexthop address on a completely different subnet from the pfSense LAN interface (192.168.83.1/24)?
In short, the information you have provided is horribly contradictory. Until you correct that I doubt I can help you.

I did a mistake I am sorry.
In effect when doing traceroute form my HOST or from the pfSense VM the result is the one I have posted.
When doing traceroute from a VM of the virtual network I just see this

C:\>tracert google.com

Tracing route to www.google.com [173.194.35.146]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  harper.localdomain [192.168.83.1]
  2     *        *        *     Request timed out.
  3     *        *        *     Request timed out.

I am sorry for the mistake…

vdecristofaro · Oct 1, 2012, 12:53 PM

@stephenw10:

This implies your WAN is in 10.169.121.* but that doesn't appear in the traceroute. However 10.69.121.* does, typo?

Steve

I do not know how the network is made because it is very complex and spreads between multiple countries.
What I know for sure is that my IP address (the Host as well as the WAN in the pfSense) is in the family 10.69.121.* and if I traceroute to google I can see that the first hop is the gateway defined statically in the NIC configuration.
Why are you saying that it does'nt appear in traceroute?

podilarius · Oct 1, 2012, 2:02 PM

If you are not NATing, then you need to make sure that the upstream routers knows how to route the traffic back to the LAN side of your pfSense machine. If you don't have control of that, then you need to stick with NATing.

In all my traceroutes under 2.1 the LAN of my firewall is the first hop. In your case that should 192.168.83.1. In my traceroutes under 2.0.1, the WAN IP if the pfSense FW is the first. Strange!?.

If you have control over the downstream routers, I would check them to make sure the routing is correct and then test by pinging them. With firewall turned off, there is no rule or NAT problem that will affect packets getting to the destination, only routing issues.

vdecristofaro · Oct 1, 2012, 2:23 PM

@podilarius:

If you are not NATing, then you need to make sure that the upstream routers knows how to route the traffic back to the LAN side of your pfSense machine. If you don't have control of that, then you need to stick with NATing.

Ok. I am almost sure that I am not NATing.
I went to the webconfigurator, setup -> advanced -> Firewall/NAT
and it is so configured:

Disable NAT reflection for port forward : Checked
Reflection timeout: empty
Disable NAT Reflection for 1:1 NAT: Checked
Automatically create outbound NAT rules […] : Not Checked
TFTP Proxy: I have selected the WAN interface and specified proxy params in the Miscellaneous TAB

@podilarius:

If you have control over the downstream routers, I would check them to make sure the routing is correct and then test by pinging them. With firewall turned off, there is no rule or NAT problem that will affect packets getting to the destination, only routing issues.

It seems so easy to me to logically understand things that you are explaining :)
But unfortunately I am not able to troubleshoot routing issues :-[
Could you please drive me in applying your suggestion? Thanks

stephenw10 · Oct 1, 2012, 3:10 PM

@podilarius:

In all my traceroutes under 2.1 the LAN of my firewall is the first hop. In your case that should 192.168.83.1. In my traceroutes under 2.0.1, the WAN IP if the pfSense FW is the first. Strange!?

Indeed I thought it should show the pfSense machine as the first hop but it doesn't. :-\

@vdecristofaro:

Ok. I am almost sure that I am not NATing.
I went to the webconfigurator, setup -> advanced -> Firewall/NAT
and it is so configured:

Automatically create outbound NAT rules […] : Not Checked

If you have turned off outbound NAT, and it looks like you have, then you will need to have all your routing tables correct or nothing knows where to go. Ping replies from your second hop do not have a route back your internal machines.

I suggest your turn Auto Outbound NAT back on unless you really need to have it disabled.

Steve

podilarius · Oct 1, 2012, 3:18 PM

If you are going to get NAT going, you need to uncheck the option to disable all firewall filtering. This will turn NAT back on so that you can use Automatic NAT.

The thing is that every router behind your public IP (which is doing the main NAT), is going to have to know how to route 192.168.83 to your pfSense machine. Without that, you are not going to get this working. (IF you are not NATing)

okay, so the main confusion is if you are going to NAT or not, firewall or not. Once you let us know, then we can help further. Otherwise, we are going to talk in generalities to help you make up your mind on NATing or not. It can be done either way, its just that the config is very different.

vdecristofaro · Oct 1, 2012, 3:50 PM

@podilarius:

…
The thing is that every router behind your public IP (which is doing the main NAT), is going to have to know how to route 192.168.83 to your pfSense machine. Without that, you are not going to get this working. (IF you are not NATing)
...

Well… I thought to this very very long time, and at the end, came to the decision to use NAT (mainly because I cannot ask nobody to configure routers behind my WAN...)
Which parameters should I setup?

:)

podilarius · Oct 1, 2012, 4:14 PM

Okay … in advanced setup -> uncheck the option to disable firewalling. Save and apply.
Then head to firewall -> advanced outbound NAT and select auto. Save and apply.
After that, head to firewall -> rules -> LAN. Setup a rule to allow any protocol with source LAN subnet to Any/Any. Save and apply.
Then go to Services -> DHCP and enable that on LAN. give is a range like 192.168.83.50-250. save and apply.
reboot the FW.

Get on a machine behind the FW and trace route to www.google.com and see how far your get.