Another Wan to Lan connectivity issue



  • Ok, so new to PF Sense, new to this configuration. We are replacing an ancient IPtables firewall with a configuration that has been long lost to anyone here. the firewall is translating between two networks and we are working on getting these to talk to each other. When i did a test change over last week it failed due to the LAN and WAN not speaking to each other. I could ping Wan side and Lan side, but not across the two. Configuration is setup on a Xen Server. Ip's are 192.168.0.X on the WAN side and 192.168.1.X on the LAN side. I am unsure how to pull the configuration from the command line as I don't have it connected into the network right now to use the GUI. I will check the information on how to do this today and post the information straight from the box as to current config. I did try many options, with and without gateway, with upstream gateways etc, all failed to get the two to talk to each other. I also tried a few fixes from here as well such as removing gateways and such and all failed. Any ideas, I know, not to much to go one until I post configs I suppose, but a direction to go int would help.



  • I also tried a few fixes from here as well such as removing gateways and such and all failed.

    Were you trying your changes from the command line?  Mucking about at that level is liable to break things or at least make it difficult to diagnose what's going on.

    From my POV step 1 is to get the pfsense box to the point you can reliably access the WAN and LAN sides.  Usually a fresh install is easiest tested with a PC/Laptop/whatever attached to a small switch on the LAN port.  Provided you enabled DHCP in the setup (a good idea to start) you should be able to get an IP address and login to the box. From there setup a basic "enable all" rule on the LAN side and allow WAN side access to the external port (80 or whatever you changed it to) and you should at least be able to talk to the box.

    Can you describe your environment in a little more detail?  Does the WAN side point to the internet or just another part of your internal network? Is DHCP already running on the WAN side and/or the LAN side?



  • I ran the configs at the GUI level, it's just atm I only have access to the command line due to the environment. This is a fresh install running on XEN server 6.2. dual LAN ports (installed on a Dell R710 server). These ports are only accessed and used by PF sense. I have 4 other VM's running on this server using the other 2 ports and they work flawlessly. We are currently running an IP tables fw on a box that is about 16 years old, this is strictly firewall, My other server (we are working on retiring) is a 12 year old system that controls DNS/DHCP. When I enabled everything last week and attempted the change over I disabled DNS/DHCP on that system (Cent OS 5.2). I then placed this in network and configured it with the IP's required. As far as what points where.

    Internet (2 companies/failover) - Router (192.168.0.1) - (192.168.0.2 WAN) Firewall (192.168.1.1 LAN) - Internal network devices (192.168.1.X).

    The system successfully handed out IP's on the Lan side, I could access devices/servers within the LAN no issues. I could see the Firewall MAC from the Router and vice versa, I could ping (in GUI PF sense) from WAN port to router and I could ping LAN side, but ping from WAN (0.0) to LAN (1.1) failed each time.

    It's possible I did not have the enable all rule set on the LAN side. I will have to check that.



  • From what your describing, the pfSense box was working exactly as it should.  Normally you wouldn't allow a ping fro the WAN to the LAN side (unless you put a rule in place to allow that).  A more appropriate test would be to ping the Router from the LAN side, assuming the Router will respond to pings.  The real acid test is to see if the LAN can reach the internet, 8.8.8.8 is always a good simple ping test.



  • Sorry I didn't make it clear, No traffic was flowing between WAN to LAN. Referencing ping is of course only used as a basic method of confirmation of connectivity. Is there a way I can pull the network config at the command line level so i can post it here?


  • Netgate Administrator

    All of pfSenses' config is stored in the /conf/config.xml file which you should be able to access from the command line. Be aware that it contains potentially sensitive info though so don't just post it complete.

    You should be able to ping from LAN clients to WAN side devices even with a default config. You won't be able to ping from WAN to LAN even if you've put in firewall rules to allow that. pfSense NATs traffic between WAN and LAN by default so there is no route from WAN to LAN directly, WAN side clients do not know about the LAN side subnet. Do you want NAT?

    Have you removed the 'block private networks' rule from the WAN interface setup? Since your WAN is in a private subnet you should.

    Steve



  • Thanks for the reply, wish i had seen it last night when I was attempting to swap this yet again. I did not remove the block private networks rule, this may be it. I went through and tied several configs and all failed I could ping the Wan interface from the lan side, so there was a little more movement. I could not however ping the router or ping from the router to the wan interface. I did take some pics from the interface before I swapped back over to my current setup.

    ![fw rules.png](/public/imported_attachments/1/fw rules.png)
    ![fw rules.png_thumb](/public/imported_attachments/1/fw rules.png_thumb)







  • The Block Private Networks rule is definitely it.  You're telling it to block private IP space from WAN, and your modem is using that exact IP space.


  • Netgate Administrator

    You should definitely not have 'block private networks' enabled on WAN since your WAN is in a private network. However the 'block private networks' rule is just a firewall rule on WAN like any other. It will block incoming traffic on the WAN interface coming from an RFC1918 network. It will not block outgoing traffic. It will not block return traffic from an existing state. It should not stop you pinging from a LAN side device to your WAN side router.

    Much more likely culprit for me is that you have two gateways listed and one appears to be on the LAN. You may have removed it from the LAN interface config but it's still there. Go to System: Routing: Gateways: and remove all but the correct WAN gateway. Make sure that is the default gateway.

    Steve



  • Ok, so I tried the settings that both you guys mentioned, and they failed to resolve the issue, so I said fine, and factory defaulted the system. Lo and behold, bam. I have internet access. But the issue then became that it was so slow that I could have connected to my 3G phone and been faster then my fiber backed office connection. Ironically facebook had 0 issues loading, however certain content did not load properly. Pages Like reddit, amazon, imgur, etc, slow, up to 2-3 minutes per page to load, and some had to be refreshed multiple times to get them to load. Other pages, notably, pfsense.org, would not load at all. I tried changing DNS settings as such:

    192.168.1.1 (internal LAN IP)
    8.8.8.8
    8.8.4.4

    I tried swapping to:

    8.8.8.8
    8.8.4.4
    192.168.1.1

    by default it of course appends 127.0.0.1 at the start of the dns settings.

    I also verified the router settings and made some tweaks. No change, or such a small change that it was not noticeable.

    Any ideas? any options I could tweak? Rules i could add to test?



  • Speed & duplex on the WAN link?  Perhaps change it from auto-select to what it actually is?


  • Netgate Administrator

    Some websites or parts of  websites not loading could be MTU issues.

    Are you still using the same subnets for your WAN and LAN? Where are you entering those DNS settings? If that's in the general setup you shouldn't have your LAN address there.
    Are you using DHCP for WAN? Do you have 'Allow DNS server list to be overridden by DHCP/PPP on WAN' enabled?

    If it is a dns issue you should still be able ping by ip without loss.

    Steve



  • Are you still using the same subnets for your WAN and LAN? Exact same as prior setup. no changes

    Where are you entering those DNS settings? DNS settings were entered into the page under where ever the DNS servers are located at.

    If that's in the general setup you shouldn't have your LAN address there. Lan is there since we have certain servers that are only used internally and wanted to make sure they are accessible. so wanted to make sure that any computers that connect up look at the fw as the first stop for dns before looking externally.

    Are you using DHCP for WAN? No. WAn side is only 2 devices, router and fw. Both set static.

    Do you have 'Allow DNS server list to be overridden by DHCP/PPP on WAN' enabled? Unsure, will have to look at that page to verify this.

    We are trying one thing tonight, which is to take and put the fw on a physical machine as it is running on a Xen Server right now.


  • Netgate Administrator

    There are two places where you might enter DNS servers. First in System: General Setup: this is where you enter the DNS servers that the pfSense box itself should use. These would typically be your ISPs DNS severs or some publically available servers like 8.8.8.8. In your case you might have your upstream router address here which would then forward the requests to whatever its using. You should not have the LAN interface address here.
    The other place is in the DHCP server setup in Services: DHCP server: LAN (tab): This is the list of servers that are sent to clients on the LAN via DHCP. Typically they are left empty because as it says there:

    leave blank to use the system default DNS servers - this interface's IP if DNS forwarder is enabled

    So clients will automatically use the LAN interface address and hence the pfSense DNS forwarder. If you have entered DNS overrides for your internal servers they will be handed to clients who will then be able to connect directly. You could have the LAN interface address here but your clients are probably using it anyway.

    Even if you do have wrong DNS entries that shouldn't be causing all the problems you're seeing though. I await the results of a ping test.  :)

    Steve



  • OK, so built a physical system last night, installed and tested. Exact same results. Slow browsing, except facebook, I suggested to my managers to move all our business to facebook, they were less then thrilled…

    Do you have 'Allow DNS server list to be overridden by DHCP/PPP on WAN' enabled? Not this time, previous setup does have it enabled as i noticed that during setup this time.

    DNS config was done at boot/setup, so no additional entries were chosen at this time. Used Google's servers for the DNS. (8.8.8.8 ; 8.8.4.4) Did not use the LAN Ip for DNS this time to test that.

    I did forget in my work to run ping tests….shoot. I will have to test that, but I can do that fairly quickly as opposed to the extra time I had last night to retest and build an entire new machine.

    In the end, I had a factory defaulted machine, running a core2duo with 3Gb of ram and no additional rules or settings in it and still could not get to pfsense.org and many many other sites. I may need to resort to purchasing support at this time if I can't resolve this very very soon.


  • Netgate Administrator

    Right time to get basic!
    The connectivity to pfsense.org is not intermittent but in fact there is no connectivity at all, yes?
    Try pinging pfsense.org from a client machine on the LAN. Try it from the pfSense console. Try running a traceroute to find out where it's not connecting.

    This could still be an MTU issue. What type of connection is your WAN?

    Steve



  • OK, ping results are in! News is not good :( I was able to run the tests, and again system had issues getting to sites. For example, I could not get to, nor ping google.com, but I could access it by IP. I also included the PIng test from my current setup so we could see a comparison.







  • Netgate Administrator

    I've been away just trying to catch up.
    What exactly is the 'oldpingtest' screen shot from?

    It looks like DNS is working fine and that you can access some IPs fine but not others. This could be an MTU issue but could also be a subnet mask problem. Reading back through the thread in your screen shot from the WAN setup page you have the WAN setup as a /32 which can't work for a static IP. At the very least the gateway IP must be in the same subnet.

    Steve



  • The old ping is from my current setup, sorry thought I made it clear. DNS is not working fine, as you can see in the pings even things like google.com are not accessible unless I use the IP. By IP they are ok, but even then the pings are coming back very, very poorly compared to current configuration. Right now to google I have a 20ms ping and with pfsense in place I have a 77ms ping time. There was 0 conectivity by name or IP to pfsense.org and other sites were similar to a dial speed access such as Amazon.com.

    As far as what type of connection it it, it's a fiber connection coming off a router to the pfsense box. and the MTU would be set to whatever defaults it has on a new config.

    if the system set it up as a /32, I was unaware. and the gateways listed are from wan to lan configuration. They do not fall under the same subnet, and we don;t want them to fall under the same subnet from WAN to LAN. The configuration is simply trying to mimic what we have here. On the clients the gateway is listed as the LAN side so falls into the same subnet. Example:

    My laptop:
    IP: 192.168.1.211
    Subnet: 255.255.255.0
    Gateway: 192.168.1.1

    On the pfsense box:
    Lan side port 192.168.1.1
    Wan side port 192.168.0.2

    Gateway was set to only use Wan side gateway on the PFsense box as per advice earlier in this thread

    Wan gateway: 192.168.0.1

    Which goes directly to a Router
    Router Ip LAN: 192.168.0.1
    Router IP Wan: public IP

    What other information do you need? there is no additional configuration on this box beyond the basic out of the box configuration. This was tested on two separate systems, one physical, one virtual. Both had the exact same results and exact same pings. Just getting frustrated here as I have never seen a firewall act like this one does as far as basic internet traffic goes.


  • Netgate Administrator

    I understand your frustration, it's an odd fault.

    The DNS resolution is in fact working in every ping example shown. You are not seeing 'unable to resolve host'. However the IP address returned does not appear to be google.  ??? What DNS servers is the pfSense box using?
    You could try entering some external dns servers, like 8.8.8.8 and 8.8.4.4, at the clients directly. They then won't be using the pfSense DNS forwarder with whatever glitches it seems to be introducing.

    The static WAN setup shown in your earlier screenshot it shows a /32 subnet which would mean it has no route to it's own gateway IP. Definitely check that has been changed since.  ;)

    Steve



  • Ok, so. I isolated the network. Basically ISP>Router>PFsense> separate switch>1port>laptop. It ran beautifully. I then had the theory that perhaps my sorta smart switches, had poor routing capabilities and had bad MAC routing info. So I rebooted my switch, reset all things to my network, and bam. went from a 77ms ping time to google and a network connection that made people with Modems laugh at to a 2ms ping to google. So in the end the issue lay outside the scope of troubleshooting we did. Which we should have thought about this after we received the same results with a separate system. All good though.

    Thanks for the help! I'm sure I'll be back for more.


  • Netgate Administrator

    Hmm, ok. Weird. I guess I'll chalk that up to experience.  ;)

    Steve



  • Just caught this one, glad to hear it's all good now. One point that I would have added is did you want Pfsense to NAT or route? Seems to me your first router connected to the ISP should be doing the NATing and PfSense should route. You will need to run a routing protocol between the two or setup static routes. You might see a speed increase as NATing can be expensive. Hell you might want to put PfSense first at the perimeter of your network.



  • Unfortunately with a dual network failover configuration my current setup won't support PFsense at the edge of the network (only 2 ports available) I have considered setting up a PFsense router/firewall device, but with the cost of ours, and the fact the the owner just put more money out on it for support/updates he would be unwilling to pay out for another device. So am stuck where I am.