(SOLVED) Problem with client connect through static IP cable internet



  • Thanks again for your answers!

    @JeGr said in Problem with client connect through static IP cable internet:

    @paprikawuerzung said in Problem with client connect through static IP cable internet:

    Is this something specific to my static IP setup on WAN? Because when I use a PPPoE WAN configuration I do not need to open the ports manually - there, the configurations are done automatically and without problems at all!

    pfSense has a default rule on LAN, allowing ANY traffic outbound to the internet. If you have that one still in place your problem is not layer 4/rules related. Your NAT is configured on automatic?

    Yes, I have all defaults configurations in the Firewall/(outbound) NAT sections.

    How about showing the rules and outbound NAT settings as well as the WAN interface settings?

    I am currently not on-premise. I will provide all screenshots later on. But as I said before: they are pretty standard!


  • LAYER 8 Moderator

    No problem. If there're indeed the defaults at work we'll have to check from the client up to the WAN to find the culprit here. That's why screens of WAN IF config, NAT outbound, firewall rules WAN/LAN and DNS resolver (unbound) config may be necessary. :)


  • LAYER 8

    but when he wrote that "nslookup google.de 1.1.1.1" is not working that can only be because port 53 is closed somehow/somewhere,that's why i suggested to check firewall logs
    anyway
    we will wait for the screenshots



  • What kind of configuration are you using on DNS? Is it Resolver or Forward?
    If on a local machine you dig google.de without any other options, which server do you query?

    Viewing on your local machine, what DHCP DNS config is being passed to clients?


  • LAYER 8 Moderator

    @kiokoman said in Problem with client connect through static IP cable internet:

    but when he wrote that "nslookup google.de 1.1.1.1" is not working that can only be because port 53 is closed somehow/somewhere,that's why i suggested to check firewall logs

    Nope, could also be a problem with unbound/dnsmasq (depending on wheter he uses forwarder or resolver) and how it is set up and configured. That's why I was asking about what DHCP settings are configured in another thread. If he pushes an external non-working DNS to the client, that would explain the broken setup. @maverickws question points into the same area :)

    Greets



  • Wow, a lot of answers!

    @JeGr said in Problem with client connect through static IP cable internet:

    @kiokoman said in Problem with client connect through static IP cable internet:

    but when he wrote that "nslookup google.de 1.1.1.1" is not working that can only be because port 53 is closed somehow/somewhere,that's why i suggested to check firewall logs

    Nope, could also be a problem with unbound/dnsmasq (depending on wheter he uses forwarder or resolver) and how it is set up and configured. That's why I was asking about what DHCP settings are configured in another thread. If he pushes an external non-working DNS to the client, that would explain the broken setup. @maverickws question points into the same area :)

    Greets

    The DNS provided by DHCP to the client does not seem to be the problem! The client has 192.168.178.1 (pfSense) as DNS and pfSense in turn uses the DNS Forwarder (unbound) to resolve the domains (which works just fine).

    The problem is: the clients are not able to make DNS queries themselves (= directly to 8.8.8.8 for example)! The gateway (192.168.178.1 - pfSense) is set correctly on the clients. But all connections with ports (layer 4) time out...

    I strongly believe that this is a gateway/NAT/firewall problem - not a DHCP config problem at all. pfSense gets all the traffic from the clients - but somehow does not handle it correctly.

    I will create a complete fresh install of pfSense, do minimal configuration (so that pfSense hat an internet connection through the cable) and will send screenshots in 3-4 hours!



  • @paprikawuerzung
    Hi there,

    I don't think you have the correct configuration, and may I add I did this mistake recently of having forwarder mode enabled thus bypassing unbound.
    btw, is 192.168.178.1 the pfSense?

    If you go to Diagnostics > DNS Lookup you can resolve properly from the pfSense?

    Basically if you are using DNS Resolver (unbound) you don't need to set any DNS servers (much less Google's, phew, try CloudFlare or OpenDNS), but anyway with this configuration you may set NONE.
    The purpose of DNS Resolver is to query the ROOT DNS servers on the top of hierarchy.
    For this you must disable the "enable forwarding mode" on DNS Resolver.

    The DHCP will then pass the pfSense itself as DNS for your clients.

    You can make queries via IP, but you cannot resolve IP's from names. That's DNS.



  • @maverickws said in Problem with client connect through static IP cable internet:

    @paprikawuerzung
    Hi there,

    I don't think you have the correct configuration, and may I add I did this mistake recently of having forwarder mode enabled thus bypassing unbound.
    btw, is 192.168.178.1 the pfSense?

    If you go to Diagnostics > DNS Lookup you can resolve properly from the pfSense?

    Yes, I can query everything without any problem. I am really, really, really sure that DNS is not the problem since HTTP and SSH (with some IP address) do not work (which does not have anything to do with name translation at all) on client side - but work without problems on pfSense (see diagram)...

    Basically if you are using DNS Resolver (unbound) you don't need to set any DNS servers (much less Google's, phew, try CloudFlare or OpenDNS), but anyway with this configuration you may set NONE.

    I have 1.1.1.1 and 8.8.8.8 set in the general setup - they are used by unbound as far as I know. But as I said before: I do not think DNS is the problem - HTTP, SSH, ... all do not work from the client - but using "Diagnostics -> Command Prompt" on pfSense they work without problems.

    The purpose of DNS Resolver is to query the ROOT DNS servers on the top of hierarchy.
    For this you must disable the "enable forwarding mode" on DNS Resolver.

    The DHCP will then pass the pfSense itself as DNS for your clients.

    You can make queries via IP, but you cannot resolve IP's from names. That's DNS.

    I know what DNS is for and I also understand that there might be a problem with it - but also IP-based traffic (without domain names at all) do not work also - they all time out.



  • When you say "HTTP, SSH all do not work from the client" in the example (diagram) you provided you have:

    curl google.de not working - still requires DNS to resolve google.de - sorry I didn't get from your previous posts that you were lacking connectivity when using IP addresses.

    I'm not about nslookup so I'll skip that one in particular.

    Anyway cause I gtg I'll leave my DNS Resolver config so you can compare to yours:

    DNS Resolver - General Settings

    Also, if you do a curl 216.58.211.35 what is the output?

    Do you have the default allow LAN to any rule on your firewall?

    Firewall > Rules > LAN

    • Action: Pass
    • Interface: LAN
    • Address Family: IPv4
    • Protocol: Any
    • Source: LAN Net
    • Destination: any


  • Thanks for your answers again!

    @maverickws said in Problem with client connect through static IP cable internet:

    When you say "HTTP, SSH all do not work from the client" in the example (diagram) you provided you have:

    curl google.de not working - still requires DNS to resolve google.de - sorry I didn't get from your previous posts that you were lacking connectivity when using IP addresses.

    Yes, nslookup google.de is working (eg. returns 172.217.21.227) on the client - so doing curl google.de or doing curl 172.217.21.227 results in the same behavior.
    Doing nslookup google.de 1.1.1.1 (ie. using Cloudflare 1.1.1.1 as the DNS server) on the client does not work. Same behaviour as with HTTP, SSH, ...

    I'm not about nslookup so I'll skip that one in particular.

    Anyway cause I gtg I'll leave my DNS Resolver config so you can compare to yours:

    Also, if you do a curl 216.58.211.35 what is the output?

    Time-out on the client. Same with SSH. Works fine on pfSense (Diagnostics -> Command Prompt).

    Do you have the default allow LAN to any rule on your firewall?

    Firewall > Rules > LAN

    • Action: Pass
    • Interface: LAN
    • Address Family: IPv4
    • Protocol: Any
    • Source: LAN Net
    • Destination: any

    I will have to look later - but the config is the default and I think it is there.

    I am really unsure what pfSense is doing with client traffic! Normally, I would assume it does outbound NAT (= gives traffic outside some temporary ports and writes its own (external) IP address in the SRC field) and does the translation backwards when the requests come back. But somehow in this chain, the packets get dropped... Is there a possibility to track all traffic (= testing NAT)?



  • @maverickws @JeGr @kiokoman

    I found the solution: I had to add a firewall rule to allow OUT traffic from LAN -> WAN. Floating Rule...
    I am not entirely sure why this was necessary but I wanted to rule out any firewall involvement. All the traffic from the LAN net to the WAN net was silently blocked by pfSense.

    As I am not that familiar with firewall rules: is this a potential security risk?



  • Post a screencap of your LAN rules so we're not guessing what you've got going on. You should NOT need floating rules unless you're doing traffic shaping or a complex config. By default, LAN allows all traffic to anywhere.



  • I currently do not have access to pfSense - I will post the screenshots tomorrow morning!

    Apart from that: thousand thanks for your help and time :)



  • This post is deleted!


  • This is my working config: config-pfSense.SCRUBBED.xml



  • Nobody wants to parse through an XML file. Screencaps are much better, a picture worth a thousand words etc.



  • @KOM said in Problem with client connect through static IP cable internet:

    Nobody wants to parse through an XML file. Screencaps are much better, a picture worth a thousand words etc.

    I am sorry but I am not on-premise. I only have the config here. But I'll try to create a VM, import the config and make screencaptures.





  • Get rid of that floating rule. You don't need it and it may be part of the problem. If you lose connectivity after deleting that rule then there is a deeper problem to be discovered and fixed because, like I said earlier, floating rules are used for QoS and special custom configs. You don't need any floating rules to get basic connectivity working.

    Your LAN rules look good but there is no traffic logged (0/0 B for each rule) which means pfSense isn't seeing anything from LAN that is hitting those rules. The floating rule may be taking precedence.

    What specific errors are the clients getting when trying to connect using various things like ssh or http/s? The exact error text can be very helpful.



  • This post is deleted!


  • @KOM said in Problem with client connect through static IP cable internet:

    Get rid of that floating rule. You don't need it and it may be part of the problem. If you lose connectivity after deleting that rule then there is a deeper problem to be discovered and fixed because, like I said earlier, floating rules are used for QoS and special custom configs. You don't need any floating rules to get basic connectivity working.

    Your LAN rules look good but there is no traffic logged (0/0 B for each rule) which means pfSense isn't seeing anything from LAN that is hitting those rules. The floating rule may be taking precedence.

    What specific errors are the clients getting when trying to connect using various things like ssh or http/s? The exact error text can be very helpful.

    Just to be 100% clear on that: without the floating rule the clients are not able to access the internet. Even IP-based traffic (like ssh to an IP) is not possible without the floating rule. Pinging specific IPs works, everything else does not work.
    With the floating rule, the internet and everything else works just perfectly.

    It is 99% a firewall issue.

    My setup is a fresh install of pfSense with only the static IP for the WAN interface and the floating rule added.

    When the floating rule is NOT enabled: curl/ssh and everything like that simply times out. There are no specific error messages (only "Connecting to ....... Connection timed-out"). The error text is very non-descriptive.

    I will provide the exact error messages after I have pfSense installed freshly (without the floating rule).



  • The default LAN IPv4 allow to any rule does not belong on the floating, so something is wrong there.
    Do you have other Floating rules?



  • @maverickws said in Problem with client connect through static IP cable internet:

    The default LAN IPv4 allow to any rule does not belong on the floating, so something is wrong there.
    Do you have other Floating rules?

    The default "LAN IPv4 allow" rule is unchanged from a fresh install (and also works when using my DSL internet through PPPoE on the WAN interface). But when using the static IP configuration for WAN this rule (for some reason) does not work (0/0B)...

    This here is a screenshot of my DSL internet firewall rule (NOT the cable static-IP internet connection where the problem persists):
    0b84dfc5-1aa9-4301-b231-eed7646e23af.png
    In this configuration for DSL with PPPoE, I do NOT have a floating rule - everything works fine.


  • LAYER 8 Moderator

    @paprikawuerzung said in Problem with client connect through static IP cable internet:

    Just to be 100% clear on that: without the floating rule the clients are not able to access the internet. Even IP-based traffic (like ssh to an IP) is not possible without the floating rule. Pinging specific IPs works, everything else does not work.

    And to be clear on our side: a floating rule is NOT needed for proper functionality. IF things work only then, you have - as already said - a deeper problem or serious configuration flaw that you didn't see/show. As outbound rules are created automatically in the background, something definetly doesn't add up. Period.

    As debugging is simply voodoo and guessing at the moment, you should gather information when you are back on site and post them. Even screens in the VM are void, as it's not the same constellation as on site with the WAN and a LAN client behind it to test.

    I'd also double check DHCP settings and the client on premise: does it have the right IP address and does it match the LAN configuration? Not the first time there was a simple small problem in numbers that got switched around and as the default LAN rules are matched against "LAN net" instead of "any" a DHCP problem can go unnoticed as "firewall access works".
    Traceroute from the Client helps checking if the packet goes the right way and if and where it hits the firewall to trigger routes and rules.



  • @JeGr said in Problem with client connect through static IP cable internet:

    @paprikawuerzung said in Problem with client connect through static IP cable internet:

    Just to be 100% clear on that: without the floating rule the clients are not able to access the internet. Even IP-based traffic (like ssh to an IP) is not possible without the floating rule. Pinging specific IPs works, everything else does not work.

    And to be clear on our side: a floating rule is NOT needed for proper functionality. IF things work only then, you have - as already said - a deeper problem or serious configuration flaw that you didn't see/show. As outbound rules are created automatically in the background, something definetly doesn't add up. Period.

    Yes, sorry for my tone in the previous posts. I am a little frustrated that at first every post was about DNS and DHCP when I already ruled that out 10 posts ago (for that matter I was sure that its not a problem before I even posted this thread) and I wanted it to be clear that it really, really is not the problem at all.

    I know that you help me with your time and I am very grateful for that!

    To the topic:
    This is a complete fresh install with only the static IP and gateway for the WAN interface added. If it is a configuration problem it is part of pfSense as it (appearently) does NOT "create the outbound rules automatically in the background".
    Since there are no definitive guides on how to configure static WAN IP setups (with UnityMedia/KabelBW) it is highly unclear whether this calls for floating rules or not. I read the sections in the pfSense regarding floating rules and understand that they are only used as a last resort - and that internet normally works out of the box with everything handled perfectly by pfSense - but this is obviously not the case.

    As debugging is simply voodoo and guessing at the moment, you should gather information when you are back on site and post them. Even screens in the VM are void, as it's not the same constellation as on site with the WAN and a LAN client behind it to test.

    The configuration (down to the IP addresses) is exactly the same so it is not completely void.

    I'd also double check DHCP settings and the client on premise: does it have the right IP address and does it match the LAN configuration? Not the first time there was a simple small problem in numbers that got switched around and as the default LAN rules are matched against "LAN net" instead of "any" a DHCP problem can go unnoticed as "firewall access works".

    DHCP on client side works and the IPs, Gateway, Domain etc. are set correctly. I am very sure about that. It is not a configuration error in the DHCP area. Which also shows that when I add the floating rule (without re-requesting a DHCP lease) -> the internet works instantly - without any further configuration of the client or pfSense.

    Traceroute from the Client helps checking if the packet goes the right way and if and where it hits the firewall to trigger routes and rules.

    I will provide screenshots and outputs after I am at the premise again.

    A thousand thanks again for your answers/guidance!


  • LAYER 8 Moderator

    @paprikawuerzung said in Problem with client connect through static IP cable internet:

    as it (appearently) does NOT "create the outbound rules automatically in the background".

    Yes it is. In default configuration. Otherwise check /tmp/rules.debug for yourself and compare it with your DSL configuration.

    @paprikawuerzung said in Problem with client connect through static IP cable internet:

    Since there are no definitive guides on how to configure static WAN IP setups (with UnityMedia/KabelBW) it is highly unclear whether this calls for floating rules or not.

    Why should there? It's completely transparent. If you configure DHCP IP, static IP or PPPoE for exmaple don't matter at all what pfSense creates for automatic rules like automatic outbound rules, default LAN rules etc. It's essentially the same if you don't configure some highly experimental or advanced options on the Interfaces/WAN setup screen.

    and that internet normally works out of the box with everything handled perfectly by pfSense - but this is obviously not the case.

    Yes in your case. In hundreds of others we configured/installed it does. Be it PPPoE, DHCP, static IP4. No matter what :)

    The configuration (down to the IP addresses) is exactly the same so it is not completely void.

    Void for me as in "I can't help or debug with you when this is not the same environment the error happens and the virtual setup isn't configured exactly the same (as in: has some sort of uplink, a test VM as client behind it etc. etc.)". So for debugging the problem it's not really well suited :)



  • @JeGr said in Problem with client connect through static IP cable internet:

    @paprikawuerzung said in Problem with client connect through static IP cable internet:

    as it (appearently) does NOT "create the outbound rules automatically in the background".

    Yes it is. In default configuration. Otherwise check /tmp/rules.debug for yourself and compare it with your DSL configuration.

    It is a complete fresh install with no firewalls added and no additional configuration made apart from adding the static IP. Whether this rule is created or not (I believe you if you say it is) does not make any difference as it does not work.

    @paprikawuerzung said in Problem with client connect through static IP cable internet:

    Since there are no definitive guides on how to configure static WAN IP setups (with UnityMedia/KabelBW) it is highly unclear whether this calls for floating rules or not.

    Why should there? It's completely transparent. If you configure DHCP IP, static IP or PPPoE for exmaple don't matter at all what pfSense creates for automatic rules like automatic outbound rules, default LAN rules etc. It's essentially the same if you don't configure some highly experimental or advanced options on the Interfaces/WAN setup screen.

    and that internet normally works out of the box with everything handled perfectly by pfSense - but this is obviously not the case.

    Yes in your case. In hundreds of others we configured/installed it does. Be it PPPoE, DHCP, static IP4. No matter what :)

    I also configured the PPPoE configuration without any problems at all. The only difference here is that I add a static IP (provided by UM) and a gateway (also provided by UM). Why this should create problems with the firewall is beyond my knowledge...

    The configuration (down to the IP addresses) is exactly the same so it is not completely void.

    Void for me as in "I can't help or debug with you when this is not the same environment the error happens and the virtual setup isn't configured exactly the same (as in: has some sort of uplink, a test VM as client behind it etc. etc.)". So for debugging the problem it's not really well suited :)

    Ok. I will provide screenshots from the running system.



  • https://docs.netgate.com/pfsense/en/latest/routing/connectivity-troubleshooting.html

    It's always something from this list. I would do a packet capture on LAN to see if client packets are hitting the interface at all, and another on WAN to see if they're being forwarded out.



  • I fixed the error by adding a WAN firewall rule. In the default installation, this rule was missing:

    Screenshot_2019-07-05 pfSense office patagona de - Firewall Rules Edit.png

    I found the rule in another pfSense installation (= DSL internet) which I compared to the (now working) cable internet installation. After adding it, everything works just fine. The floating rule was overkill but essentially did the same thing.
    I am still a little confused why this rule was not created automatically (as it was in the PPPoE DSL installation). Also, there were strange issues when booting the Fritz!Box BEFORE booting pfSense - even with reboots. In the last installation, everything worked out just fine and with no hassle - strange, strange... But now everything seems to be working just as I want it to 😄

    Thousand thanks for your time and suggestions, @JeGr, @kiokoman, @KOM and @maverickws ! 🙂



  • No, you still have a problem. You shouldn't have ANY rules on WAN except for Block RFC1918, and any port-forwards you have defined. You do NOT need a WAN rule to get Internet access from LAN. You have something weird going on with your installation or environment.



  • @KOM said in (SOLVED) Problem with client connect through static IP cable internet:

    No, you still have a problem. You shouldn't have ANY rules on WAN except for Block RFC1918, and any port-forwards you have defined. You do NOT need a WAN rule to get Internet access from LAN. You have something weird going on with your installation or environment.

    I am little frustrated by these comments suggesting the configuration is strange: my configuration (as I told at least 5 times) is done completely from a clean installation (factory reset on pfSense or even new installation). I only added (as I told at least 5 times) only the static IP (+ Gateway) given by UnityMedia and left everything apart from the (global) DNS servers (which arguably are really, really not the problem) as default.

    The Fritz!Box 6490 is factory reset (with IT'S public, static IP defined/put there by UnityMedia) and apart from disabling the WLAN, I did not change anything. This public, static IP defined in the Fritz!Box (and provided by UnityMedia) is my gateway in pfSense. This setup was suggested by three different technicians at UnityMedia and most likely is not the culprit.
    So, I would argue that my setup or configuration is not special in any strange way.



  • I am not sure whether this helps but this is the output of /tmp/rules.debug

    (My added rule is in there also)



  • Wait, you have Snort installed?? I assumed that you were using a default config to get it going without installing any extra packages. If that is the case, please disable ALL packages that may interfere with traffic, such as Snort, Suricata, pfBlocker, squidguard.



  • @paprikawuerzung said in (SOLVED) Problem with client connect through static IP cable internet:

    Fritz!Box 6490

    Your modem is a Puma 6 powered model and AFAIC a piece of crap. But my guess is that you are stuck with it. (badmodems.com)

    That model is a gateway model and is still routing as opposed to true bridge mode.

    How many static addresses did they give you? The /30 surprises me a little.



  • @KOM said in (SOLVED) Problem with client connect through static IP cable internet:

    Wait, you have Snort installed?? I assumed that you were using a default config to get it going without installing any extra packages. If that is the case, please disable ALL packages that may interfere with traffic, such as Snort, Suricata, pfBlocker, squidguard.

    I have nothing installed which is not in a fresh install. I added "OpenVPN Client Export" as a plugin just now - but apart from that I enabled/installed no package.



  • Oh OK. I saw references in the rules to snort2c sets and I jumped to conclusions. Those types of packages are notorious for causing LAN traffic problems when not configured properly. I'm not sure what else to add. It just works out of the box for literally thousands of people. Have you gone through the Connectivity guide I linked to step by step?

    Also, my suggestion for packet capture is still valid.



  • @chpalmer said in (SOLVED) Problem with client connect through static IP cable internet:

    @paprikawuerzung said in (SOLVED) Problem with client connect through static IP cable internet:

    Fritz!Box 6490

    Your modem is a Puma 6 powered model and AFAIC a piece of crap. But my guess is that you are stuck with it. (badmodems.com)

    That model is a gateway model and is still routing as opposed to true bridge mode.

    How many static addresses did they give you? The /30 surprises me a little.

    Hi @chpalmer: I have 1 static IP assigned by the ISP. The other IPs in the subnet are the network IP, broadcast and gateway (4 hosts per /30 subnet) as far as I know.



  • @KOM said in (SOLVED) Problem with client connect through static IP cable internet:

    Oh OK. I saw references in the rules to snort2c sets and I jumped to conclusions. Those types of packages are notorious for causing LAN traffic problems when not configured properly. I'm not sure what else to add. It just works out of the box for literally thousands of people. Have you gone through the Connectivity guide I linked to step by step?

    I have gone through the connectivity guide before, yes. The answers to all the questions in the troubleshooting guide are scattered through this thread.

    Also, my suggestion for packet capture is still valid.

    I am not exactly positive that this will add more to our understanding. The WAN firewall rule allowing traffic from the WAN to the LAN network "fixes" the problem.
    I guess my only question is: does this pose a security threat and how can I test it? If it does not create any further problems and does not compromise the security - I am really not that keen on investigating the problem further. Maybe it got anything to do with the ISP or a quirk of the modem or... or... or...



  • Yes, having a rule on WAN that allows all traffic to LAN is certainly not best practice. It allows access to WebGUI from WAN, for one thing. A packet capture would tell you if LAN is even seeing the traffic, and the WAN capture would show if it's leaving for the Internet.



  • @KOM said in (SOLVED) Problem with client connect through static IP cable internet:

    Yes, having a rule on WAN that allows all traffic to LAN is certainly not best practice. It allows access to WebGUI from WAN, for one thing. A packet capture would tell you if LAN is even seeing the traffic, and the WAN capture would show if it's leaving for the Internet.

    This is not a problem since pfSense is behind the gateway which in turn does not expose pfSense (and therefor the WebGUI is not seen) to the outside. I already did a full port scan with nmap on both the gateway and the pfSense IPs - no ports open (of course there are hidden ports - but no response from them).

    I assume that especially since the Fritz!Box 6490 Cable does not simply do bridging but instead also some routing/firewalling by itself, this setup is not more insecure than a normal Fritz!Box setup. But of course I am not entirely sure about that.

    I will try a packet capture tomorrow - but I strongly suspect that the packet leaves pfSense but the firewall blocks the response. Why it only does this when using TCP/UDP (layer-4) traffic and not ping is beyond my knowledge.
    I will provide the results tomorrow.


Log in to reply