Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    HELP PLSS, Internet access issues with pfSense behind an ISP router (double NAT + VLANs on a switch)

    Scheduled Pinned Locked Moved L2/Switching/VLANs
    19 Posts 2 Posters 539 Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • johnpozJ Online
      johnpoz LAYER 8 Global Moderator @HidekiSenpai
      last edited by

      @HidekiSenpai said in HELP PLSS, Internet access issues with pfSense behind an ISP router (double NAT + VLANs on a switch):

      Hi, I've run the packet capture test and it's not capturing anything.

      If you capture and your not seeing anything.. Then how exactly is it showing you your gateway is up? Pfsense knows its gateway is up by sending pings to the gateway..

      An intelligent man is sometimes forced to be drunk to spend time with his fools
      If you get confused: Listen to the Music Play
      Please don't Chat/PM me for help, unless mod related
      SG-4860 25.07.1 | Lab VMs 2.8.1, 25.07.1

      HidekiSenpaiH 1 Reply Last reply Reply Quote 0
      • HidekiSenpaiH Offline
        HidekiSenpai @johnpoz
        last edited by HidekiSenpai

        @johnpoz I have been doing the pings from Diagnostics → Ping in pfSense

        johnpozJ 1 Reply Last reply Reply Quote 0
        • johnpozJ Online
          johnpoz LAYER 8 Global Moderator @HidekiSenpai
          last edited by johnpoz

          @HidekiSenpai maybe your not sniffing on the correct interface..

          But pfsense shows that its gateway is online by pinging it.. So if yours shows up then clearly pfsense can talk to the gateway - unless you set it to be always online in the routing section?

          gateway.jpg

          But if pfsense knows the mac address of the gateway - even if it doesn't answer ping, a packet capture would show it sending pings.. Do you get some other error other than timeout?

          show your arp table in pfsense. Show your routing..

          routes-arp.jpg

          This shows the default gateway.. This shows the arp table.. Even if that gateway IP didn't answer ping, if pfsense knows the mac it would send the ping and you would see it in the packet capture.. If you are not seeing it send on the ping then pfsense does not know the mac, or your sniffing on the wrong interface to where it would send it.. The routing table will show what IP pfsense has on its wan and the network, etc..

          But if pfsense does not know the mac address of its gateway, and have a default gateway to to send traffic to it - then no it would never work. No matter if it answers ping or not.. If pfsense can not actually talk to its gateway then its never going to work.

          maybe you have wrong mask on your lan, and this is overlapping your wan 192.168.1.0/24 network?

          So lets see this info via arp table and routing table.

          testing from cmd line (ssh to your pfsense) might give you bit more info as well.. So for example I try to ping an IP that doesn't exist on pfsense wan 192.168.3.0/24 network

          [2.8.0-RELEASE][admin@pfSense.test.home.arpa]/root: ping 192.168.3.42
          PING 192.168.3.42 (192.168.3.42): 56 data bytes
          ping: sendto: Host is down
          ping: sendto: Host is down
          ping: sendto: Host is down
          
          [2.8.0-RELEASE][admin@pfSense.test.home.arpa]/root: arp -a
          ? (192.168.3.42) at (incomplete) on em0 expired [ethernet]
          

          So the error is just not a timeout, but saying host is down, if then look in the arp table it was unable to find a mac for that IP.. See where it says incomplete and shows no mac address.

          An intelligent man is sometimes forced to be drunk to spend time with his fools
          If you get confused: Listen to the Music Play
          Please don't Chat/PM me for help, unless mod related
          SG-4860 25.07.1 | Lab VMs 2.8.1, 25.07.1

          HidekiSenpaiH 1 Reply Last reply Reply Quote 0
          • HidekiSenpaiH Offline
            HidekiSenpai @johnpoz
            last edited by

            @johnpoz Here you have some attached images of how I have everything
            Captura de pantalla 2025-08-30 154403.png Captura de pantalla 2025-08-30 154425.png Captura de pantalla 2025-08-30 154505.png

            johnpozJ 1 Reply Last reply Reply Quote 0
            • johnpozJ Online
              johnpoz LAYER 8 Global Moderator @HidekiSenpai
              last edited by johnpoz

              @HidekiSenpai I show no gateway there, And what are those 80 address on the same interface as your wan icg0 interface.

              Without a gateway no pfsense is not going to be able talk to anything other than what its connected too.. So is that 192.168.1.1 suppose to be your gateway?

              Your saying you can not ping 192.168.1.74 or what I assume should be your gateway 192.168.1.1? You don't even see pfsense send the ping request??

              An intelligent man is sometimes forced to be drunk to spend time with his fools
              If you get confused: Listen to the Music Play
              Please don't Chat/PM me for help, unless mod related
              SG-4860 25.07.1 | Lab VMs 2.8.1, 25.07.1

              HidekiSenpaiH 1 Reply Last reply Reply Quote 0
              • HidekiSenpaiH Offline
                HidekiSenpai @johnpoz
                last edited by HidekiSenpai

                @johnpoz 192.168.1.1 is my gateway 192.168.1.74 I don't know where it came from and 192.168.1.40 is the address of my igc0 WAN interface

                johnpozJ 1 Reply Last reply Reply Quote 0
                • johnpozJ Online
                  johnpoz LAYER 8 Global Moderator @HidekiSenpai
                  last edited by johnpoz

                  @HidekiSenpai well its never going to work if you have no gateway.. but you should still be able to see a packet capture trying to ping it. Even if it doesn't answer ping.

                  Your saying pfsense shows your gateway is up? Lets see your gateways and add the gateway widget to your dashboard

                  gateway.jpg

                  Because from your routing table you have no gateway at all.. Only thing pfsense could talk to would be things directly connected to it.

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 25.07.1 | Lab VMs 2.8.1, 25.07.1

                  HidekiSenpaiH 1 Reply Last reply Reply Quote 0
                  • HidekiSenpaiH Offline
                    HidekiSenpai @johnpoz
                    last edited by HidekiSenpai

                    @johnpoz
                    Captura de pantalla 2025-08-30 164656.png Captura de pantalla 2025-08-30 164853.png

                    Now for some reason it appears as connected and before not even that, and I haven't touched anything, the only thing I've done is add the gateways widget to the dashboard and that's it (it has nothing to do with it), what I did do a while ago is add the pfSense WAN IP to the DMZ of the ISP router, and it's possible that it has been applied now, I don't know.
                    Now this doesn't mean it's fixed, because it says connected but it doesn't load the pages or Discord or Spotify correctly, it's like it wants to load it but can't
                    Captura de pantalla 2025-08-30 165129.pngCaptura de pantalla 2025-08-30 170017.png

                    johnpozJ 1 Reply Last reply Reply Quote 0
                    • johnpozJ Online
                      johnpoz LAYER 8 Global Moderator @HidekiSenpai
                      last edited by johnpoz

                      @HidekiSenpai well what your gateways and your widget show is not what your routing table showed.. Has that changed?

                      You had no default route in your routing table.. And not sure where your showing that red 4 as some sort of connection.. But if your device is actually behind pfsense it sure and the hell wouldn't show that as the connection.

                      That name lines up with the name of those 80 IPs you showed on pfsense though

                      inetnum:        80.58.61.248 - 80.58.61.255
                      netname:        RIMA
                      descr:          Red de Servicios IP
                      country:        ES
                      

                      Seems to me your client that says its connection is Red 4 isn't actually behind pfsense.. What does the IPconfig show on that device.. You should see pfsense 192.168.2.1 as your gateway and it should have a 192.168.2 address.

                      Clearly from your gateways and widget you can ping 192.168.1.1 - that is how pfsense knows its online.. But you said when you tried to ping it you got no answer and didn't even see anything on your packet capture.. Do you have some sort of vpn setup on pfsense? Maybe that is where those 80 IPs came from??

                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                      If you get confused: Listen to the Music Play
                      Please don't Chat/PM me for help, unless mod related
                      SG-4860 25.07.1 | Lab VMs 2.8.1, 25.07.1

                      HidekiSenpaiH 1 Reply Last reply Reply Quote 0
                      • HidekiSenpaiH Offline
                        HidekiSenpai @johnpoz
                        last edited by

                        @johnpoz Network 4 is a Windows computer connected to pfSense, and I'm now realizing that the 80 IPs you mentioned are the default DNS settings on the ISP router.

                        I also tried to set up an IPsec VPN, although it didn't quite work, so I disabled it for now.

                        And regarding pings, from the 192.168.2.0/24 network, the ping is successful on both the WAN and LAN to the ISP router's gateway (192.168.1.1), but the packets are dropped, and nothing appears in the packet capture when I ping 8.8.8.8.
                        And from the 192.168.1.0/24 network, when I ping the pfSense LAN gateway (192.168.2.1), all packets are dropped, however, the ISP router detects it. to pfSense, so that's a matter for the pfSense firewall rules.

                        What I'm thinking now is that, for the 192.168.2.0/24 network to reach the internet, does the router need to have access to pfSense? Or what?

                        johnpozJ 1 Reply Last reply Reply Quote 0
                        • johnpozJ Online
                          johnpoz LAYER 8 Global Moderator @HidekiSenpai
                          last edited by

                          @HidekiSenpai no, pfsense is just another client on your isp routers network. Your isp router wouldn't know anything about a 192.168.2 address since pfsense would nat everything to its wan IP when a 192.168.2 wanted to go to 8.8.8.8

                          If your not seeing a packet capture on pfsense when you ping 8.8.8.8 then that traffic isn't going through pfsense.

                          Lets see your routing table again - if there is no default then pinging 8.8.8.8 would never be sent to your isp router at 192.168.1.1, so no you wouldn't see it via a packet capture.

                          Maybe try setting your gateway in routing as default vs automatic.. But if you do not see a default route in routes then no it would never work.

                          default.jpg

                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                          If you get confused: Listen to the Music Play
                          Please don't Chat/PM me for help, unless mod related
                          SG-4860 25.07.1 | Lab VMs 2.8.1, 25.07.1

                          HidekiSenpaiH 1 Reply Last reply Reply Quote 0
                          • HidekiSenpaiH Offline
                            HidekiSenpai @johnpoz
                            last edited by

                            @johnpoz I did what you told me about changing the gateway from automatic to default, and it's working for now.
                            a8d77c7d-4be4-4a8d-a841-fe49e5a7c73d-image.png

                            It also takes a long time to load pages, Spotify takes a long time to load songs, etc.

                            Why could that be?

                            HidekiSenpaiH 1 Reply Last reply Reply Quote 0
                            • HidekiSenpaiH Offline
                              HidekiSenpai @HidekiSenpai
                              last edited by HidekiSenpai

                              I think it's a DNS issue that seems to resolve but then it gives a time out

                              I changed the ISP's DNS and disabled the "DNS Server Override" option so that it only uses the DNS that I have established, which are 8.8.8.8 and 8.8.4.4

                              And it doesn't do nslookup even from the WAN interface

                              johnpozJ 1 Reply Last reply Reply Quote 0
                              • johnpozJ Online
                                johnpoz LAYER 8 Global Moderator @HidekiSenpai
                                last edited by

                                @HidekiSenpai what does it show in your dns lookup diag..

                                An intelligent man is sometimes forced to be drunk to spend time with his fools
                                If you get confused: Listen to the Music Play
                                Please don't Chat/PM me for help, unless mod related
                                SG-4860 25.07.1 | Lab VMs 2.8.1, 25.07.1

                                HidekiSenpaiH 1 Reply Last reply Reply Quote 0
                                • HidekiSenpaiH Offline
                                  HidekiSenpai @johnpoz
                                  last edited by HidekiSenpai

                                  @johnpoz I disabled IPv6 entirely, in case there was a conflict, and after doing so, it started resolving queries with external DNS, but the response is not authoritative.
                                  consulta a dns externa.png

                                  And then if I make a normal query, it's using 8.8.8.8, it doesn't recognize the server, and I get this:
                                  consulta.png

                                  But if I make a direct query to 192.168.2.1 (which is the pfSense LAN), it recognizes the server but gives an error:
                                  consulta directa a pfSense.png

                                  Here are my unbound settings:
                                  Captura de pantalla 2025-09-02 145314.png Captura de pantalla 2025-09-02 145329.png

                                  johnpozJ 1 Reply Last reply Reply Quote 0
                                  • johnpozJ Online
                                    johnpoz LAYER 8 Global Moderator @HidekiSenpai
                                    last edited by johnpoz

                                    @HidekiSenpai not sure why you think a query to quad9 would be authoritative.. quad9 is not the authoritative ns for google.com

                                    Your unbound setting there are resolver mode, for you to be able to resolve you would have to be able to talk to all the NS on port 53.. If your upstream is blocking this then yeah your going to have issues.

                                    What does dns lookup on the diagnostic menu dns lookup report?

                                    To test if you can resolve and to see where you might be having issues do a dig + trace on pfsense.

                                    [25.07.1-RELEASE][admin@sg4860.home.arpa]/root: dig google.com +trace
                                    
                                    ; <<>> DiG 9.20.6 <<>> google.com +trace
                                    ;; global options: +cmd
                                    .                       84617   IN      NS      d.root-servers.net.
                                    .                       84617   IN      NS      f.root-servers.net.
                                    .                       84617   IN      NS      e.root-servers.net.
                                    .                       84617   IN      NS      m.root-servers.net.
                                    .                       84617   IN      NS      j.root-servers.net.
                                    .                       84617   IN      NS      b.root-servers.net.
                                    .                       84617   IN      NS      c.root-servers.net.
                                    .                       84617   IN      NS      g.root-servers.net.
                                    .                       84617   IN      NS      k.root-servers.net.
                                    .                       84617   IN      NS      l.root-servers.net.
                                    .                       84617   IN      NS      a.root-servers.net.
                                    .                       84617   IN      NS      h.root-servers.net.
                                    .                       84617   IN      NS      i.root-servers.net.
                                    .                       84617   IN      RRSIG   NS 8 0 518400 20250915050000 20250902040000 46441 . r2EKEjvLOSDMWT4XAMJK+3McQntRgJ/wtG2WXCZ90DdKxUgNUCU1Q1R+ YDovtNQExt87dM1gu8S10al5FJPNkLM6pbQM010+1E2AnyCQyt4DQrJh JgMhwcYONIbT/gGrXfQS7sdN8B5g0ob2HcqXRxqMkDOldxdBCJy7B5ZM AufoQlrCrdazkGHVxC+vzsDIDVYnAFLlLkoHtcpbLmiK1w6MiVNfzfWt EC4v7Bibau5rMYzhYZ0EwGv4CCG6dn8HiGEg0rNBmMi7onXndKhq2S4H T9b1jkIj1qG1GfVOzVuqmzv7OWgW9+0jbqel3VR7AAfO9plH7JLeVNY1 EmTLTg==
                                    ;; Received 525 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms
                                    
                                    com.                    172800  IN      NS      a.gtld-servers.net.
                                    com.                    172800  IN      NS      b.gtld-servers.net.
                                    com.                    172800  IN      NS      c.gtld-servers.net.
                                    com.                    172800  IN      NS      d.gtld-servers.net.
                                    com.                    172800  IN      NS      e.gtld-servers.net.
                                    com.                    172800  IN      NS      f.gtld-servers.net.
                                    com.                    172800  IN      NS      g.gtld-servers.net.
                                    com.                    172800  IN      NS      h.gtld-servers.net.
                                    com.                    172800  IN      NS      i.gtld-servers.net.
                                    com.                    172800  IN      NS      j.gtld-servers.net.
                                    com.                    172800  IN      NS      k.gtld-servers.net.
                                    com.                    172800  IN      NS      l.gtld-servers.net.
                                    com.                    172800  IN      NS      m.gtld-servers.net.
                                    com.                    86400   IN      DS      19718 13 2 8ACBB0CD28F41250A80A491389424D341522D946B0DA0C0291F2D3D7 71D7805A
                                    com.                    86400   IN      RRSIG   DS 8 1 86400 20250915050000 20250902040000 46441 . PuEt7PPZTytXpON7kI4PR4ePmn1RbbZwWwksIwQqStFADSXkHLtaCWBk 6rjtDQogfGqqcRZnJzXTwq7FD+lsB//y3DBBkzBB+ag7XmldiFGtkV3Y 9ueUEL4ydZnyftPClzOtBYbtzMVA2oC6gfNbi7LyIFUUH8xc0IZUPJah 9IQF443ZocHNNl8jPpSilA7QVkSf6rKRH5CNUdTsJ6qhfXUEOWgNqIaV yLCrPzsnyl7+PoU1dBpPmsbUY0DUO2A0E5Zs5lBpcgjThoEK/SMokB1v Rb75/7Yvb+MGyDWmZVwd9uKdVadxzn6jdJgxgSM+SBuxaSpkWlnqhJnx fYnP/w==
                                    ;; Received 1170 bytes from 2001:503:c27::2:30#53(j.root-servers.net) in 9 ms
                                    
                                    google.com.             172800  IN      NS      ns2.google.com.
                                    google.com.             172800  IN      NS      ns1.google.com.
                                    google.com.             172800  IN      NS      ns3.google.com.
                                    google.com.             172800  IN      NS      ns4.google.com.
                                    CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 900 IN NSEC3 1 1 0 - CK0Q3UDG8CEKKAE7RUKPGCT1DVSSH8LL NS SOA RRSIG DNSKEY NSEC3PARAM
                                    CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 900 IN RRSIG NSEC3 13 2 900 20250908002603 20250831231603 20545 com. I5bq7mPPNzfXbaaD27hOUwaUOIQJi6EcJYwN+Ab4FiMqp5GgoHWsfgSm LHUn2Mg3jXAGfxykTCnJXfUQYtJ+oQ==
                                    S84BOR4DK28HNHPLC218O483VOOOD5D8.com. 900 IN NSEC3 1 1 0 - S84BR9CIB2A20L3ETR1M2415ENPP99L8 NS DS RRSIG
                                    S84BOR4DK28HNHPLC218O483VOOOD5D8.com. 900 IN RRSIG NSEC3 13 2 900 20250909012623 20250902001623 20545 com. 1Sn2h2Xvf9GUFWqqEDwCOD+aZFVhrEhV+87H/RxeCGuNoA42E7tz5Oq6 A7hnIkd0J8coWN0C9M9gQlJLjrrfvw==
                                    ;; Received 644 bytes from 192.26.92.30#53(c.gtld-servers.net) in 27 ms
                                    
                                    google.com.             300     IN      A       172.217.2.46
                                    ;; Received 55 bytes from 216.239.36.10#53(ns3.google.com) in 25 ms
                                    
                                    

                                    The dig + trace is exactly what the resolver would do - so seeing all the steps can show you were you might be failing in the process.

                                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                                    If you get confused: Listen to the Music Play
                                    Please don't Chat/PM me for help, unless mod related
                                    SG-4860 25.07.1 | Lab VMs 2.8.1, 25.07.1

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.