Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Ping not working on CLI or with "default" interface on GUI

    Scheduled Pinned Locked Moved General pfSense Questions
    24 Posts 2 Posters 1.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Is the gateway actually set on the interface itself? In Interfaces > WAN?

      That's what pfSense uses to identify it as a wan and triggers route-to on firewall rules.

      If you ping out without setting a source and check the state table do you see a state created? On which interface?

      Steve

      L 1 Reply Last reply Reply Quote 0
      • L
        llavecita @stephenw10
        last edited by llavecita

        Hello @stephenw10 ,

        Thank you for your answer.

        I confirm the gateway is set on WAN interface, and it is also set as default gateway.

        When pinging out to ie. 8.8.8.8, I confirm there is a state in WAN interface, and it seems it is trying to be sent as 0.0.0.0. That is odd.

        WAN	icmp	0.0.0.0:1836 -> 8.8.8.8:1836	0:0	84 / 0	7 KiB / 0 B
        

        Now the question would be why WAN is using 0.0.0.0 for egress traffic if no source IP is selected. WAN currently has a few IP addresses, which are (redacted to random IPs):
        123.123.123.12/25, which is the one assigned by the datacenter
        80.80.80.1/32, which is from a prefix I am announcing from this server, so it is pingable. This is a Virtual IP.
        90.90.90.1/32, which is from another prefix I am announcing from this server, so it is pingable. This is a Virtual IP.

        I remember there was a bug where if a WAN interface had Virtual IPs assigned, it could break things. But I think that was fixed?

        Thanks a lot!

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          Yes, that should be fixed but I agree that's exactly what came to mind when seeing that.

          I assume you can ping out from any of VIPs on WAN if you set them as source?

          Do you have Outbound NAT set to automatic still? That should prevent it NATing it's own traffic but if it's using manual more something might be over-matching.

          Steve

          L 1 Reply Last reply Reply Quote 0
          • L
            llavecita @stephenw10
            last edited by

            @stephenw10

            Thank you for your reply.

            NAT Outbound is set as manual, and it has nothing, other than the rules that come in pfSense by default, which are:

            WAN	127.0.0.0/8	*	*	500 (ISAKMP)	WAN address	*			  
            WAN	127.0.0.0/8	*	*	*	WAN address	*			  
            WAN	::1/128	*	*	500 (ISAKMP)	WAN address	*			  
            WAN	::1/128	*	*	*	WAN address	*			  
            

            I confirm I can ping from both VIPs (as well as WAN, when WAN is selected) from both GUI (selecting the relevant virtual IP or interface) or CLI (specifying the source with ping -S <SRC_IP> <DST_IP>)

            Thanks a lot!

            L 1 Reply Last reply Reply Quote 0
            • L
              llavecita @llavecita
              last edited by llavecita

              I have now deleted both VIPs, so WAN only has one allocated IP (in case it was the Virtual IPs bug I was referring to), but the problem still persists. I have also set NAT Outbound to Automatic to no avail, so I just rolled it back to Manual.

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Hmm, if you run a pcap on WAN is it actually sending traffic with source 0.0.0.0?

                L 1 Reply Last reply Reply Quote 0
                • L
                  llavecita @stephenw10
                  last edited by llavecita

                  @stephenw10

                  That is correct. This is the excerpt of the Packet Capture:

                  17:29:56.826861 IP 0.0.0.0 > 8.8.8.8: ICMP echo request, id 1836, seq 4001, length 64
                  

                  I am attaching the exported pcap for your reference, with the corresponding ping packets to 8.8.8.8. capture.pcap

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    If you resave the WAN config does it stop?

                    Was the WAN ever configured as DHCP? Is there a dhclient process running? Anything in the dhcp logs?
                    That's the only time seeing 0.0.0.0 might be expected.

                    Steve

                    L 1 Reply Last reply Reply Quote 0
                    • L
                      llavecita @stephenw10
                      last edited by

                      @stephenw10

                      I have re-saved WAN interface configuration, and applied it. Problem still persists, unfortunately.

                      DHCP logs show one entry from yesterday (2023-10-25 18:18:20.595596+00:00 dhclient 5829 Cannot open or create pidfile: No such file or directory), and a few entries for the date when the server was deployed, on 20th of October.

                      There seems to be a dhclient process running, which keeps changing the PID (second column) incrementally. Maybe it is restarting all the time?

                      $ ps aux | grep dhclient
                      root    40010    0.0  0.0  12768   2424  0  S+   17:50       0:00.00 grep dhclient
                      
                      $ ps aux | grep dhclient
                      root    40711    0.0  0.0  12768   2432  0  S+   17:50       0:00.00 grep dhclient
                      
                      $ ps aux | grep dhclient
                      root    41154    0.0  0.0  12768   2432  0  S+   17:50       0:00.00 grep dhclient
                      
                      $ ps aux | grep dhclient
                      root    41588    0.0  0.0  12768   2432  0  S+   17:50       0:00.00 grep dhclient
                      

                      Thanks a lot!

                      L 1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Hmm, that's not right! There shouldn;t be any dhclient there if it's all static IPs.

                        If you run ps auxwwd | grep dhclient you should be able to see which interface it's running on.

                        I would first try just killing that process.

                        1 Reply Last reply Reply Quote 0
                        • L
                          llavecita @llavecita
                          last edited by

                          Oh sorry, my brain stopped working after chasing this for too long today. The command above shows grep dhclient process which I am spinning up when running ps aux | grep dhclient, not dhclient process itself. That is why the PID kept incrementing, because I was invoking it all the time. Facepalm.

                          So yes, there is no dhclient process running, and therefore we cannot kill it.

                          My apologies. I will continue the research tomorrow and try to figure out why WAN is using 0.0.0.0 as SRC when no address is specified, as it is clear brain fog is kicking in.

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Ah, yes, I should have spotted that!

                            I guess I'd try disabling wireguard as a test. There's not much else it could be....

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Hmm, do you have a Wireguard gateway defined?

                              Is your default IPv4 gateway still set to automatic in System > Routing > Gateways?
                              If it is try setting it to the WAN gateway.

                              L 1 Reply Last reply Reply Quote 0
                              • L
                                llavecita @stephenw10
                                last edited by llavecita

                                Hello again @stephenw10 . New day, new time to chase this mystery.

                                I have stopped WireGuard service, but issue still persists.

                                In regards to IPv4 gateway, it is not automatic - it is statically defined. It is the only Gateway in this system (no WireGuard gateways or anything extra), and it is also set as default. Puzzling.

                                I have tried to add a NAT outbound rule on top of the list, selecting "any" as source on WAN interface, and translating it to the "interface address", and ping works now when not specifying an interface. But this is a dirty patch, as I assume WAN is still sending the packets with 0.0.0.0 as source, just that we are translating that address locally. On top of that, nslookup still does not work even with that workaround, ie:

                                $ nslookup dns.google
                                ;; UDP setup with 1.1.1.2#53(1.1.1.2) for dns.google failed: host unreachable.
                                ;; UDP setup with 1.1.1.2#53(1.1.1.2) for dns.google failed: host unreachable.
                                ;; UDP setup with 1.1.1.2#53(1.1.1.2) for dns.google failed: host unreachable.
                                ;; UDP setup with 9.9.9.9#53(9.9.9.9) for dns.google failed: host unreachable.
                                

                                And pfsense updates also don't work. So I am reverting that NAT outbound rule.

                                I will continue to chase it for a few hours more and, if no luck, I will try reinstalling the system and observe behaviour when freshly installed and no config.xml restores.

                                1 Reply Last reply Reply Quote 0
                                • L
                                  llavecita
                                  last edited by llavecita

                                  Here is some additional info, in case it helps

                                  $ ifconfig -a
                                  
                                  igb0: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
                                  	options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
                                  	ether 3c:ec:ef:a7:22:e8
                                  	media: Ethernet autoselect
                                  	status: no carrier
                                  	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                                  
                                  igb1: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
                                  	options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
                                  	ether 3c:ec:ef:a7:22:e9
                                  	media: Ethernet autoselect
                                  	status: no carrier
                                  	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                                  
                                  ix0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
                                  	description: WAN
                                  	options=48138b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,NOMAP>
                                  	ether 9c:69:b4:64:96:7a
                                  	inet6 fe80::9e69:b4ff:fe64:967a%ix0 prefixlen 64 scopeid 0x3
                                  	inet 123.123.123.123(redacted) netmask 0xffffff80 broadcast 123.123.123.1(redacted)
                                  	media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
                                  	status: active
                                  	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                                  
                                  ix1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
                                  	description: OPT1
                                  	options=48138b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,NOMAP>
                                  	ether 9c:69:b4:64:96:7b
                                  	inet6 fe80::9e69:b4ff:fe64:967b%ix1 prefixlen 64 scopeid 0x4
                                  	media: Ethernet autoselect
                                  	status: no carrier
                                  	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                                  
                                  enc0: flags=0<> metric 0 mtu 1536
                                  	groups: enc
                                  	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                                  
                                  lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
                                  	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
                                  	inet6 ::1 prefixlen 128
                                  	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
                                  	inet 127.0.0.1 netmask 0xff000000
                                  	groups: lo
                                  	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                                  
                                  pflog0: flags=100<PROMISC> metric 0 mtu 33152
                                  	groups: pflog
                                  
                                  pfsync0: flags=0<> metric 0 mtu 1500
                                  	maxupd: 128 defer: off
                                  	syncok: 1
                                  	groups: pfsync
                                  
                                  tun_wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420
                                  	description: WG_EG
                                  	options=80000<LINKSTATE>
                                  	inet 80.80.80.1(redacted) netmask 0xffffff00
                                  	inet 90.90.90.1(redacted) netmask 0xffffff00
                                  	inet 10.95.32.99 netmask 0xffffff00
                                  	groups: wg WireGuard
                                  	nd6 options=101<PERFORMNUD,NO_DAD>
                                  

                                  I also verified that in the uploaded .pcap file, the MAC address of the source matches the one in ix0 interface.

                                  stephenw10S 1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator @llavecita
                                    last edited by

                                    @llavecita said in Ping not working on CLI or with "default" interface on GUI:

                                    inet 123.123.123.123(redacted) netmask 0xffffff80 broadcast 123.123.123.1(redacted)

                                    Is that the other way around? I expect the broadcast address to be .127 on a /25.

                                    L 1 Reply Last reply Reply Quote 0
                                    • L
                                      llavecita @stephenw10
                                      last edited by

                                      @stephenw10 I redacted the real values, but you are indeed correct - shown broadcast address in ifconfig for interface ix0 is a .127 (and gateway is a .126)

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Hmm, OK. We discussed this internally last night and there's nothing immediately that presents like that.

                                        Can I assume this behaviour survives a reboot?

                                        L 1 Reply Last reply Reply Quote 0
                                        • L
                                          llavecita @stephenw10
                                          last edited by

                                          @stephenw10 Correct, the server has been rebooted several times to no avail. I will find some time today to reinstall it from scratch, without dragging any previous config.xml, to see if behaviour persists on a fresh install. I will report back later today.

                                          Thanks again for your time and dedication!

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Yeah, I really like to know what's causing that. Must be something very obscure as I see almost zero references to it.

                                            L 2 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.