Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Intermittent "no route to host" on my LAN-port

    Scheduled Pinned Locked Moved General pfSense Questions
    35 Posts 3 Posters 12.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      tsmalmbe
      last edited by

      When the problem (which is now different but the same) occurs, the firewall can now ping everything so no problem there. But now I lose

      • VLAN (LAN) -> ovpns5,
      • VLAN (LAN)->VLAN (SERVERS1) and
      • VLAN (LAN) -> VLAN (SERVERS2).

      VLAN (LAN) to the internet is all fine and working.

      There has to be some way to debug this.

      Security Consultant at Mint Security Ltd - www.mintsecurity.fi

      1 Reply Last reply Reply Quote 0
      • T
        tsmalmbe
        last edited by

        Dumping the routing when the issue occurs:

        [2.3.4-RELEASE][root@firewall.ccccccccc.fi]/root: netstat -rn
        Routing tables

        Internet:
        Destination        Gateway            Flags      Netif Expire
        default            217.30.178.254    UGS        sk0
        10.10.1.0/24      link#4            U          sk3
        10.10.1.1          link#4            UHS        lo0
        10.99.0.0/16      10.100.100.2      UGS      ovpns5
        10.100.100.1      link#13            UHS        lo0
        10.100.100.2      link#13            UH      ovpns5
        10.100.101.1      link#14            UHS        lo0
        10.100.101.2      link#14            UH      ovpns8
        10.200.200.0/24    10.200.200.2      UGS      ovpns1
        10.200.200.1      link#12            UHS        lo0
        10.200.200.2      link#12            UH      ovpns1
        10.200.210.0/24    10.100.100.2      UGS      ovpns5
        83.145.193.133    217.30.178.254    UGHS        sk0
        127.0.0.1          link#8            UH          lo0
        192.168.1.0/24    link#11            U      sk1_vlan
        192.168.1.1        link#11            UHS        lo0
        192.168.1.3        link#11            UHS        lo0
        192.168.2.21      link#11            UHS        lo0
        192.168.2.21/32    link#11            U      sk1_vlan
        192.168.10.0/24    link#9            U      sk1_vlan
        192.168.10.1      link#9            UHS        lo0
        192.168.20.0/24    link#10            U      sk1_vlan
        192.168.20.1      link#10            UHS        lo0
        192.168.69.0/24    10.100.101.2      UGS      ovpns8
        192.168.100.0/24  link#3            U          sk2
        192.168.100.1      link#3            UHS        lo0
        217.30.178.0/24    link#1            U          sk0
        217.30.178.237    link#1            UHS        lo0

        Security Consultant at Mint Security Ltd - www.mintsecurity.fi

        1 Reply Last reply Reply Quote 0
        • T
          tsmalmbe
          last edited by

          After reboot

          Routing tables

          Internet:
          Destination        Gateway            Flags      Netif Expire
          default            217.30.178.254    UGS        sk0
          10.10.1.0/24      link#4            U          sk3
          10.10.1.1          link#4            UHS        lo0
          10.99.0.0/16      10.100.100.2      UGS      ovpns5
          10.100.100.1      link#13            UHS        lo0
          10.100.100.2      link#13            UH      ovpns5
          10.100.101.1      link#14            UHS        lo0
          10.100.101.2      link#14            UH      ovpns8
          10.200.200.0/24    10.200.200.2      UGS      ovpns1
          10.200.200.1      link#12            UHS        lo0
          10.200.200.2      link#12            UH      ovpns1
          10.200.210.0/24    10.100.100.2      UGS      ovpns5
          83.145.193.133    217.30.178.254    UGHS        sk0
          127.0.0.1          link#8            UH          lo0
          192.168.1.0/24    link#11            U      sk1_vlan
          192.168.1.1        link#11            UHS        lo0
          192.168.1.3        link#11            UHS        lo0
          192.168.2.21      link#11            UHS        lo0
          192.168.2.21/32    link#11            U      sk1_vlan
          192.168.10.0/24    link#9            U      sk1_vlan
          192.168.10.1      link#9            UHS        lo0
          192.168.20.0/24    link#10            U      sk1_vlan
          192.168.20.1      link#10            UHS        lo0
          192.168.69.0/24    10.100.101.2      UGS      ovpns8
          192.168.100.0/24  link#3            U          sk2
          192.168.100.1      link#3            UHS        lo0
          217.30.178.0/24    link#1            U          sk0
          217.30.178.237    link#1            UHS        lo0

          Security Consultant at Mint Security Ltd - www.mintsecurity.fi

          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate
            last edited by

            As far as I know, almost nobody is using those sk NICs.

            At the same time I haven't heard of anything like what you're seeing either.

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            1 Reply Last reply Reply Quote 0
            • DerelictD
              Derelict LAYER 8 Netgate
              last edited by

              192.168.1.0/24    link#11            U      sk1_vlan
              192.168.1.1        link#11            UHS        lo0
              192.168.1.3        link#11            UHS        lo0
              192.168.2.21      link#11            UHS        lo0
              192.168.2.21/32    link#11            U      sk1_vlan

              Why are both of those on link #11?

              netstat -rnWfinet

              Chattanooga, Tennessee, USA
              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
              Do Not Chat For Help! NO_WAN_EGRESS(TM)

              1 Reply Last reply Reply Quote 0
              • T
                tsmalmbe
                last edited by

                1.1 is the router itself
                1.3 is a virtual ip
                2.21 is a virtual ip

                192.168.1.3/32 LAN  IP Alias Virtual IP for wpad.malmberg.fi
                192.168.2.21/32 LAN  IP Alias Virtual IP for wiki.malmberg.fi

                I also realized that I have had it defined as 1.3/24 instead of 1.3/32 - this I have now changed.

                I have run your suggested netstat -command BEFORE and AFTER this change.

                BEFORE

                netstat -rnWfinet
                Routing tables

                Internet:
                Destination        Gateway            Flags      Use    Mtu      Netif Expire
                default            217.30.178.254    UGS      307699  1500        sk0
                10.10.1.0/24      link#4            U          401  1500        sk3
                10.10.1.1          link#4            UHS          0  16384        lo0
                10.99.0.0/16      10.100.100.2      UGS      66905  1500    ovpns5
                10.100.100.1      link#13            UHS          0  16384        lo0
                10.100.100.2      link#13            UH            6  1500    ovpns5
                10.100.101.1      link#14            UHS          0  16384        lo0
                10.100.101.2      link#14            UH            0  1500    ovpns8
                10.200.200.0/24    10.200.200.2      UGS          0  1500    ovpns1
                10.200.200.1      link#12            UHS          0  16384        lo0
                10.200.200.2      link#12            UH            0  1500    ovpns1
                10.200.210.0/24    10.100.100.2      UGS          0  1500    ovpns5
                83.145.193.133    217.30.178.254    UGHS    111003  1500        sk0
                127.0.0.1          link#8            UH      354961  16384        lo0
                192.168.1.0/24    link#11            U      2433840  1500  sk1_vlan5
                192.168.1.1        link#11            UHS          0  16384        lo0
                192.168.1.3        link#11            UHS          0  16384        lo0
                192.168.2.21      link#11            UHS        810  16384        lo0
                192.168.2.21/32    link#11            U            0  1500  sk1_vlan5
                192.168.10.0/24    link#9            U          487  1500  sk1_vlan6
                192.168.10.1      link#9            UHS          0  16384        lo0
                192.168.20.0/24    link#10            U        36126  1500  sk1_vlan7
                192.168.20.1      link#10            UHS          0  16384        lo0
                192.168.69.0/24    10.100.101.2      UGS          0  1500    ovpns8
                192.168.100.0/24  link#3            U            0  1500        sk2
                192.168.100.1      link#3            UHS          0  16384        lo0
                217.30.178.0/24    link#1            U            0  1500        sk0
                217.30.178.237    link#1            UHS          0  16384        lo0

                AFTER

                Routing tables

                Internet:
                Destination        Gateway            Flags      Use    Mtu      Netif Expire
                default            217.30.178.254    UGS      309567  1500        sk0
                10.10.1.0/24      link#4            U          401  1500        sk3
                10.10.1.1          link#4            UHS          0  16384        lo0
                10.99.0.0/16      10.100.100.2      UGS      67281  1500    ovpns5
                10.100.100.1      link#13            UHS          0  16384        lo0
                10.100.100.2      link#13            UH            6  1500    ovpns5
                10.100.101.1      link#14            UHS          0  16384        lo0
                10.100.101.2      link#14            UH            0  1500    ovpns8
                10.200.200.0/24    10.200.200.2      UGS          0  1500    ovpns1
                10.200.200.1      link#12            UHS          0  16384        lo0
                10.200.200.2      link#12            UH            0  1500    ovpns1
                10.200.210.0/24    10.100.100.2      UGS          0  1500    ovpns5
                83.145.193.133    217.30.178.254    UGHS    111477  1500        sk0
                127.0.0.1          link#8            UH      357517  16384        lo0
                192.168.1.0/24    link#11            U      2435972  1500  sk1_vlan5
                192.168.1.1        link#11            UHS          0  16384        lo0
                192.168.1.3        link#11            UHS          0  16384        lo0
                192.168.1.3/32    link#11            U            0  1500  sk1_vlan5
                192.168.2.21      link#11            UHS          0  16384        lo0
                192.168.2.21/32    link#11            U            0  1500  sk1_vlan5
                192.168.10.0/24    link#9            U          490  1500  sk1_vlan6
                192.168.10.1      link#9            UHS          0  16384        lo0
                192.168.20.0/24    link#10            U        36269  1500  sk1_vlan7
                192.168.20.1      link#10            UHS          0  16384        lo0
                192.168.69.0/24    10.100.101.2      UGS          0  1500    ovpns8
                192.168.100.0/24  link#3            U            0  1500        sk2
                192.168.100.1      link#3            UHS          0  16384        lo0
                217.30.178.0/24    link#1            U            0  1500        sk0
                217.30.178.237    link#1            UHS          0  16384        lo0

                Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                1 Reply Last reply Reply Quote 0
                • T
                  tsmalmbe
                  last edited by

                  I know this is not a perfect timeline, but two simultaneous windows though.

                  11:33:52.822204 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                  11:33:52.822442 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                  11:33:52.822452 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11405, length 40
                  11:33:52.822642 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11405, length 40
                  11:33:53.822241 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                  11:33:53.822469 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                  11:33:53.822482 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11407, length 40
                  11:33:53.822669 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11407, length 40
                  11:33:54.823260 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                  11:33:54.823490 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                  11:33:54.823501 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11409, length 40
                  11:33:54.823690 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11409, length 40
                  11:33:55.825579 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11411, length 40
                  11:33:55.825812 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11411, length 40
                  11:33:56.826596 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11414, length 40
                  11:33:56.826836 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11414, length 40
                  11:33:57.829819 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11417, length 40
                  11:33:57.830054 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11417, length 40

                  Reply from 192.168.1.1: Destination host unreachable.
                  Reply from 192.168.1.1: Destination host unreachable.
                  Reply from 192.168.1.1: Destination host unreachable.
                  Reply from 192.168.1.1: Destination host unreachable.
                  Reply from 192.168.1.1: Destination host unreachable.
                  Reply from 192.168.1.1: Destination host unreachable.
                  Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=114ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=2ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=6ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=1ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=2ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=2ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=1ms TTL=63
                  Reply from 192.168.20.7: bytes=32 time=3ms TTL=63

                  So WTF is going on with these ARP's.

                  Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                  1 Reply Last reply Reply Quote 0
                  • T
                    tsmalmbe
                    last edited by

                    11:35:52.476415 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                    11:35:52.476692 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                    11:35:52.476703 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11614, length 40
                    11:35:52.476892 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11614, length 40
                    11:35:53.475585 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                    11:35:53.475821 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                    11:35:53.475833 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11616, length 40
                    11:35:53.476019 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11616, length 40
                    11:35:53.890421 IP 192.168.20.7.138 > 192.168.1.255.138: UDP, length 227
                    11:35:54.477796 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11619, length 40
                    11:35:54.478040 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11619, length 40
                    11:35:55.477931 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11622, length 40
                    11:35:55.478166 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11622, length 40
                    11:35:56.478949 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11625, length 40
                    11:35:56.479188 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11625, length 40
                    11:35:57.479970 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11628, length 40
                    11:35:57.480212 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11628, length 40
                    11:35:58.480948 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11631, length 40
                    11:35:58.481133 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11631, length 40
                    11:35:59.482083 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                    11:35:59.482256 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                    11:35:59.482261 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11634, length 40
                    11:35:59.482455 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11634, length 40
                    11:36:00.483100 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11637, length 40
                    11:36:00.483379 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11637, length 40
                    11:36:01.483127 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11640, length 40
                    11:36:01.483305 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11640, length 40
                    11:36:02.485267 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11643, length 40
                    11:36:02.485527 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11643, length 40
                    11:36:03.485210 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11646, length 40
                    11:36:03.485452 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11646, length 40
                    11:36:04.485242 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11649, length 40
                    11:36:04.485476 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11649, length 40
                    11:36:04.485776 ARP, Request who-has 192.168.20.1 tell 192.168.20.7, length 42
                    11:36:04.485786 ARP, Reply 192.168.20.1 is-at 00:90:7f:40:45:54, length 28
                    11:36:04.597913 IP 192.168.20.7.139 > 192.168.1.166.57005: tcp 4
                    11:36:05.369166 IP 192.168.1.14.137 > 192.168.20.7.137: UDP, length 50
                    11:36:05.369769 IP 192.168.20.7.137 > 192.168.1.14.137: UDP, length 62
                    11:36:05.370391 IP 192.168.1.14.138 > 192.168.20.7.138: UDP, length 177
                    11:36:05.374596 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 0
                    11:36:05.374863 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 0
                    11:36:05.375069 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 0
                    11:36:05.388350 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 72
                    11:36:05.388543 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 0
                    11:36:05.389041 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 4
                    11:36:05.389148 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 0
                    11:36:05.389347 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 194
                    11:36:05.394133 IP 192.168.20.7.57303 > 192.168.1.1.53: UDP, length 37
                    11:36:05.428884 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 0
                    11:36:05.487307 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11652, length 40
                    11:36:05.487495 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11652, length 40
                    11:36:06.488447 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11655, length 40
                    11:36:06.488719 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11655, length 40
                    11:36:07.488499 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11658, length 40
                    11:36:07.488743 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11658, length 40
                    11:36:08.490590 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11661, length 40
                    11:36:08.490764 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11661, length 40
                    11:36:09.491648 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11664, length 40
                    11:36:09.491889 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11664, length 40
                    11:36:10.399253 IP 192.168.20.7.57303 > 192.168.1.1.53: UDP, length 37
                    11:36:10.722106 IP 192.168.1.1.53 > 192.168.20.7.57303: UDP, length 103
                    11:36:10.722473 IP 192.168.20.7.32781 > 192.168.1.1.53: UDP, length 49
                    11:36:10.743034 IP 192.168.1.1.53 > 192.168.20.7.32781: UDP, length 115
                    11:36:10.744939 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:10.744963 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:10.945952 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:10.945993 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:11.146854 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:11.146915 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                    11:36:11.147147 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                    11:36:11.147157 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:11.347851 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:11.347879 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:11.548874 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:11.548950 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:11.749860 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:11.749892 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:11.950968 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:11.951009 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:12.151867 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:12.151908 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:12.352873 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:12.352916 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:12.553878 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:12.553927 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:12.754878 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:12.754921 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:12.955897 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:12.955969 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:13.156900 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:13.156968 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:13.357906 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:13.357972 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:13.558892 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:13.558928 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:13.759896 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:13.759921 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:13.960906 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                    11:36:13.960936 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                    11:36:14.161910 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163

                    Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                    1 Reply Last reply Reply Quote 0
                    • T
                      tsmalmbe
                      last edited by

                      netstat -m
                      1855/1940/3795 mbufs in use (current/cache/total)
                      1438/338/1776/26368 mbuf clusters in use (current/cache/total/max)
                      1438/333 mbuf+clusters out of packet secondary zone in use (current/cache)
                      0/120/120/13184 4k (page size) jumbo clusters in use (current/cache/total/max)
                      0/0/0/3906 9k jumbo clusters in use (current/cache/total/max)
                      0/0/0/2197 16k jumbo clusters in use (current/cache/total/max)
                      3352K/1641K/4993K bytes allocated to network (current/cache/total)
                      0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
                      0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
                      0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
                      0/0/0 requests for jumbo clusters denied (4k/9k/16k)
                      0/10/6656 sfbufs in use (current/peak/max)
                      0 requests for sfbufs denied
                      0 requests for sfbufs delayed
                      0 requests for I/O initiated by sendfile

                      I also did service netif restart and then no traffic on any interface worked so I had to reboot again.

                      Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                      1 Reply Last reply Reply Quote 0
                      • DerelictD
                        Derelict LAYER 8 Netgate
                        last edited by

                        1.1 is the router itself
                        1.3 is a virtual ip
                        2.21 is a virtual ip

                        192.168.1.3/32    LAN    IP Alias    Virtual IP for wpad.malmberg.fi
                        192.168.2.21/32    LAN    IP Alias    Virtual IP for wiki.malmberg.fi

                        So you're playing games with multiple Layer 3 networks on VLAN 5 and you are having problems. Perhaps don't do silly things like that. Makes me wonder what other questionable design decisions you have made elsewhere. Seems they are coming home to roost. What kind of switch are you using? If you post more packet captures please indicate where they were taken and how.

                        Chattanooga, Tennessee, USA
                        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                        Do Not Chat For Help! NO_WAN_EGRESS(TM)

                        1 Reply Last reply Reply Quote 0
                        • T
                          tsmalmbe
                          last edited by

                          So, the 192.168.2-base ip is now gone. I will look into adding a network solely for my HAProxy addresses some other way, perhaps using a VLAN which is only available on pfSense or something along those lines. But I will leave that for now as it most obviously can be a part of the problem. I added 1.6/32 as a VIP instead to serve my wiki.

                          My packet dumps are from pfSense.

                          My switch is a Procurve 2848 (J4904A).

                          Let's see if the removal of the VIP makes a difference.

                          It would be foolish to say that this was the only questionable design I have made, although most of my decisions are preceded by somewhat thorough investigations.

                          Routing tables

                          Internet:
                          Destination        Gateway            Flags      Use    Mtu      Netif Expire
                          default            217.30.178.254    UGS      275029  1500        sk0
                          10.10.1.0/24      link#4            U            0  1500        sk3
                          10.10.1.1          link#4            UHS          0  16384        lo0
                          10.99.0.0/16      10.100.100.2      UGS      67877  1500    ovpns5
                          10.100.100.1      link#13            UHS          0  16384        lo0
                          10.100.100.2      link#13            UH            0  1500    ovpns5
                          10.100.101.1      link#14            UHS          0  16384        lo0
                          10.100.101.2      link#14            UH            0  1500    ovpns8
                          10.200.200.0/24    10.200.200.2      UGS          0  1500    ovpns1
                          10.200.200.1      link#12            UHS          0  16384        lo0
                          10.200.200.2      link#12            UH            0  1500    ovpns1
                          10.200.210.0/24    10.100.100.2      UGS          0  1500    ovpns5
                          83.145.193.133    217.30.178.254    UGHS      72257  1500        sk0
                          127.0.0.1          link#8            UH      231466  16384        lo0
                          192.168.1.0/24    link#11            U        640239  1500  sk1_vlan5
                          192.168.1.1        link#11            UHS          0  16384        lo0
                          192.168.1.3        link#11            UHS          0  16384        lo0
                          192.168.1.3/32    link#11            U            0  1500  sk1_vlan5
                          192.168.1.6        link#11            UHS          0  16384        lo0
                          192.168.1.6/32    link#11            U            0  1500  sk1_vlan5
                          192.168.10.0/24    link#9            U          319  1500  sk1_vlan6
                          192.168.10.1      link#9            UHS          0  16384        lo0
                          192.168.20.0/24    link#10            U        65604  1500  sk1_vlan7
                          192.168.20.1      link#10            UHS          0  16384        lo0
                          192.168.69.0/24    10.100.101.2      UGS          0  1500    ovpns8
                          192.168.100.0/24  link#3            U            0  1500        sk2
                          192.168.100.1      link#3            UHS          0  16384        lo0
                          217.30.178.0/24    link#1            U            0  1500        sk0
                          217.30.178.237    link#1            UHS          0  16384        lo0

                          Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                          1 Reply Last reply Reply Quote 0
                          • T
                            tsmalmbe
                            last edited by

                            Made no real difference.

                            00:44:13.624423 IP 192.168.1.191.60359 > 192.168.1.1.53: UDP, length 38
                            00:44:13.624597 IP 192.168.1.1.53 > 192.168.1.191.60359: UDP, length 54
                            00:44:13.627115 IP 192.168.1.191.54459 > 192.168.1.1.53: UDP, length 38
                            00:44:13.627161 IP 192.168.1.1.53 > 192.168.1.191.54459: UDP, length 54
                            00:44:13.920983 IP 192.168.1.191.41804 > 192.168.1.1.443: tcp 1
                            00:44:13.921020 IP 192.168.1.1.443 > 192.168.1.191.41804: tcp 0
                            00:44:14.113198 IP 192.168.1.191.41889 > 192.168.1.1.3128: tcp 0
                            00:44:14.113255 IP 192.168.1.1.3128 > 192.168.1.191.41889: tcp 0
                            00:44:14.114696 IP 192.168.1.191.41889 > 192.168.1.1.3128: tcp 0
                            00:44:14.132373 IP 192.168.1.191.41889 > 192.168.1.1.3128: tcp 0
                            00:44:14.132397 IP 192.168.1.1.3128 > 192.168.1.191.41889: tcp 0
                            00:44:14.132481 IP 192.168.1.1.3128 > 192.168.1.191.41889: tcp 0
                            00:44:14.133968 IP 192.168.1.191.41889 > 192.168.1.1.3128: tcp 0
                            00:44:14.372141 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:15.372264 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:15.647337 IP 192.168.1.191.41827 > 192.168.1.1.3128: tcp 46
                            00:44:15.647388 IP 192.168.1.1.3128 > 192.168.1.191.41827: tcp 0
                            00:44:15.647834 IP 192.168.1.191.41827 > 192.168.1.1.3128: tcp 31
                            00:44:15.647850 IP 192.168.1.1.3128 > 192.168.1.191.41827: tcp 0
                            00:44:15.648333 IP 192.168.1.191.41827 > 192.168.1.1.3128: tcp 0
                            00:44:15.648354 IP 192.168.1.1.3128 > 192.168.1.191.41827: tcp 0
                            00:44:15.648586 IP 192.168.1.1.3128 > 192.168.1.191.41827: tcp 0
                            00:44:15.649732 IP 192.168.1.191.41827 > 192.168.1.1.3128: tcp 0
                            00:44:16.373381 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:17.374319 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:18.375471 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:19.377456 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:20.212401 IP 192.168.1.191.40996 > 192.168.1.1.3128: tcp 1
                            00:44:20.212425 IP 192.168.1.1.3128 > 192.168.1.191.40996: tcp 0
                            00:44:20.378476 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:20.414702 IP 192.168.1.191.41703 > 192.168.1.1.3128: tcp 36
                            00:44:20.414729 IP 192.168.1.1.3128 > 192.168.1.191.41703: tcp 0
                            00:44:21.072818 ARP, Request who-has 192.168.1.169 tell 192.168.1.1, length 28
                            00:44:21.134440 ARP, Reply 192.168.1.169 is-at d8:0f:99:2d:58:a5, length 46
                            00:44:21.379496 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:22.380614 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:23.381640 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:23.606706 IP 192.168.1.191.41861 > 192.168.1.1.443: tcp 1
                            00:44:23.606781 IP 192.168.1.1.443 > 192.168.1.191.41861: tcp 0
                            00:44:23.929637 IP 192.168.1.191.41804 > 192.168.1.1.443: tcp 1
                            00:44:23.929717 IP 192.168.1.1.443 > 192.168.1.191.41804: tcp 0
                            00:44:24.382789 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:25.383800 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:26.182792 ARP, Request who-has 192.168.1.1 tell 192.168.1.169, length 46
                            00:44:26.182806 ARP, Reply 192.168.1.1 is-at 00:90:7f:40:45:54, length 28
                            00:44:26.385013 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:27.385931 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:28.388054 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:28.600625 IP 192.168.1.191.39585 > 192.168.1.1.3128: tcp 805
                            00:44:28.600657 IP 192.168.1.1.3128 > 192.168.1.191.39585: tcp 0
                            00:44:29.374504 IP 192.168.1.1.3128 > 192.168.1.191.39585: tcp 1460
                            00:44:29.374521 IP 192.168.1.1.3128 > 192.168.1.191.39585: tcp 279
                            00:44:29.377592 IP 192.168.1.191.39585 > 192.168.1.1.3128: tcp 0
                            00:44:29.389036 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:30.214045 IP 192.168.1.191.40996 > 192.168.1.1.3128: tcp 1
                            00:44:30.214070 IP 192.168.1.1.3128 > 192.168.1.191.40996: tcp 0
                            00:44:30.390005 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:31.391255 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:32.392346 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:33.394514 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:33.607239 IP 192.168.1.191.41861 > 192.168.1.1.443: tcp 1
                            00:44:33.607283 IP 192.168.1.1.443 > 192.168.1.191.41861: tcp 0
                            00:44:33.933459 IP 192.168.1.191.41804 > 192.168.1.1.443: tcp 1
                            00:44:33.933483 IP 192.168.1.1.443 > 192.168.1.191.41804: tcp 0
                            00:44:34.395412 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:35.396425 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:36.397542 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:37.398571 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:38.021286 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 92
                            00:44:38.222530 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 0
                            00:44:38.399706 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:39.400709 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:40.220600 IP 192.168.1.191.40996 > 192.168.1.1.3128: tcp 1
                            00:44:40.220682 IP 192.168.1.1.3128 > 192.168.1.191.40996: tcp 0
                            00:44:40.308870 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 388
                            00:44:40.308948 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 0
                            00:44:40.309052 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 156
                            00:44:40.309069 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 0
                            00:44:40.401746 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:40.533902 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 107
                            00:44:40.593171 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 401
                            00:44:40.595328 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 0
                            00:44:41.402919 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:42.404878 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:43.405796 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:43.604789 IP 192.168.1.191.41861 > 192.168.1.1.443: tcp 1
                            00:44:43.604823 IP 192.168.1.1.443 > 192.168.1.191.41861: tcp 0
                            00:44:43.938897 IP 192.168.1.191.41804 > 192.168.1.1.443: tcp 1
                            00:44:43.938918 IP 192.168.1.1.443 > 192.168.1.191.41804: tcp 0
                            00:44:44.406929 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:45.408085 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:46.410172 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:47.200184 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 42
                            00:44:47.200211 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 0
                            00:44:47.411191 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:48.209196 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 471
                            00:44:48.209226 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 0
                            00:44:48.405610 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 97
                            00:44:48.412215 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                            00:44:48.412443 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 197

                            From pfSense,  capturing 192.168.1.1 this time.

                            The pinging is done from 1.191.

                            Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                            1 Reply Last reply Reply Quote 0
                            • DerelictD
                              Derelict LAYER 8 Netgate
                              last edited by

                              I would try a different NIC. Or capture on a monitor port on the switch and see if the pings are really going out on the wire.

                              Almost nobody uses those sk NICs.

                              Chattanooga, Tennessee, USA
                              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                              Do Not Chat For Help! NO_WAN_EGRESS(TM)

                              1 Reply Last reply Reply Quote 0
                              • T
                                tsmalmbe
                                last edited by

                                I'll go ahead and move my ports around a bit later tonight just to cross out the possibility of hardware failure.

                                I've liked the idea of using these Watchguards as they are "proper" pedigree firewalls. I have been looking at migratiing to an XTM-series, but I just had huge issues getting the latest nano-release even starting on that one.

                                Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                                1 Reply Last reply Reply Quote 0
                                • T
                                  tsmalmbe
                                  last edited by

                                  Both the primary WG and my secondary WG got totally screwed up and wont even boot in an orderly fashion.

                                  Luckily I took a backup before venturing into testin.

                                  As a workaround, I installed pfSense on proxmox, hooked up my VLANs and now this technically works. Technically, not optimally - because now I'm firewalling in the host-environment where my crownjewels are, instead of firewalling before  even touching this hardware. But for now, I'm in business again.

                                  Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post
                                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.