Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Intermittent "no route to host" on my LAN-port

    Scheduled Pinned Locked Moved General pfSense Questions
    35 Posts 3 Posters 12.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      tsmalmbe
      last edited by

      So now everything is pretty fucked until I once again reboot the firewall.

      Security Consultant at Mint Security Ltd - www.mintsecurity.fi

      1 Reply Last reply Reply Quote 0
      • T
        tsmalmbe
        last edited by

        When the problem (which is now different but the same) occurs, the firewall can now ping everything so no problem there. But now I lose

        • VLAN (LAN) -> ovpns5,
        • VLAN (LAN)->VLAN (SERVERS1) and
        • VLAN (LAN) -> VLAN (SERVERS2).

        VLAN (LAN) to the internet is all fine and working.

        There has to be some way to debug this.

        Security Consultant at Mint Security Ltd - www.mintsecurity.fi

        1 Reply Last reply Reply Quote 0
        • T
          tsmalmbe
          last edited by

          Dumping the routing when the issue occurs:

          [2.3.4-RELEASE][root@firewall.ccccccccc.fi]/root: netstat -rn
          Routing tables

          Internet:
          Destination        Gateway            Flags      Netif Expire
          default            217.30.178.254    UGS        sk0
          10.10.1.0/24      link#4            U          sk3
          10.10.1.1          link#4            UHS        lo0
          10.99.0.0/16      10.100.100.2      UGS      ovpns5
          10.100.100.1      link#13            UHS        lo0
          10.100.100.2      link#13            UH      ovpns5
          10.100.101.1      link#14            UHS        lo0
          10.100.101.2      link#14            UH      ovpns8
          10.200.200.0/24    10.200.200.2      UGS      ovpns1
          10.200.200.1      link#12            UHS        lo0
          10.200.200.2      link#12            UH      ovpns1
          10.200.210.0/24    10.100.100.2      UGS      ovpns5
          83.145.193.133    217.30.178.254    UGHS        sk0
          127.0.0.1          link#8            UH          lo0
          192.168.1.0/24    link#11            U      sk1_vlan
          192.168.1.1        link#11            UHS        lo0
          192.168.1.3        link#11            UHS        lo0
          192.168.2.21      link#11            UHS        lo0
          192.168.2.21/32    link#11            U      sk1_vlan
          192.168.10.0/24    link#9            U      sk1_vlan
          192.168.10.1      link#9            UHS        lo0
          192.168.20.0/24    link#10            U      sk1_vlan
          192.168.20.1      link#10            UHS        lo0
          192.168.69.0/24    10.100.101.2      UGS      ovpns8
          192.168.100.0/24  link#3            U          sk2
          192.168.100.1      link#3            UHS        lo0
          217.30.178.0/24    link#1            U          sk0
          217.30.178.237    link#1            UHS        lo0

          Security Consultant at Mint Security Ltd - www.mintsecurity.fi

          1 Reply Last reply Reply Quote 0
          • T
            tsmalmbe
            last edited by

            After reboot

            Routing tables

            Internet:
            Destination        Gateway            Flags      Netif Expire
            default            217.30.178.254    UGS        sk0
            10.10.1.0/24      link#4            U          sk3
            10.10.1.1          link#4            UHS        lo0
            10.99.0.0/16      10.100.100.2      UGS      ovpns5
            10.100.100.1      link#13            UHS        lo0
            10.100.100.2      link#13            UH      ovpns5
            10.100.101.1      link#14            UHS        lo0
            10.100.101.2      link#14            UH      ovpns8
            10.200.200.0/24    10.200.200.2      UGS      ovpns1
            10.200.200.1      link#12            UHS        lo0
            10.200.200.2      link#12            UH      ovpns1
            10.200.210.0/24    10.100.100.2      UGS      ovpns5
            83.145.193.133    217.30.178.254    UGHS        sk0
            127.0.0.1          link#8            UH          lo0
            192.168.1.0/24    link#11            U      sk1_vlan
            192.168.1.1        link#11            UHS        lo0
            192.168.1.3        link#11            UHS        lo0
            192.168.2.21      link#11            UHS        lo0
            192.168.2.21/32    link#11            U      sk1_vlan
            192.168.10.0/24    link#9            U      sk1_vlan
            192.168.10.1      link#9            UHS        lo0
            192.168.20.0/24    link#10            U      sk1_vlan
            192.168.20.1      link#10            UHS        lo0
            192.168.69.0/24    10.100.101.2      UGS      ovpns8
            192.168.100.0/24  link#3            U          sk2
            192.168.100.1      link#3            UHS        lo0
            217.30.178.0/24    link#1            U          sk0
            217.30.178.237    link#1            UHS        lo0

            Security Consultant at Mint Security Ltd - www.mintsecurity.fi

            1 Reply Last reply Reply Quote 0
            • DerelictD
              Derelict LAYER 8 Netgate
              last edited by

              As far as I know, almost nobody is using those sk NICs.

              At the same time I haven't heard of anything like what you're seeing either.

              Chattanooga, Tennessee, USA
              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
              Do Not Chat For Help! NO_WAN_EGRESS(TM)

              1 Reply Last reply Reply Quote 0
              • DerelictD
                Derelict LAYER 8 Netgate
                last edited by

                192.168.1.0/24    link#11            U      sk1_vlan
                192.168.1.1        link#11            UHS        lo0
                192.168.1.3        link#11            UHS        lo0
                192.168.2.21      link#11            UHS        lo0
                192.168.2.21/32    link#11            U      sk1_vlan

                Why are both of those on link #11?

                netstat -rnWfinet

                Chattanooga, Tennessee, USA
                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                1 Reply Last reply Reply Quote 0
                • T
                  tsmalmbe
                  last edited by

                  1.1 is the router itself
                  1.3 is a virtual ip
                  2.21 is a virtual ip

                  192.168.1.3/32 LAN  IP Alias Virtual IP for wpad.malmberg.fi
                  192.168.2.21/32 LAN  IP Alias Virtual IP for wiki.malmberg.fi

                  I also realized that I have had it defined as 1.3/24 instead of 1.3/32 - this I have now changed.

                  I have run your suggested netstat -command BEFORE and AFTER this change.

                  BEFORE

                  netstat -rnWfinet
                  Routing tables

                  Internet:
                  Destination        Gateway            Flags      Use    Mtu      Netif Expire
                  default            217.30.178.254    UGS      307699  1500        sk0
                  10.10.1.0/24      link#4            U          401  1500        sk3
                  10.10.1.1          link#4            UHS          0  16384        lo0
                  10.99.0.0/16      10.100.100.2      UGS      66905  1500    ovpns5
                  10.100.100.1      link#13            UHS          0  16384        lo0
                  10.100.100.2      link#13            UH            6  1500    ovpns5
                  10.100.101.1      link#14            UHS          0  16384        lo0
                  10.100.101.2      link#14            UH            0  1500    ovpns8
                  10.200.200.0/24    10.200.200.2      UGS          0  1500    ovpns1
                  10.200.200.1      link#12            UHS          0  16384        lo0
                  10.200.200.2      link#12            UH            0  1500    ovpns1
                  10.200.210.0/24    10.100.100.2      UGS          0  1500    ovpns5
                  83.145.193.133    217.30.178.254    UGHS    111003  1500        sk0
                  127.0.0.1          link#8            UH      354961  16384        lo0
                  192.168.1.0/24    link#11            U      2433840  1500  sk1_vlan5
                  192.168.1.1        link#11            UHS          0  16384        lo0
                  192.168.1.3        link#11            UHS          0  16384        lo0
                  192.168.2.21      link#11            UHS        810  16384        lo0
                  192.168.2.21/32    link#11            U            0  1500  sk1_vlan5
                  192.168.10.0/24    link#9            U          487  1500  sk1_vlan6
                  192.168.10.1      link#9            UHS          0  16384        lo0
                  192.168.20.0/24    link#10            U        36126  1500  sk1_vlan7
                  192.168.20.1      link#10            UHS          0  16384        lo0
                  192.168.69.0/24    10.100.101.2      UGS          0  1500    ovpns8
                  192.168.100.0/24  link#3            U            0  1500        sk2
                  192.168.100.1      link#3            UHS          0  16384        lo0
                  217.30.178.0/24    link#1            U            0  1500        sk0
                  217.30.178.237    link#1            UHS          0  16384        lo0

                  AFTER

                  Routing tables

                  Internet:
                  Destination        Gateway            Flags      Use    Mtu      Netif Expire
                  default            217.30.178.254    UGS      309567  1500        sk0
                  10.10.1.0/24      link#4            U          401  1500        sk3
                  10.10.1.1          link#4            UHS          0  16384        lo0
                  10.99.0.0/16      10.100.100.2      UGS      67281  1500    ovpns5
                  10.100.100.1      link#13            UHS          0  16384        lo0
                  10.100.100.2      link#13            UH            6  1500    ovpns5
                  10.100.101.1      link#14            UHS          0  16384        lo0
                  10.100.101.2      link#14            UH            0  1500    ovpns8
                  10.200.200.0/24    10.200.200.2      UGS          0  1500    ovpns1
                  10.200.200.1      link#12            UHS          0  16384        lo0
                  10.200.200.2      link#12            UH            0  1500    ovpns1
                  10.200.210.0/24    10.100.100.2      UGS          0  1500    ovpns5
                  83.145.193.133    217.30.178.254    UGHS    111477  1500        sk0
                  127.0.0.1          link#8            UH      357517  16384        lo0
                  192.168.1.0/24    link#11            U      2435972  1500  sk1_vlan5
                  192.168.1.1        link#11            UHS          0  16384        lo0
                  192.168.1.3        link#11            UHS          0  16384        lo0
                  192.168.1.3/32    link#11            U            0  1500  sk1_vlan5
                  192.168.2.21      link#11            UHS          0  16384        lo0
                  192.168.2.21/32    link#11            U            0  1500  sk1_vlan5
                  192.168.10.0/24    link#9            U          490  1500  sk1_vlan6
                  192.168.10.1      link#9            UHS          0  16384        lo0
                  192.168.20.0/24    link#10            U        36269  1500  sk1_vlan7
                  192.168.20.1      link#10            UHS          0  16384        lo0
                  192.168.69.0/24    10.100.101.2      UGS          0  1500    ovpns8
                  192.168.100.0/24  link#3            U            0  1500        sk2
                  192.168.100.1      link#3            UHS          0  16384        lo0
                  217.30.178.0/24    link#1            U            0  1500        sk0
                  217.30.178.237    link#1            UHS          0  16384        lo0

                  Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                  1 Reply Last reply Reply Quote 0
                  • T
                    tsmalmbe
                    last edited by

                    I know this is not a perfect timeline, but two simultaneous windows though.

                    11:33:52.822204 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                    11:33:52.822442 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                    11:33:52.822452 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11405, length 40
                    11:33:52.822642 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11405, length 40
                    11:33:53.822241 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                    11:33:53.822469 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                    11:33:53.822482 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11407, length 40
                    11:33:53.822669 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11407, length 40
                    11:33:54.823260 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                    11:33:54.823490 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                    11:33:54.823501 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11409, length 40
                    11:33:54.823690 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11409, length 40
                    11:33:55.825579 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11411, length 40
                    11:33:55.825812 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11411, length 40
                    11:33:56.826596 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11414, length 40
                    11:33:56.826836 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11414, length 40
                    11:33:57.829819 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11417, length 40
                    11:33:57.830054 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11417, length 40

                    Reply from 192.168.1.1: Destination host unreachable.
                    Reply from 192.168.1.1: Destination host unreachable.
                    Reply from 192.168.1.1: Destination host unreachable.
                    Reply from 192.168.1.1: Destination host unreachable.
                    Reply from 192.168.1.1: Destination host unreachable.
                    Reply from 192.168.1.1: Destination host unreachable.
                    Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=114ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=2ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=6ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=1ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=2ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=2ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=1ms TTL=63
                    Reply from 192.168.20.7: bytes=32 time=3ms TTL=63

                    So WTF is going on with these ARP's.

                    Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                    1 Reply Last reply Reply Quote 0
                    • T
                      tsmalmbe
                      last edited by

                      11:35:52.476415 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                      11:35:52.476692 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                      11:35:52.476703 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11614, length 40
                      11:35:52.476892 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11614, length 40
                      11:35:53.475585 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                      11:35:53.475821 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                      11:35:53.475833 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11616, length 40
                      11:35:53.476019 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11616, length 40
                      11:35:53.890421 IP 192.168.20.7.138 > 192.168.1.255.138: UDP, length 227
                      11:35:54.477796 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11619, length 40
                      11:35:54.478040 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11619, length 40
                      11:35:55.477931 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11622, length 40
                      11:35:55.478166 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11622, length 40
                      11:35:56.478949 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11625, length 40
                      11:35:56.479188 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11625, length 40
                      11:35:57.479970 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11628, length 40
                      11:35:57.480212 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11628, length 40
                      11:35:58.480948 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11631, length 40
                      11:35:58.481133 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11631, length 40
                      11:35:59.482083 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                      11:35:59.482256 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                      11:35:59.482261 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11634, length 40
                      11:35:59.482455 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11634, length 40
                      11:36:00.483100 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11637, length 40
                      11:36:00.483379 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11637, length 40
                      11:36:01.483127 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11640, length 40
                      11:36:01.483305 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11640, length 40
                      11:36:02.485267 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11643, length 40
                      11:36:02.485527 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11643, length 40
                      11:36:03.485210 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11646, length 40
                      11:36:03.485452 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11646, length 40
                      11:36:04.485242 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11649, length 40
                      11:36:04.485476 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11649, length 40
                      11:36:04.485776 ARP, Request who-has 192.168.20.1 tell 192.168.20.7, length 42
                      11:36:04.485786 ARP, Reply 192.168.20.1 is-at 00:90:7f:40:45:54, length 28
                      11:36:04.597913 IP 192.168.20.7.139 > 192.168.1.166.57005: tcp 4
                      11:36:05.369166 IP 192.168.1.14.137 > 192.168.20.7.137: UDP, length 50
                      11:36:05.369769 IP 192.168.20.7.137 > 192.168.1.14.137: UDP, length 62
                      11:36:05.370391 IP 192.168.1.14.138 > 192.168.20.7.138: UDP, length 177
                      11:36:05.374596 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 0
                      11:36:05.374863 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 0
                      11:36:05.375069 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 0
                      11:36:05.388350 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 72
                      11:36:05.388543 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 0
                      11:36:05.389041 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 4
                      11:36:05.389148 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 0
                      11:36:05.389347 IP 192.168.1.14.3514 > 192.168.20.7.139: tcp 194
                      11:36:05.394133 IP 192.168.20.7.57303 > 192.168.1.1.53: UDP, length 37
                      11:36:05.428884 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 0
                      11:36:05.487307 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11652, length 40
                      11:36:05.487495 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11652, length 40
                      11:36:06.488447 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11655, length 40
                      11:36:06.488719 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11655, length 40
                      11:36:07.488499 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11658, length 40
                      11:36:07.488743 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11658, length 40
                      11:36:08.490590 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11661, length 40
                      11:36:08.490764 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11661, length 40
                      11:36:09.491648 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11664, length 40
                      11:36:09.491889 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11664, length 40
                      11:36:10.399253 IP 192.168.20.7.57303 > 192.168.1.1.53: UDP, length 37
                      11:36:10.722106 IP 192.168.1.1.53 > 192.168.20.7.57303: UDP, length 103
                      11:36:10.722473 IP 192.168.20.7.32781 > 192.168.1.1.53: UDP, length 49
                      11:36:10.743034 IP 192.168.1.1.53 > 192.168.20.7.32781: UDP, length 115
                      11:36:10.744939 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:10.744963 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:10.945952 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:10.945993 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:11.146854 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:11.146915 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
                      11:36:11.147147 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
                      11:36:11.147157 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:11.347851 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:11.347879 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:11.548874 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:11.548950 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:11.749860 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:11.749892 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:11.950968 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:11.951009 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:12.151867 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:12.151908 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:12.352873 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:12.352916 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:12.553878 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:12.553927 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:12.754878 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:12.754921 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:12.955897 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:12.955969 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:13.156900 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:13.156968 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:13.357906 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:13.357972 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:13.558892 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:13.558928 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:13.759896 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:13.759921 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:13.960906 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163
                      11:36:13.960936 IP 192.168.20.1 > 192.168.20.7: ICMP host 192.168.1.14 unreachable, length 60
                      11:36:14.161910 IP 192.168.20.7.139 > 192.168.1.14.3514: tcp 163

                      Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                      1 Reply Last reply Reply Quote 0
                      • T
                        tsmalmbe
                        last edited by

                        netstat -m
                        1855/1940/3795 mbufs in use (current/cache/total)
                        1438/338/1776/26368 mbuf clusters in use (current/cache/total/max)
                        1438/333 mbuf+clusters out of packet secondary zone in use (current/cache)
                        0/120/120/13184 4k (page size) jumbo clusters in use (current/cache/total/max)
                        0/0/0/3906 9k jumbo clusters in use (current/cache/total/max)
                        0/0/0/2197 16k jumbo clusters in use (current/cache/total/max)
                        3352K/1641K/4993K bytes allocated to network (current/cache/total)
                        0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
                        0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
                        0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
                        0/0/0 requests for jumbo clusters denied (4k/9k/16k)
                        0/10/6656 sfbufs in use (current/peak/max)
                        0 requests for sfbufs denied
                        0 requests for sfbufs delayed
                        0 requests for I/O initiated by sendfile

                        I also did service netif restart and then no traffic on any interface worked so I had to reboot again.

                        Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                        1 Reply Last reply Reply Quote 0
                        • DerelictD
                          Derelict LAYER 8 Netgate
                          last edited by

                          1.1 is the router itself
                          1.3 is a virtual ip
                          2.21 is a virtual ip

                          192.168.1.3/32    LAN    IP Alias    Virtual IP for wpad.malmberg.fi
                          192.168.2.21/32    LAN    IP Alias    Virtual IP for wiki.malmberg.fi

                          So you're playing games with multiple Layer 3 networks on VLAN 5 and you are having problems. Perhaps don't do silly things like that. Makes me wonder what other questionable design decisions you have made elsewhere. Seems they are coming home to roost. What kind of switch are you using? If you post more packet captures please indicate where they were taken and how.

                          Chattanooga, Tennessee, USA
                          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                          1 Reply Last reply Reply Quote 0
                          • T
                            tsmalmbe
                            last edited by

                            So, the 192.168.2-base ip is now gone. I will look into adding a network solely for my HAProxy addresses some other way, perhaps using a VLAN which is only available on pfSense or something along those lines. But I will leave that for now as it most obviously can be a part of the problem. I added 1.6/32 as a VIP instead to serve my wiki.

                            My packet dumps are from pfSense.

                            My switch is a Procurve 2848 (J4904A).

                            Let's see if the removal of the VIP makes a difference.

                            It would be foolish to say that this was the only questionable design I have made, although most of my decisions are preceded by somewhat thorough investigations.

                            Routing tables

                            Internet:
                            Destination        Gateway            Flags      Use    Mtu      Netif Expire
                            default            217.30.178.254    UGS      275029  1500        sk0
                            10.10.1.0/24      link#4            U            0  1500        sk3
                            10.10.1.1          link#4            UHS          0  16384        lo0
                            10.99.0.0/16      10.100.100.2      UGS      67877  1500    ovpns5
                            10.100.100.1      link#13            UHS          0  16384        lo0
                            10.100.100.2      link#13            UH            0  1500    ovpns5
                            10.100.101.1      link#14            UHS          0  16384        lo0
                            10.100.101.2      link#14            UH            0  1500    ovpns8
                            10.200.200.0/24    10.200.200.2      UGS          0  1500    ovpns1
                            10.200.200.1      link#12            UHS          0  16384        lo0
                            10.200.200.2      link#12            UH            0  1500    ovpns1
                            10.200.210.0/24    10.100.100.2      UGS          0  1500    ovpns5
                            83.145.193.133    217.30.178.254    UGHS      72257  1500        sk0
                            127.0.0.1          link#8            UH      231466  16384        lo0
                            192.168.1.0/24    link#11            U        640239  1500  sk1_vlan5
                            192.168.1.1        link#11            UHS          0  16384        lo0
                            192.168.1.3        link#11            UHS          0  16384        lo0
                            192.168.1.3/32    link#11            U            0  1500  sk1_vlan5
                            192.168.1.6        link#11            UHS          0  16384        lo0
                            192.168.1.6/32    link#11            U            0  1500  sk1_vlan5
                            192.168.10.0/24    link#9            U          319  1500  sk1_vlan6
                            192.168.10.1      link#9            UHS          0  16384        lo0
                            192.168.20.0/24    link#10            U        65604  1500  sk1_vlan7
                            192.168.20.1      link#10            UHS          0  16384        lo0
                            192.168.69.0/24    10.100.101.2      UGS          0  1500    ovpns8
                            192.168.100.0/24  link#3            U            0  1500        sk2
                            192.168.100.1      link#3            UHS          0  16384        lo0
                            217.30.178.0/24    link#1            U            0  1500        sk0
                            217.30.178.237    link#1            UHS          0  16384        lo0

                            Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                            1 Reply Last reply Reply Quote 0
                            • T
                              tsmalmbe
                              last edited by

                              Made no real difference.

                              00:44:13.624423 IP 192.168.1.191.60359 > 192.168.1.1.53: UDP, length 38
                              00:44:13.624597 IP 192.168.1.1.53 > 192.168.1.191.60359: UDP, length 54
                              00:44:13.627115 IP 192.168.1.191.54459 > 192.168.1.1.53: UDP, length 38
                              00:44:13.627161 IP 192.168.1.1.53 > 192.168.1.191.54459: UDP, length 54
                              00:44:13.920983 IP 192.168.1.191.41804 > 192.168.1.1.443: tcp 1
                              00:44:13.921020 IP 192.168.1.1.443 > 192.168.1.191.41804: tcp 0
                              00:44:14.113198 IP 192.168.1.191.41889 > 192.168.1.1.3128: tcp 0
                              00:44:14.113255 IP 192.168.1.1.3128 > 192.168.1.191.41889: tcp 0
                              00:44:14.114696 IP 192.168.1.191.41889 > 192.168.1.1.3128: tcp 0
                              00:44:14.132373 IP 192.168.1.191.41889 > 192.168.1.1.3128: tcp 0
                              00:44:14.132397 IP 192.168.1.1.3128 > 192.168.1.191.41889: tcp 0
                              00:44:14.132481 IP 192.168.1.1.3128 > 192.168.1.191.41889: tcp 0
                              00:44:14.133968 IP 192.168.1.191.41889 > 192.168.1.1.3128: tcp 0
                              00:44:14.372141 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:15.372264 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:15.647337 IP 192.168.1.191.41827 > 192.168.1.1.3128: tcp 46
                              00:44:15.647388 IP 192.168.1.1.3128 > 192.168.1.191.41827: tcp 0
                              00:44:15.647834 IP 192.168.1.191.41827 > 192.168.1.1.3128: tcp 31
                              00:44:15.647850 IP 192.168.1.1.3128 > 192.168.1.191.41827: tcp 0
                              00:44:15.648333 IP 192.168.1.191.41827 > 192.168.1.1.3128: tcp 0
                              00:44:15.648354 IP 192.168.1.1.3128 > 192.168.1.191.41827: tcp 0
                              00:44:15.648586 IP 192.168.1.1.3128 > 192.168.1.191.41827: tcp 0
                              00:44:15.649732 IP 192.168.1.191.41827 > 192.168.1.1.3128: tcp 0
                              00:44:16.373381 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:17.374319 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:18.375471 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:19.377456 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:20.212401 IP 192.168.1.191.40996 > 192.168.1.1.3128: tcp 1
                              00:44:20.212425 IP 192.168.1.1.3128 > 192.168.1.191.40996: tcp 0
                              00:44:20.378476 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:20.414702 IP 192.168.1.191.41703 > 192.168.1.1.3128: tcp 36
                              00:44:20.414729 IP 192.168.1.1.3128 > 192.168.1.191.41703: tcp 0
                              00:44:21.072818 ARP, Request who-has 192.168.1.169 tell 192.168.1.1, length 28
                              00:44:21.134440 ARP, Reply 192.168.1.169 is-at d8:0f:99:2d:58:a5, length 46
                              00:44:21.379496 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:22.380614 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:23.381640 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:23.606706 IP 192.168.1.191.41861 > 192.168.1.1.443: tcp 1
                              00:44:23.606781 IP 192.168.1.1.443 > 192.168.1.191.41861: tcp 0
                              00:44:23.929637 IP 192.168.1.191.41804 > 192.168.1.1.443: tcp 1
                              00:44:23.929717 IP 192.168.1.1.443 > 192.168.1.191.41804: tcp 0
                              00:44:24.382789 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:25.383800 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:26.182792 ARP, Request who-has 192.168.1.1 tell 192.168.1.169, length 46
                              00:44:26.182806 ARP, Reply 192.168.1.1 is-at 00:90:7f:40:45:54, length 28
                              00:44:26.385013 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:27.385931 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:28.388054 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:28.600625 IP 192.168.1.191.39585 > 192.168.1.1.3128: tcp 805
                              00:44:28.600657 IP 192.168.1.1.3128 > 192.168.1.191.39585: tcp 0
                              00:44:29.374504 IP 192.168.1.1.3128 > 192.168.1.191.39585: tcp 1460
                              00:44:29.374521 IP 192.168.1.1.3128 > 192.168.1.191.39585: tcp 279
                              00:44:29.377592 IP 192.168.1.191.39585 > 192.168.1.1.3128: tcp 0
                              00:44:29.389036 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:30.214045 IP 192.168.1.191.40996 > 192.168.1.1.3128: tcp 1
                              00:44:30.214070 IP 192.168.1.1.3128 > 192.168.1.191.40996: tcp 0
                              00:44:30.390005 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:31.391255 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:32.392346 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:33.394514 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:33.607239 IP 192.168.1.191.41861 > 192.168.1.1.443: tcp 1
                              00:44:33.607283 IP 192.168.1.1.443 > 192.168.1.191.41861: tcp 0
                              00:44:33.933459 IP 192.168.1.191.41804 > 192.168.1.1.443: tcp 1
                              00:44:33.933483 IP 192.168.1.1.443 > 192.168.1.191.41804: tcp 0
                              00:44:34.395412 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:35.396425 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:36.397542 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:37.398571 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:38.021286 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 92
                              00:44:38.222530 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 0
                              00:44:38.399706 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:39.400709 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:40.220600 IP 192.168.1.191.40996 > 192.168.1.1.3128: tcp 1
                              00:44:40.220682 IP 192.168.1.1.3128 > 192.168.1.191.40996: tcp 0
                              00:44:40.308870 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 388
                              00:44:40.308948 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 0
                              00:44:40.309052 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 156
                              00:44:40.309069 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 0
                              00:44:40.401746 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:40.533902 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 107
                              00:44:40.593171 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 401
                              00:44:40.595328 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 0
                              00:44:41.402919 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:42.404878 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:43.405796 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:43.604789 IP 192.168.1.191.41861 > 192.168.1.1.443: tcp 1
                              00:44:43.604823 IP 192.168.1.1.443 > 192.168.1.191.41861: tcp 0
                              00:44:43.938897 IP 192.168.1.191.41804 > 192.168.1.1.443: tcp 1
                              00:44:43.938918 IP 192.168.1.1.443 > 192.168.1.191.41804: tcp 0
                              00:44:44.406929 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:45.408085 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:46.410172 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:47.200184 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 42
                              00:44:47.200211 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 0
                              00:44:47.411191 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:48.209196 IP 192.168.1.191.39609 > 192.168.1.1.3128: tcp 471
                              00:44:48.209226 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 0
                              00:44:48.405610 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 97
                              00:44:48.412215 IP 192.168.1.1 > 192.168.1.191: ICMP host 192.168.20.7 unreachable, length 36
                              00:44:48.412443 IP 192.168.1.1.3128 > 192.168.1.191.39609: tcp 197

                              From pfSense,  capturing 192.168.1.1 this time.

                              The pinging is done from 1.191.

                              Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                              1 Reply Last reply Reply Quote 0
                              • DerelictD
                                Derelict LAYER 8 Netgate
                                last edited by

                                I would try a different NIC. Or capture on a monitor port on the switch and see if the pings are really going out on the wire.

                                Almost nobody uses those sk NICs.

                                Chattanooga, Tennessee, USA
                                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                1 Reply Last reply Reply Quote 0
                                • T
                                  tsmalmbe
                                  last edited by

                                  I'll go ahead and move my ports around a bit later tonight just to cross out the possibility of hardware failure.

                                  I've liked the idea of using these Watchguards as they are "proper" pedigree firewalls. I have been looking at migratiing to an XTM-series, but I just had huge issues getting the latest nano-release even starting on that one.

                                  Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                                  1 Reply Last reply Reply Quote 0
                                  • T
                                    tsmalmbe
                                    last edited by

                                    Both the primary WG and my secondary WG got totally screwed up and wont even boot in an orderly fashion.

                                    Luckily I took a backup before venturing into testin.

                                    As a workaround, I installed pfSense on proxmox, hooked up my VLANs and now this technically works. Technically, not optimally - because now I'm firewalling in the host-environment where my crownjewels are, instead of firewalling before  even touching this hardware. But for now, I'm in business again.

                                    Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.