Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    GW configuration - some questions

    HA/CARP/VIPs
    3
    32
    5.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      SteveITS Galactic Empire @empbilly
      last edited by

      @empbilly Can you ping the VLAN IP if you allow ICMP?

      Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
      When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
      Upvote 👍 helpful posts!

      1 Reply Last reply Reply Quote 0
      • V
        viragomann @empbilly
        last edited by

        @empbilly said in GW configuration - some questions:

        because if I set CARP VIP as GW in each vlan, the users have no internet access.

        Can you rule out a DNS issue? Did you try to ping e.g. 8.8.8.8?

        empbillyE 1 Reply Last reply Reply Quote 0
        • empbillyE
          empbilly @viragomann
          last edited by

          @viragomann and @SteveITS

          I have allowed all on a vlan and am testing.

          In GW I put the VIP CARP 10.10.56.1
          In DNS I already put VIP, google, our internal DNS, etc.

          When I ping google, I get the message "Destination host unreachable".
          When I ping the GW VIP CARP, the same message occurs.

          If I remove the GW and DNS, I get internet access again, ping, etc.

          https://eliasmoraispereira.wordpress.com/

          S 1 Reply Last reply Reply Quote 0
          • S
            SteveITS Galactic Empire @empbilly
            last edited by

            @empbilly What's the subnet mask on the interface? I believe pfSense defaults to /32 not /24.

            Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
            When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
            Upvote 👍 helpful posts!

            empbillyE 1 Reply Last reply Reply Quote 0
            • empbillyE
              empbilly @SteveITS
              last edited by empbilly

              @steveits

              The interface I am testing is a /27. From what I read in the doc, CARP needs to be in the same mask.

              Subnet 10.10.56.0/27
              CARP 10.10.56.1/27

              1c35d38b-689f-4c1f-81b6-8588b476f73d-image.png

              https://eliasmoraispereira.wordpress.com/

              1 Reply Last reply Reply Quote 0
              • V
                viragomann
                last edited by

                @empbilly
                Are the CARP VIPs shown as master in Status > CARP?

                If the devices cannot access / ping the VIP even it is permitted by firewall rules, it's possibly a layer 2 issue.
                Check if the CARP VIPs are shown up in the devices ARP table.

                empbillyE 1 Reply Last reply Reply Quote 0
                • empbillyE
                  empbilly @viragomann
                  last edited by

                  @viragomann said in GW configuration - some questions:

                  Are the CARP VIPs shown as master in Status > CARP?

                  yes

                  @viragomann said in GW configuration - some questions:

                  Check if the CARP VIPs are shown up in the devices ARP table.

                  None appears.

                  @viragomann said in GW configuration - some questions:

                  If the devices cannot access / ping the VIP even it is permitted by firewall rules, it's possibly a layer 2 issue.

                  They are connected to a dell N1524 switch with 4 ports in lag + dynamic lacp.

                  Do you have any direction regarding this layer 2 problem?

                  https://eliasmoraispereira.wordpress.com/

                  1 Reply Last reply Reply Quote 0
                  • V
                    viragomann
                    last edited by

                    @empbilly
                    Sniff the traffic for the ARP communication.
                    If you try to ping the CARP VIP from a device it broadcasts an ARP request at first if the IP isn't in itsARP table yet. This request should be seen on all NICs in the layer 2 network.
                    Maybe you can capture the packets on the switch ports.

                    The CARP MAC address is somewhat 00:00:5e:00:01:XX. The last byte is the VHID. Hence the VHID must be unique within a L2.

                    empbillyE 1 Reply Last reply Reply Quote 0
                    • empbillyE
                      empbilly @viragomann
                      last edited by

                      @viragomann

                      sw-core

                      IP Address       MAC Address        Interface       Type      Age
---------------  -----------------  --------------  --------  -----------
172.16.0.1       0000.5E00.0113     Vl1             Gateway    0h 12m 10s
...
172.18.0.254     0000.5E00.0118     Vl900           Gateway    0h  7m 27s
250.251.252.254  0202.BC80.0002     Management      Dynamic   n/a

                      I accessed the SW where pfmaster is connected via lag and below are the only ARPs referring to the GW CARP.

                      Vlan1 (DEFAULT) and vlan900 (GERENCIA - management vlan of our switches)

                      0d6afda4-b26b-4f51-b222-ca3e2c269caf-image.png

                      Any clue?

                      https://eliasmoraispereira.wordpress.com/

                      V 1 Reply Last reply Reply Quote 0
                      • V
                        viragomann @empbilly
                        last edited by

                        @empbilly
                        As suggested, sniff the ARP traffic to find out more about the problem.
                        I suspect, ARP requests for the CARP VIP from devices behind the switch do not arrive at pfSense. Hence the device cannot get an ARP info for it.

                        There are devices which don't like different MAC addresses on a single port by default. But this is how CARP and VRRP works. Maybe your switch is concerned of this. If so there might be a setting to allow it, however.

                        empbillyE 1 Reply Last reply Reply Quote 0
                        • empbillyE
                          empbilly @viragomann
                          last edited by

                          @viragomann

                          This switch has the VRRP protocol for configuration. Knowing this, we can conclude that it supports CARP?

                          I just didn't find anything about this model or series that talks about CARP.

                          22e5d6ab-17ae-4e65-a274-6b6d083a3771-image.png

                          https://eliasmoraispereira.wordpress.com/

                          V 1 Reply Last reply Reply Quote 0
                          • V
                            viragomann @empbilly
                            last edited by

                            @empbilly
                            The switch has nothing to do with CARP as long as he is working only on layer 2. For a working CARP, the switch only needs to support full layer 2.
                            The same is true for ARP.
                            If CARP wasn't working due to the protocol is not passing the switch, both nodes would show master status.

                            But that it supports VRRP means that the device is layer 3 capable. This could impact the ARP protocol though.
                            But without sniffing the traffic, you are in the dark here.

                            empbillyE 1 Reply Last reply Reply Quote 0
                            • empbillyE
                              empbilly @viragomann
                              last edited by empbilly

                              @viragomann

                              I am not at work these days, but I logged in remotely and checked the following output:

                              sw-core2#show mac address-table interface port-channel 1 | include 0000.5E00

1        0000.5E00.0113        Dynamic     Po1
25       0000.5E00.0102        Dynamic     Po1
26       0000.5E00.0101        Dynamic     Po1
40       0000.5E00.0111        Dynamic     Po1
50       0000.5E00.010E        Dynamic     Po1
51       0000.5E00.010F        Dynamic     Po1
53       0000.5E00.0112        Dynamic     Po1
77       0000.5E00.0115        Dynamic     Po1
200      0000.5E00.0104        Dynamic     Po1
301      0000.5E00.0105        Dynamic     Po1
302      0000.5E00.0106        Dynamic     Po1
303      0000.5E00.0107        Dynamic     Po1
304      0000.5E00.0108        Dynamic     Po1
305      0000.5E00.0109        Dynamic     Po1
306      0000.5E00.010A        Dynamic     Po1
307      0000.5E00.010B        Dynamic     Po1
308      0000.5E00.0116        Dynamic     Po1
525      0000.5E00.0114        Dynamic     Po1
610      0000.5E00.0117        Dynamic     Po1
800      0000.5E00.010C        Dynamic     Po1
900      0000.5E00.0118        Dynamic     Po1
911      0000.5E00.010D        Dynamic     Po1
912      0000.5E00.0110        Dynamic     Po1
920      0000.5E00.0119        Dynamic     Po1

                              Doing the conversion, it looks to me like the VHIDs according to vlan.

                              0000.5E00.0113: VHID 19
0000.5E00.0102: VHID 2
0000.5E00.0101: VHID 1
0000.5E00.0111: VHID 17
0000.5E00.010E: VHID 14
0000.5E00.010F: VHID 15
0000.5E00.0112: VHID 18
0000.5E00.0115: VHID 21
0000.5E00.0104: VHID 4
0000.5E00.0105: VHID 5
0000.5E00.0106: VHID 6
0000.5E00.0107: VHID 7
0000.5E00.0108: VHID 8
0000.5E00.0109: VHID 9
0000.5E00.010A: VHID 10
0000.5E00.0116: VHID 22
0000.5E00.0114: VHID 20
0000.5E00.0117: VHID 23
0000.5E00.010C: VHID 12
0000.5E00.0118: VHID 24
0000.5E00.0110: VHID 16
0000.5E00.0119: VHID 25

                              With this information, can we consider that the CARP is passing through the switch, at first? I will still try to sniff the physical interfaces.

                              The only one that doesn't show up is the WAN, which has VHID 3.

                              https://eliasmoraispereira.wordpress.com/

                              empbillyE 1 Reply Last reply Reply Quote 0
                              • empbillyE
                                empbilly @empbilly
                                last edited by empbilly

                                @viragomann

                                I was able to capture the packets.

                                Can you help me analyze the packets? Can I post the file here?

                                https://eliasmoraispereira.wordpress.com/

                                V 1 Reply Last reply Reply Quote 0
                                • V
                                  viragomann @empbilly
                                  last edited by

                                  @empbilly
                                  Did you run the capture on VLAN interface set the protocol filter to ARP?
                                  So it shouldn't show any public IP. You can hide the MACs if you want.

                                  empbillyE 1 Reply Last reply Reply Quote 0
                                  • empbillyE
                                    empbilly @viragomann
                                    last edited by

                                    @viragomann

                                    I capture all packets.
                                    https://www.dell.com/support/kbdoc/pt-br/000184095/dell-emc-networking-os6-series-switches-how-to

                                    I scp the .pcap file from the switch to my computer and opened it in wireshark.

                                    I couldn't find an option where I could specify the vlan for packet capture.

                                    Can it be with all traffic?

                                    https://eliasmoraispereira.wordpress.com/

                                    V 1 Reply Last reply Reply Quote 0
                                    • V
                                      viragomann @empbilly
                                      last edited by

                                      @empbilly said in GW configuration - some questions:

                                      I couldn't find an option where I could specify the vlan for packet capture.

                                      This should be possible in the capture configuration. Isn't there any interface setting?

                                      Can it be with all traffic?

                                      Yes.

                                      Run the capture on the port which your PC is connected. Start it and try to ping a CARP VIP.
                                      If the IP isn't in the PCs ARP table there should be an ARP request and a response in the capture. But presumably it doesn't get a response.

                                      Then run a capture on the port which is connected to pfSense to see if the request is passed through.

                                      empbillyE 1 Reply Last reply Reply Quote 0
                                      • empbillyE
                                        empbilly @viragomann
                                        last edited by

                                        I have configured VIP CARP of vlan IOT as GW in dhcp. The file below is from a direct capture from pfsense.
                                        packetcapture.cap

                                        @viragomann said in GW configuration - some questions:

                                        Then run a capture on the port which is connected to pfSense to see if the request is passed through.

                                        Tomorrow I will do that part.

                                        https://eliasmoraispereira.wordpress.com/

                                        V 1 Reply Last reply Reply Quote 0
                                        • V
                                          viragomann @empbilly
                                          last edited by

                                          @empbilly said in GW configuration - some questions:

                                          I have configured VIP CARP of vlan IOT as GW in dhcp.

                                          And which IP are we lookinf for?

                                          empbillyE 1 Reply Last reply Reply Quote 0
                                          • empbillyE
                                            empbilly @viragomann
                                            last edited by

                                            @viragomann

                                            vlan IOT: 10.10.56.0/27
                                            VIP CARP/GW: 10.10.56.1
                                            DHCP server/interface: 10.10.56.2 (pfmaster)

                                            Test from IP: 10.10.56.13

                                            https://eliasmoraispereira.wordpress.com/

                                            V 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.