GW configuration - some questions
-
@empbilly
Sniff the traffic for the ARP communication.
If you try to ping the CARP VIP from a device it broadcasts an ARP request at first if the IP isn't in itsARP table yet. This request should be seen on all NICs in the layer 2 network.
Maybe you can capture the packets on the switch ports.The CARP MAC address is somewhat 00:00:5e:00:01:XX. The last byte is the VHID. Hence the VHID must be unique within a L2.
-
sw-core
IP Address MAC Address Interface Type Age --------------- ----------------- -------------- -------- ----------- 172.16.0.1 0000.5E00.0113 Vl1 Gateway 0h 12m 10s ... 172.18.0.254 0000.5E00.0118 Vl900 Gateway 0h 7m 27s 250.251.252.254 0202.BC80.0002 Management Dynamic n/aI accessed the SW where pfmaster is connected via lag and below are the only ARPs referring to the GW CARP.
Vlan1 (DEFAULT) and vlan900 (GERENCIA - management vlan of our switches)
Any clue?
-
@empbilly
As suggested, sniff the ARP traffic to find out more about the problem.
I suspect, ARP requests for the CARP VIP from devices behind the switch do not arrive at pfSense. Hence the device cannot get an ARP info for it.There are devices which don't like different MAC addresses on a single port by default. But this is how CARP and VRRP works. Maybe your switch is concerned of this. If so there might be a setting to allow it, however.
-
This switch has the VRRP protocol for configuration. Knowing this, we can conclude that it supports CARP?
I just didn't find anything about this model or series that talks about CARP.
-
@empbilly
The switch has nothing to do with CARP as long as he is working only on layer 2. For a working CARP, the switch only needs to support full layer 2.
The same is true for ARP.
If CARP wasn't working due to the protocol is not passing the switch, both nodes would show master status.But that it supports VRRP means that the device is layer 3 capable. This could impact the ARP protocol though.
But without sniffing the traffic, you are in the dark here. -
I am not at work these days, but I logged in remotely and checked the following output:
sw-core2#show mac address-table interface port-channel 1 | include 0000.5E00 1 0000.5E00.0113 Dynamic Po1 25 0000.5E00.0102 Dynamic Po1 26 0000.5E00.0101 Dynamic Po1 40 0000.5E00.0111 Dynamic Po1 50 0000.5E00.010E Dynamic Po1 51 0000.5E00.010F Dynamic Po1 53 0000.5E00.0112 Dynamic Po1 77 0000.5E00.0115 Dynamic Po1 200 0000.5E00.0104 Dynamic Po1 301 0000.5E00.0105 Dynamic Po1 302 0000.5E00.0106 Dynamic Po1 303 0000.5E00.0107 Dynamic Po1 304 0000.5E00.0108 Dynamic Po1 305 0000.5E00.0109 Dynamic Po1 306 0000.5E00.010A Dynamic Po1 307 0000.5E00.010B Dynamic Po1 308 0000.5E00.0116 Dynamic Po1 525 0000.5E00.0114 Dynamic Po1 610 0000.5E00.0117 Dynamic Po1 800 0000.5E00.010C Dynamic Po1 900 0000.5E00.0118 Dynamic Po1 911 0000.5E00.010D Dynamic Po1 912 0000.5E00.0110 Dynamic Po1 920 0000.5E00.0119 Dynamic Po1Doing the conversion, it looks to me like the VHIDs according to vlan.
0000.5E00.0113: VHID 19 0000.5E00.0102: VHID 2 0000.5E00.0101: VHID 1 0000.5E00.0111: VHID 17 0000.5E00.010E: VHID 14 0000.5E00.010F: VHID 15 0000.5E00.0112: VHID 18 0000.5E00.0115: VHID 21 0000.5E00.0104: VHID 4 0000.5E00.0105: VHID 5 0000.5E00.0106: VHID 6 0000.5E00.0107: VHID 7 0000.5E00.0108: VHID 8 0000.5E00.0109: VHID 9 0000.5E00.010A: VHID 10 0000.5E00.0116: VHID 22 0000.5E00.0114: VHID 20 0000.5E00.0117: VHID 23 0000.5E00.010C: VHID 12 0000.5E00.0118: VHID 24 0000.5E00.0110: VHID 16 0000.5E00.0119: VHID 25With this information, can we consider that the CARP is passing through the switch, at first? I will still try to sniff the physical interfaces.
The only one that doesn't show up is the WAN, which has VHID 3.
-
I was able to capture the packets.
Can you help me analyze the packets? Can I post the file here?
-
@empbilly
Did you run the capture on VLAN interface set the protocol filter to ARP?
So it shouldn't show any public IP. You can hide the MACs if you want. -
I capture all packets.
https://www.dell.com/support/kbdoc/pt-br/000184095/dell-emc-networking-os6-series-switches-how-toI scp the .pcap file from the switch to my computer and opened it in wireshark.
I couldn't find an option where I could specify the vlan for packet capture.
Can it be with all traffic?
-
@empbilly said in GW configuration - some questions:
I couldn't find an option where I could specify the vlan for packet capture.
This should be possible in the capture configuration. Isn't there any interface setting?
Can it be with all traffic?
Yes.
Run the capture on the port which your PC is connected. Start it and try to ping a CARP VIP.
If the IP isn't in the PCs ARP table there should be an ARP request and a response in the capture. But presumably it doesn't get a response.Then run a capture on the port which is connected to pfSense to see if the request is passed through.
-
I have configured VIP CARP of vlan IOT as GW in dhcp. The file below is from a direct capture from pfsense.
packetcapture.cap@viragomann said in GW configuration - some questions:
Then run a capture on the port which is connected to pfSense to see if the request is passed through.
Tomorrow I will do that part.
-
@empbilly said in GW configuration - some questions:
I have configured VIP CARP of vlan IOT as GW in dhcp.
And which IP are we lookinf for?
-
vlan IOT: 10.10.56.0/27
VIP CARP/GW: 10.10.56.1
DHCP server/interface: 10.10.56.2 (pfmaster)Test from IP: 10.10.56.13
-
@empbilly
So you can see the ARP requests for the VIP and responses with the CARP MAC.
If these responses reach the device at 10.10.56.13, it should add the MAC to its ARP table.But obviously there is something going wrong. Even though there are ARP requests and correct responses for the CARP, the VIP is never addressed.
-
@viragomann said in GW configuration - some questions:
Even though there are ARP requests and correct responses for the CARP, the VIP is never addressed.
That is not good!!! :(
Yesterday I tried disabling the IGMP protocol on the switch, but anyway, I still can't get internet access when I configure VIP CARP as GW in DHCP.
I made new packet captures and they seem to say the same thing as the one above that I shared.
I would have to disable IGMP on all the switches in our infrastructure, correct?
EDIT1:
Anyway I disabled IGMP Snooping on the switches connected from our NOC to our office, but it still didn't go. Either they don't support the CARP traffic or there is some configuration to be done. -
@empbilly
To you see the ARP responses from 10.10.56.1 also on 10.10.56.13?
And if so, do you see the entry in its ARP table?If the ARP responses from the VIP reache the device, the issue can't be caused by the switch.
Is it the same result if you try to access a CARP VIP from another device?
-
@viragomann said in GW configuration - some questions:
To you see the ARP responses from 10.10.56.1 also on 10.10.56.13?
And if so, do you see the entry in its ARP table?10.10.56.13, now .11 is my celphone
packetcapture.cap (new capture)
@viragomann said in GW configuration - some questions:
Is it the same result if you try to access a CARP VIP from another device?
From the machine I use for work, I can ping all the CARP VIPs.
In pfsense packet capture I have selected the vlan (ADM_LAN) corresponding to the IP of my work machine.
The result is below:
23 2.599865 10.60.0.7 10.10.56.1 ICMP 98 Echo (ping) request id=0x002a, seq=1/256, ttl=64 (reply in 24)
24 2.599953 10.10.56.1 10.60.0.7 ICMP 98 Echo (ping) reply id=0x002a, seq=1/256, ttl=64 (request in 23)
31 3.614359 10.60.0.7 10.10.56.1 ICMP 98 Echo (ping) request id=0x002a, seq=2/512, ttl=64 (reply in 32)
32 3.614376 10.10.56.1 10.60.0.7 ICMP 98 Echo (ping) reply id=0x002a, seq=2/512, ttl=64 (request in 31)
35 4.638379 10.60.0.7 10.10.56.1 ICMP 98 Echo (ping) request id=0x002a, seq=3/768, ttl=64 (reply in 36)
36 4.638397 10.10.56.1 10.60.0.7 ICMP 98 Echo (ping) reply id=0x002a, seq=3/768, ttl=64 (request in 35)If I configure it to capture packets from the IOT vlan and ping from my work machine to GW 10.10.56.1, nothing is captured.
But, I tried pinging another IP (10.10.56.7 - Solar Converter), it didn't ping, but the captured packets appeared.
62 13.745105 10.60.0.7 10.10.56.7 ICMP 98 Echo (ping) request id=0x002e, seq=1/256, ttl=63 (no response found!)
64 14.767583 10.60.0.7 10.10.56.7 ICMP 98 Echo (ping) request id=0x002e, seq=2/512, ttl=63 (no response found!)
67 15.791683 10.60.0.7 10.10.56.7 ICMP 98 Echo (ping) request id=0x002e, seq=3/768, ttl=63 (no response found!)
72 16.815664 10.60.0.7 10.10.56.7 ICMP 98 Echo (ping) request id=0x002e, seq=4/1024, ttl=63 (no response found!)
77 17.839493 10.60.0.7 10.10.56.7 ICMP 98 Echo (ping) request id=0x002e, seq=5/1280,Other information. I removed VIP CARP 10.10.56.1 from Virtual VIP, tried to ping from my machine and it did not ping. I reconfigured and it pinged.
-
EDIT:
-
Other information. I removed VIP CARP 10.10.56.1 from Virtual VIP, tried to ping from my machine and it did not ping. I reconfigured and it pinged.
-
show arp on n1524 shows:
#show arp Static ARP entries are only active when the IP address is reachable on a local subnet Age Time (seconds)............................. 1200 Response Time (seconds)........................ 1 Retries........................................ 4 Cache Size..................................... 893 Dynamic Renew Mode ............................ Enable Total Entry Count Current / Peak .............. 14 / 18 Static Entry Count Configured / Active / Max .. 0 / 0 / 16 IP Address MAC Address Interface Type Age --------------- ----------------- -------------- -------- ----------- 172.16.0.2 8444.6478.FE49 Vl1 Gateway 0h 0m 1s 172.16.4.49 20FD.F146.7102 Vl1 Dynamic 0h 2m 59s 172.16.4.56 D07E.28D5.9073 Vl1 Dynamic 0h 18m 13s 172.16.4.217 E4F0.04D8.8F21 Vl1 Local n/a 172.16.7.129 D07E.28D7.1A9D Vl1 Dynamic 0h 18m 41s 172.16.15.16 D07E.28E4.CD3D Vl1 Dynamic 0h 0m 14s 172.18.0.1 20FD.F150.1389 Vl900 Dynamic 0h 8m 28s 172.18.0.5 2401.C7B1.35C1 Vl900 Dynamic 0h 17m 22s 172.18.0.14 04DA.D206.F1C1 Vl900 Dynamic 0h 17m 11s 172.18.0.17 2401.C7B1.3541 Vl900 Dynamic 0h 3m 16s 172.18.0.21 04DA.D206.F4C1 Vl900 Dynamic 0h 13m 18s 172.18.0.29 E4F0.04D8.8F21 Vl900 Local n/a 172.18.0.252 8444.6478.FE49 Vl900 Dynamic 0h 0m 12s 172.18.0.254 0000.5E00.0118 Vl900 Gateway 0h 0m 40s 250.251.252.254 0202.BC80.0002 Management Dynamic n/aThe vlan 900 is the management vlan for the switches and is the native vlan on the uplink port. The only GW VIP CARP that shows on the arp is from this vlan.
172.18.0.254 0000.5E00.0118 Vl900 Gateway -
-
@empbilly said in GW configuration - some questions:
The vlan 900 is the management vlan for the switches and is the native vlan on the uplink port. The only GW VIP CARP that shows on the arp is from this vlan.
I don't expect to see other CARP VIP in the switch ARP table.
If the switch provides layer 2 for the devices it must not communicate with the VIPs of other subnets, but only its management subnet.
The connected devices need to access them, however, since they use them as gateways. -
@viragomann said in GW configuration - some questions:
I don't expect to see other CARP VIP in the switch ARP table.
ok.
Do I need to disable the IGMP protocol only on the switch that pfsense is directly connected to or do I need to disable it on the other switches in our infrastructure as well?
