GW configuration - some questions
-
hi,
We have more or less 15 vlans in our infrastructure. I have set up CARP VIP for each one.
Example:
vlan 10 pfmaster GW: 10.10.10.2 pfbackup GW: 10.10.10.3 CARP VIP GW: 10.10.10.1In DHCP I set CARP VIP as GW.
Even if I haven't put pfbackup on the network yet, the GW CARP VIP should work, shouldn't it? If so, I have a problem, because if I set CARP VIP as GW in each vlan, the users have no internet access.
-
@empbilly Yes the VIP should work. Do you have rules on each VLAN allowing access? By default only LAN has rules so other interfaces have only the default block rule. So it will need appropriate rules, e.g. allow DNS to pfSense, block to pfSense port 80/443/22, block to LAN, allow to any, whatever is appropriate.
-
Yes I do!!!
We have been using this appliance with pfsense for many years and previously the GW of each vlan is the CARP VIP of now.
-
@empbilly Can you ping the VLAN IP if you allow ICMP?
-
@empbilly said in GW configuration - some questions:
because if I set CARP VIP as GW in each vlan, the users have no internet access.
Can you rule out a DNS issue? Did you try to ping e.g. 8.8.8.8?
-
@viragomann and @SteveITS
I have allowed all on a vlan and am testing.
In GW I put the VIP CARP 10.10.56.1
In DNS I already put VIP, google, our internal DNS, etc.When I ping google, I get the message "Destination host unreachable".
When I ping the GW VIP CARP, the same message occurs.If I remove the GW and DNS, I get internet access again, ping, etc.
-
@empbilly What's the subnet mask on the interface? I believe pfSense defaults to /32 not /24.
-
The interface I am testing is a /27. From what I read in the doc, CARP needs to be in the same mask.
Subnet 10.10.56.0/27
CARP 10.10.56.1/27 -
@empbilly
Are the CARP VIPs shown as master in Status > CARP?If the devices cannot access / ping the VIP even it is permitted by firewall rules, it's possibly a layer 2 issue.
Check if the CARP VIPs are shown up in the devices ARP table. -
@viragomann said in GW configuration - some questions:
Are the CARP VIPs shown as master in Status > CARP?
yes
@viragomann said in GW configuration - some questions:
Check if the CARP VIPs are shown up in the devices ARP table.
None appears.
@viragomann said in GW configuration - some questions:
If the devices cannot access / ping the VIP even it is permitted by firewall rules, it's possibly a layer 2 issue.
They are connected to a dell N1524 switch with 4 ports in lag + dynamic lacp.
Do you have any direction regarding this layer 2 problem?
-
@empbilly
Sniff the traffic for the ARP communication.
If you try to ping the CARP VIP from a device it broadcasts an ARP request at first if the IP isn't in itsARP table yet. This request should be seen on all NICs in the layer 2 network.
Maybe you can capture the packets on the switch ports.The CARP MAC address is somewhat 00:00:5e:00:01:XX. The last byte is the VHID. Hence the VHID must be unique within a L2.
-
sw-core
IP Address MAC Address Interface Type Age --------------- ----------------- -------------- -------- ----------- 172.16.0.1 0000.5E00.0113 Vl1 Gateway 0h 12m 10s ... 172.18.0.254 0000.5E00.0118 Vl900 Gateway 0h 7m 27s 250.251.252.254 0202.BC80.0002 Management Dynamic n/aI accessed the SW where pfmaster is connected via lag and below are the only ARPs referring to the GW CARP.
Vlan1 (DEFAULT) and vlan900 (GERENCIA - management vlan of our switches)
Any clue?
-
@empbilly
As suggested, sniff the ARP traffic to find out more about the problem.
I suspect, ARP requests for the CARP VIP from devices behind the switch do not arrive at pfSense. Hence the device cannot get an ARP info for it.There are devices which don't like different MAC addresses on a single port by default. But this is how CARP and VRRP works. Maybe your switch is concerned of this. If so there might be a setting to allow it, however.
-
This switch has the VRRP protocol for configuration. Knowing this, we can conclude that it supports CARP?
I just didn't find anything about this model or series that talks about CARP.
-
@empbilly
The switch has nothing to do with CARP as long as he is working only on layer 2. For a working CARP, the switch only needs to support full layer 2.
The same is true for ARP.
If CARP wasn't working due to the protocol is not passing the switch, both nodes would show master status.But that it supports VRRP means that the device is layer 3 capable. This could impact the ARP protocol though.
But without sniffing the traffic, you are in the dark here. -
I am not at work these days, but I logged in remotely and checked the following output:
sw-core2#show mac address-table interface port-channel 1 | include 0000.5E00 1 0000.5E00.0113 Dynamic Po1 25 0000.5E00.0102 Dynamic Po1 26 0000.5E00.0101 Dynamic Po1 40 0000.5E00.0111 Dynamic Po1 50 0000.5E00.010E Dynamic Po1 51 0000.5E00.010F Dynamic Po1 53 0000.5E00.0112 Dynamic Po1 77 0000.5E00.0115 Dynamic Po1 200 0000.5E00.0104 Dynamic Po1 301 0000.5E00.0105 Dynamic Po1 302 0000.5E00.0106 Dynamic Po1 303 0000.5E00.0107 Dynamic Po1 304 0000.5E00.0108 Dynamic Po1 305 0000.5E00.0109 Dynamic Po1 306 0000.5E00.010A Dynamic Po1 307 0000.5E00.010B Dynamic Po1 308 0000.5E00.0116 Dynamic Po1 525 0000.5E00.0114 Dynamic Po1 610 0000.5E00.0117 Dynamic Po1 800 0000.5E00.010C Dynamic Po1 900 0000.5E00.0118 Dynamic Po1 911 0000.5E00.010D Dynamic Po1 912 0000.5E00.0110 Dynamic Po1 920 0000.5E00.0119 Dynamic Po1Doing the conversion, it looks to me like the VHIDs according to vlan.
0000.5E00.0113: VHID 19 0000.5E00.0102: VHID 2 0000.5E00.0101: VHID 1 0000.5E00.0111: VHID 17 0000.5E00.010E: VHID 14 0000.5E00.010F: VHID 15 0000.5E00.0112: VHID 18 0000.5E00.0115: VHID 21 0000.5E00.0104: VHID 4 0000.5E00.0105: VHID 5 0000.5E00.0106: VHID 6 0000.5E00.0107: VHID 7 0000.5E00.0108: VHID 8 0000.5E00.0109: VHID 9 0000.5E00.010A: VHID 10 0000.5E00.0116: VHID 22 0000.5E00.0114: VHID 20 0000.5E00.0117: VHID 23 0000.5E00.010C: VHID 12 0000.5E00.0118: VHID 24 0000.5E00.0110: VHID 16 0000.5E00.0119: VHID 25With this information, can we consider that the CARP is passing through the switch, at first? I will still try to sniff the physical interfaces.
The only one that doesn't show up is the WAN, which has VHID 3.
-
I was able to capture the packets.
Can you help me analyze the packets? Can I post the file here?
-
@empbilly
Did you run the capture on VLAN interface set the protocol filter to ARP?
So it shouldn't show any public IP. You can hide the MACs if you want. -
I capture all packets.
https://www.dell.com/support/kbdoc/pt-br/000184095/dell-emc-networking-os6-series-switches-how-toI scp the .pcap file from the switch to my computer and opened it in wireshark.
I couldn't find an option where I could specify the vlan for packet capture.
Can it be with all traffic?
-
@empbilly said in GW configuration - some questions:
I couldn't find an option where I could specify the vlan for packet capture.
This should be possible in the capture configuration. Isn't there any interface setting?
Can it be with all traffic?
Yes.
Run the capture on the port which your PC is connected. Start it and try to ping a CARP VIP.
If the IP isn't in the PCs ARP table there should be an ARP request and a response in the capture. But presumably it doesn't get a response.Then run a capture on the port which is connected to pfSense to see if the request is passed through.