CARP IP not being used via manual outbound NAT?
-
Right. Don't NAT connections from 127.0.0.0/8 to the CARP VIP.
Can you ping 24.248.X.161 in Diagnostics > Ping setting the source to both WAN and WAN CARP on the node holding CARP MASTER?
Yes, a /28 is fine, though I would rather see a /29 on the interface and the /28 routed to the WAN CARP VIP there. What you have will work but a routed subnet is more flexible.
-
No i cannot ping from WAN, yes i can ping from WAN CARP
-
Then you need to figure out why that is. pcap on the WAN for host 24.248.X.161 and run both ping tests again and and see what's really going on out there.
What is upstream of this WAN interface between you and the ISP handoff?
Feel free to upload the pcap to the nextcloud link I sent in chat if you like.
-
You write:
CARP IP 24.248.x.163 (this is the original static IP assigned to me via ISP)
Appliance 1 WAN Interface IP: 24.248.x.164
Appliance 2 WAN Interface IP: 24.248.x.165So .164 and .165 are not assigned to you?
You can't just put any public IP to your WAN and think things work.-Rico
-
@rico this is kind of my issue. My predecessor obtained a 16 IP block from the ISP and there is no clear documentation whatsoever of what it is.
I have a public IP address range starting in 72.215.x.x
but my gateway and static IP address for outbound connectivity are in 24.248.x.x
So I don't know what the ISP is doing on their routing side, and that's probably what's screwing me all up.I was under the likely mistaken assumption that both nodes would effectively be exclusively talking out via the CARP IP and that their physical interface IPs were largely vestigial.
-
Using the address that used to be on the WAN interface as the CARP VIP is a solid strategy.
@teltech said in CARP IP not being used via manual outbound NAT?:
ISP specific connection info:
Customer Static IPV4 address: 24.248.x.163
Gateway IP: 24.248.x.161
Subnet: 255.255.255.240With that configuration, .162 through .174 should be available to use on WAN as you see fit.
-
@teltech said in CARP IP not being used via manual outbound NAT?:
@rico this is kind of my issue. My predecessor obtained a 16 IP block from the ISP and there is no clear documentation whatsoever of what it is.
I have a public IP address range starting in 72.215.x.x
but my gateway and static IP address for outbound connectivity are in 24.248.x.x
So I don't know what the ISP is doing on their routing side, and that's probably what's screwing me all up.I was under the likely mistaken assumption that both nodes would effectively be exclusively talking out via the CARP IP and that their physical interface IPs were largely vestigial.
Call them and ask?
-
Yeah first get in touch with your ISP to get things clear.
-Rico
-
@derelict said in CARP IP not being used via manual outbound NAT?:
I have a public IP address range starting in 72.215.x.x
What is the range? What is the netmask? Is it routed? What address is it routed to?
-
72.215.x.0/28 - Routed - C
24.248.x.163 - Routed - Dwith a gateway of 24.248.3.161 /28
^ this is the info emailed to the individual who ordered the contract from the ISP, with 1 octet masked by me.
contract was for "17 IPs"so i can only assume the public IPs we use for port forwarding range from 72.215.x.0 - 72.215.x.16
in addition to 24.248.x.163 i'm going to get myself put on the phone with someone who knows more about the actual account as they seemed eager to avoid helping on the first pass around and i'm admittedly lacking in routing knowledge.It is almost certainly an issue between myself and the ISP, rather than a failure of pfSense. It usually is, in my experience.
-
No idea what C and D are supposed to mean.
24.248.x.163 - Routed
If they were routed there would be no gateway on that subnet. If they are routed they would need to tell you which address in the interface subnet it is being routed to. That is the address you would want to use for the CARP VIP.
You can learn a lot about what is going on with a packet capture. They either send traffic for a routed subnet directly to the MAC address in the ARP entry for the CARP VIP (The address they are routing to) or they ARP for it because they think it is on the connected interface subnet.
-
Two things I see:
Upstream is not responding at all when sourced from .164. Did you filter that packet capture on icmp? I would expect to see ARP or something there if not.
The replies to pings sourced from .163 should be destined to the CARP MAC address, not the interface MAC address.
It looks like something upstream does not like moving MAC addresses around like CARP does but just a guess at this point.
The ISP Layer 2 device will see the CARP MAC as the source MAC in the CARP advertisements. They are sent to the Layer 2 Multicast address 01:00:5e:00:00:12 (all points multicast) to Layer 3 multicast address 224.0.0.18. That MAC address has to be added to the switch port's MAC address table based on those. This MAC address will change ports on a failover event. The ISP device must move the MAC address to the new port as any switch should.
The ISP Layer 3 gear will get the CARP MAC in the "IS AT" response to ARP "WHO HAS" requests for the CARP VIP address. Their gear needs to do the right thing with it. The ARP reply from the WAN interface that is currently CARP MASTER will contain the CARP MAC in the ARP "IS AT" response. This ARP response will be sourced from the interface IP and MAC address.
The ISP Layer 3 gear also needs to honor the interface addresses that will ARP as normal. The ISP device will only ever see the interface MAC address on the port connected to that node.
