My connection requires a unique MAC for each static IP. What do?
-
RTFM eh? I found a reference to Virtual IPs in the docs that seems to precisely address my situation, yet the proposed solution does not seem to work:
https://doc.pfsense.org/index.php/What_are_Virtual_IP_Addresses
Some upstream equipment requires each distinct IP address to have a unique MAC address. In such cases, the use of CARP VIP types may allow the additional addresses to function where they otherwise would not work with IP alias or Proxy ARP VIPs. This has been common to see in the past with AT&T Uverse equipment.
The MAC address of a VIP will change if the VIP entry is changed between a type that has a unique MAC address, such as CARP, to one that shares a MAC address with a parent interface, such as IP alias or Proxy ARP. Due to the MAC address change, other equipment on the segment may need to have its ARP cache cleared, it may need to be rebooted (cable modems especially), or there may be some other time period that must expire for the ARP cache to update. This may be as few as a couple minutes or up to four hours.
If a particular configuration does not work with IP alias or Proxy ARP type VIPs, try with a CARP VIP instead, or vice versa. Address or wait out the potential ARP concerns before declaring one particular type a failure, and always be on the lookout for IP conflicts.
This sounds fantastic… however I thought CARP was specifically designed for creating redundancy by sharing IPs. And that seems to be borne out in my experiment. I tried changing the Virtual IP type to CARP, but it won't let me create a CARP IP without putting a VHID password, which of course makes no sense in this use case. Anyway, I tried CARP and as soon as I created a CARP VIP on the interface, ALL IPs on that interface stopped responding to ping. Not just intermittent ping, NO CONNECTION AT ALL.
I also tried IP Alias and Other VIP for fun, and had no success.
In summary, I found these results:
IP Alias: Tried subnets of both /24 and /32 for the VIP. Rock-steady ping for the pfsense IP, but no response from the VIP. The 1:1NAT does not seem to work at all.
CARP: Lots of settings to fool with here, including VHID password, VHID group, and Base and Skew. I tried several different options, including /24 and /32 subnets, and changing the group/base/skew to random numbers, but in all cases using CARP just gets me NO PING on the pfsense IP AND the VIP
Proxy ARP: As detailed above, results in intermittent ping for both the pfsense IP and the VIP. With single host selected, there are no subnets to select, nor any other options.
Other: Again not really any options to change here for single host. Rock steady ping from pfsense IP, but no response from the VIP. 1:1NAT does not seem to work. -
With a cable modem it more than likely doesn't require a unique MAC per-IP, but you have to power cycle the modem often after changing, adding, or removing IPs. If it does require a unique MAC, then you want CARP IPs, and power cycle the modem after adding them.
-
@cmb:
With a cable modem it more than likely doesn't require a unique MAC per-IP, but you have to power cycle the modem often after changing, adding, or removing IPs. If it does require a unique MAC, then you want CARP IPs, and power cycle the modem after adding them.
I have tried restarting (power-cycling) both the modem and the pfsense box. It does not make a difference. Besides, I don't believe power-cycling is required in this case, seeing as how whenever I add the VIP I see an immediate change in pings.
But yeah… adding a CARP and power-cycling does not change my situation.
What settings, exactly, should I be using for CARP (VHID password, group, base, skew)?
Btw, using CARP really seems like a piss-poor way of dealing with this situation. It seems to me that a simple, straightforward, and less obtuse solution would be a VIP option that has a field for MAC ID spoofing.
-
Use the defaults for all the CARP settings.
There is no way in the underlying OS to add an IP alias on a diff MAC. CARP's a perfectly fine means of doing so, and is widely used as such.
Most cable modems require a power cycle before they'll pick up a MAC change on static IPs. Changes in reachability from inside your network have no dependency on your cable modem even existing much less picking up changes you're making.
-
I've seen this MAC:IP limitation on a number of occasions, cable modems, DSL modems, WISP connections. The big telco ISPs raison d'être is all about making life hard any anyone but the most basic common user. If switching ISP is an option, let them know you are displeased by voting with your wallet.
In the meantime, CARP works because each VHID translates into a unique MAC address. As cmb pointed out, use the default CARP settings, just make each alias a unique VHID.
So…
VHID =1 = MAC: 00:00:5e:00:01:01
VHID =127 = MAC: 00:00:5e:00:01:7f
etc...
You can create a unique MAC for up to 255 VHIDs. -
Can you afford to lose two public IP addresses?
Install ESX on the hardware and create your pfsense as a virtual. Create three virtual switches and tie a physical NIC to each. Connect the three ISP connections to the three NICs. Create a fourth virtual switch and tie it to another physical NIC for your inside network.
For the pfsense virtual machine, create 10 (vmware limit) virtual NICs: 4 on cable#1 virtual switch, 4 on cable#2 virtual switch, 1 on DSL virtual switch, and 1 on inside virtual switch.
This assume that pfsense can handle 9 outside interfaces and one inside.
NOTE, this would also give you the flexibility to create multiple pfsense instances - each tied to different ISPs or public IP addresses. That way, you won't lose any public IP addresses. Depending on what you are doing, this may suit your purposes better.
If you've never used vmware before, don't be put off by learning it. ESX is fairly simple to configure for simple setups like this.
-
1. There are very limited choices in the US for ISPs as it is. I'm in a third-world country doing this setup and we have even less choices here, so voting with my wallet is not really an option.
2. pfsense is already running on ESXi v6 actually. I have two virtual routers on the machine, using the same physical interface. They both have 1 static IP each and that works fine. It is only when I assign another static IP to pfsense that the two pfsense IPs crash and burn.
3. If I could get more statics from my ISP, then I could afford to burn two, but 5 IPs is the max for this ISP, and as I said before, I don't have any other choices.
4. I thought of making a virtual NIC for each static IP, but the problem then becomes that pfsense can not deal with more than one interface using the same gateway.
5. I've tried unplugging the modem completely, adding the CARP VIP, and then plugging the modem in. Still destroys the reliability of the two IPs, as if there was some kind of conflict.
I am at wits end trying to figure out what the problem is here. It doesn't seem like the CARP VIP is working as it should be.
-
With ESX, you probably missed the config at the ESX level to allow multiple MACs to the VM.
https://doc.pfsense.org/index.php/CARP_Configuration_Troubleshooting#Hypervisor_users_.28Especially_VMware_ESX.2FESXi.29 -
Can you tell us what it you want to do with three ISP connections? Are you just needing a whole bunch of public IP addresses?
-
Maybe get a tunnel with static IPs on it instead?
-
You could look around for an ipv4 tunnel broker in any country you choose and setup a tunnel with them. You stay on dynamic IP, your fixed addresses are routed to you.
-
Setup an AWS micro instance, run pfSense in it and setup an OpenVPN link from AWS with fixed IP to your dynamic IP.
-
