Bug: IP Alias VIP interferes with primary interface when dhclient in use

whorfin

I have a WAN interface which receives its primary interface address (public IP) via DHCP
I have a Virtual IP [IP Alias] (private local IP) on the same interface, using NAT to allow communication with the actual VDSL modem

It Would Be Great if pfSense kept the Virtual IP and interface IP distinct, and always returned the interface IP when using the "WAN address" firewirall and NAT macro, and the "WAN" interface selector of OpenVPN, etc.

This does not happen

What I observe is that "sometimes" the IP address reported for the interface under "Status->Interfaces" is correct, and sometimes it
is the Virtual IP. When it is the virtual IP, all hell breaks loose.
If I manually hit the release/renew button [again under Status->Interfaces->WAN] I seem to always get the right interface.
When the interface is updated via DHCP, however, I often get the wrong [Virtual IP] showing up. Thus, random hell breaking loose.

Digging in a little I see that everything seems to call

get_interface_ip($interface)
```from```
/etc/inc/interfaces.inc

This even contains a warning that it returns the first address found, and one can confirm with

ifconfig igb1
```that the first ip indeed seems to change to be the Virtual IP when "the badness" happens.

It is interesting that "hitting the button" always seems to result in the desired order, while automatic DHCP updates do not.

Would it be possible for pfSense to at least offer an option to fix this?  Beyond the inconsistency being dangerous and confusing,
it renders the macros unusable in this situation with no easy workaround.

One completely backwards-compatible possibility would be for a```
get_interface_ip()
```function which skips "Virtual IP (alias)" addressess on a given interface, and to offer "WAN (non-virtual) address" macros/'interface-entries which utilize that function.
Other methods which keep primary interface/virtual IP addressess truly distinct would be welcome.

Cheers

whorfin

Testing whether or not this is fixed in 2.4 requires nontrivial multi-box jiggling of my configuration.
Is there any reason to believe it has been fixed? interfaces.inc looks unchanged in this
regard

Cheers

Derelict

I have the same config. I never see what you are seeing. DHCP from the cable modem/provider and 192.168.100.2 as a VIP. pfSense 2.3.4_1

whorfin

If you do```
ifconfig igb1

that the cable modem/provider address is always first?
If you reset the modem, and force a refresh of pfsense DHCP while it is still connecting, do you still see the same scenario, or does your WAN turn to 192.168.100.2?
That's my scenario - if/when modem cycles an address [and it takes minutes, as it is syncing bonded-pair vDSL2], pfSense gets stuck with the VIP.
Even after the modem has fully come up, the auto-DHCP address ends up second on the
interface.
Once```
ifconfig igb1
```  shows the correct address second, I'm hosed.

The problem is that nothing in the pfSense interfaces.inc code distinguishes VIP from
"actual primary interface".

Derelict

I think you mean ifconfig. Can't go unplugging and plugging in my modem to test that at the moment. But I have never seen anything like you are describing on any system ever. I'll see if I can think of a way to test something similar in a non-disruptive manner.

whorfin

I think you mean ifconfig.

Oops. Yes. Got it wrong once then copy-pasta fail.

As far as testing goes -
given that the order of occurrence for addresses on an interface changes, wouldn't it be desirable to distinguish VIP from primary addresses?

To put it another way, the IP address reported for the interface under "Status->Interfaces"
for WAN is always whichever happens to be first as reported by ifconfig. The code says
as much in a warning.
DHCP responses from the modem cause this to change, ie the VIP is sometimes first.
Then badness.

In my particular case, I have tracked this to the modem changing the DHCP lease time to 10 seconds whenever it does not have a real internet address. During that time, it either returns an address in the same subnet where my VIP lives to enable direct modem communication, or it eventually returns a real internet address, and sets lease time back to the number of hours it requires for a certificate checkin. This is an AT&T Uverse thing, yes, but at least with the hardware I have, can be reliably triggered by forcing a modem reboot.
Odds are high I end up with the real address second, the VIP first, and my system hosed until I release/renew the DHCP release manually. [where "hosed" means any rule or configuration dependent on knowing the IP address of the WAN is b0rked, as the VIP address is now incorrectly considered the primary WAN address, until the release/renew dance]

Even if you are lucky enough to have not observed this, can you see how it might happen?
Guarding against this by providing a mechanism to [optionally] exclude VIPS would provably prevent this.

Derelict

You will probably need to post ifconfig igb1 output from the interface it a proper state and what you say is a broken state.

whorfin

Yes, I guess I should have done that to begin with.

When things are working vs not,```
ifconfig igb1

working:

inet 108.245.XXX.XXX netmask 0xfffffc00 broadcast 108.245.XXX.255
inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255


The obfuscated 108.245 address is the modem's address on the internet.
192.168.10.2 is the VIP, used for modem administration as the modem is at
192.168.10.1
In this scenario, 108.245.xxx.xxx is what will be used by "WAN" for firewall rules, OpenVPN, etc
"Status->Interfaces" shows this address for WAN

When things go wrong, I see:

inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255
inet 108.245.XXX.XXX netmask 0xfffffc00 broadcast 108.245.XXX.255


"Status->Interfaces" shows 192.168.10.2 address for WAN, and things are broken.