Unexpected alias behaviour - two ranges

stephenw10

Yup. However I also can't replicate the original issue where an empty line is created. Adding multiple subnets there creates a valid alias with all subnets expanded.

So I suspect there was a rogue character in @Patch's imported list as you suggested.

As such I'm inclined to do nothing here. Anything we did do would likely break the workflow for anyone using the undocumented behaviour.

SteveITS

Now that I know about this “feature” I can see where it’s going to save me some time. :)

Patch

Sorry about the delay in responding.
pfsense was behaving very strangely which was complicated by

testing on my home active router
pfsense configuration evolved over many years and includes a 3 packages (eg pfBlockerNG)
frequent restarts required
fault detection I suspect was using failure to include specified entries in an alias -> hybrid NAT rule failed -> after firewall restart failure to register of 1 of 4 VoIP suppliers

Solution, simplify the test set up

Create a new VM on another appliance, with at least 2 NIC (actually 3 passed through)
Clean install pfsense v2.8.1 via installer v1.1. Allow DHCP over ride of DNS & network time.
Enable Kea, enable pfsense GUI from Wan (wan pass rule, un-check Block private network and loopback addresses)
Slowly rebuild aliases of interest, regularly saving configuration after each error free entry
Error detection based on comparing the Alias definition shown in Firewall -> Aliases -> IP -> <Alias> -> edit. Compared to Diagnostics -> Tables -> <Alias>

The result of which is:

@stephenw10 said in Unexpected alias behaviour - two ranges:

I suspect there was a rogue character in @Patch's imported list

You are correct. The networks were separated by a double space. Sorry the forum removes double spaces so you could not copy from my post. Using a single space prevents creation of the blank host. But doing so does not fix the problem with creating the desired Alias.

I believe the task which creates the table from the alias definition is too resource limited and the performance optimisation some times means the desired table is never created.

For example

Create an Alias (network type). Add a couple of /24 networks and a couple of FQDN or /32 addresses. Save and apply changes. They show up in the Alias table as desired. Edit the Alias definition to delete the /24 networks and on of the /32 addresses. Save and apply the changes. The /32 address will be removed but the /24 networks will stay. Reboot pfsense and the /32 will also be removed leaving only the valid /32 entry
Create an Alias (host type). Add a FQDN and two /24 networks one of which includes the FQDN IPv4 address. Save and apply. Look at the filter reload screen, when complete look at the created table for the Alias. For me it shows about 470 entries initially. Manually triggering Status -> Filter Reload -> adds about a couple more entries to the table for the alias. Wait 5 minutes for the Aliases Hostnames Resolve Interval to time out -> Alias table shows all 512 entries (duplicate removed)
Delete all the /24 network entries from the above alias definition, save and apply. Look at the Alias table, in now contains no entries. The single table entry for the duplicate address was deleted on removal of the explicit IP address. Filter reload and waiting >5min makes no change. Restart pfsense and the entry is repopulated.
Repeat test 2. but use with a single line entry of both /24 networks in one line. For me about 150 entries appear initially in the table, a few more with each Filter reload, all on waiting 5 minutes
A similar difference is seen in loading the 50 Crazytel IP. Initially pasting the 50 IP in on line sorted in reverse order loaded 7 entries. Repeating the test after the VM had been running for several hours loaded 25 entries. Pasting them in sorted in ascending order may have resulted in a couple more being initially loaded. As before manually triggering a filter reload added a few more. Wait 5 minutes and they are all loaded. Manually entering each IP into a separate line in the Alias and all 50 are loaded initially.
Running similar tests on configuration with many other Aliases (and pfblocker) and less is done initially. The 5 minute alias reload appears not to have the resources / time to ever complete some aliases

Summary

pasting multiple single space separated entries into as single line is very efficient for data entry but appears to reduce initial processing done on alias entry by about a halve to a third.
pfsense spreads alias processing over multiple Aliases Hostnames Resolve Intervals if required. More processing is done on reboot. Less processing is done on Filter reload.
More complex configurations slow the Alias processing (perhaps to a stand still at times)
Alias processing optimisation maybe over zealous at times requiring pfsense reboot to resolve.
a force alias full regeneration option would be nice during configuration updates. Or perhaps more time devoted to it on manual filter reload

Patch

The alias processing does to some cool things which I like such as:

enter range IP xx.xx.xx.xx-yy.yy.yy.yy in Host or network aliases
in Network alias paste xx.xx.xx.xx/yy -> sets mask without needing to manually select via menu
Change Host alias to Network alias to enable adding networks to an existing alias (The reverse can be done however that expanded networks which are not automatically re-compressed if you change your mind).
Alias can included an alias simplifying maintenance and documentation.
Multiple entries can be added in a single line which pfsense automatically expands. Doing so reduces initial table processing which is not trivial to accelerate.
Duplicate host entries are automatically removed. However there is some risk of no entry being left there if one of the entries is delete (or maybe also changed). Duplicate covered by a network range appear not to be removed (depending on the filter design this may have minimal performance impact and maintaining a single entry for these duplicates maybe difficult). Similarly a relatively long consecutive sequence of host IP appears not to be converted to a range (but maybe else where in actual filter design)

tinfoilmatt

@Patch said in Unexpected alias behaviour - two ranges:

Using a single space prevents creation of the blank host. But doing so does not fix the problem with creating the desired Alias.

Yes it does. The rest is a super longwinded way of saying, 'my bad.'

SteveITS

@Patch said in Unexpected alias behaviour - two ranges:

5 minute alias reload appears not to have the resources / time to ever complete some aliases

Are you talking about FQDNs or the /24 subnets here? I thought the 5 minute timer was for resolving FQDNs via DNS. We have a scenario where some either stop resolving or are maybe never added to the table...hard to tell since 99% of the time they're not used. However per your description, some may overlap (laptop goes to an allowed public IP) so I'm wondering if one fails it is removing "both" IPs/entries?

stephenw10

Hmm OK I can replicate case 2 here. Digging....

Do you see the full alias set shown in the Resolver logs when you add it?

For me I see that and it does load all 512 entries after a delay.

Patch

@stephenw10
I was testing last night and I think I have found the case which actually started my searching.

create an alias type network containing two /24 network and a couple of. /32 I have been using FQDN here which I think triggers alias table creation.
create an alias type host containing two expanded /24 network and a couple of single hosts
create an alias containing the above two alias.
I also created a firewall rule using the last alias but don’t think that’s essential
for test repeatability I have been clearing the alias tables, saving the configuration, then restoring the configuration

For me alias table generation locks up completely at about 300 entries. I think it also blocks other alias calculations. Tested in v2.8.1 and v2.7.2
Yet to test if using 512 random rather than sequential IPv4 addresses prevents the lock up

Ps
I have not looked for or at the Resolver logs. I will look when I’m able to test further

stephenw10

@Patch said in Unexpected alias behaviour - two ranges:

For me alias table generation locks up completely at about 300 entries.

That's until you run filter-reload?

And it definitely still does it without any FQDNs present? Because otherwise it looks like it hangs filterdns but that can't be the case with FQDNs.

Patch

@stephenw10 said in Unexpected alias behaviour - two ranges:

That's until you run filter-reload?

No.
Makes no difference to me. Similarly leaving it running for 24 hours makes no difference.

After some experimentation I can lock up the system immediately if I
Create the following alias (I have been using random IPv4 addresses but only minor differences occur if sequential IP addresses are used. Creating the Aliases via the bulk import option also makes no difference.)

IP_set1 : host type, 50 IPv4 hosts (/32) and at least 1 FQDN
IP_set2 : host type, 512 IPv4 hosts (/32) and at least 1 FQDN
IP_set3 : host type, 50 IPv4 hosts (/32) and at least 1 FQDN
Combined_IP : Host type consisting of the above 3 aliases (IP_set1 IP_set2 IP_set3)

Then

Create a firewall rule which uses the alias Combined_IP
Diagnostic -> Tables -> select each of the above aliases and "Empty table"
Save the configuration

To test, Restore the above configuration. My results

Diagnostic -> tables -> records: Combined_IP = 256, IP_set1=50, IP_set2=206, IP_set3=0
Waiting longer makes no difference, Filter reload makes no difference.
Create a new Alias with hosts forum.netgate.com & redmine.pfsense.org -> empty table only generated

Testing with pfsense v2.7.2 results in similar results

Combined_IP = 256, IP_set1=50, IP_set2=156, IP_set3=50

It appears pfsense alias capacity is way less than 5000 entries if an alias contains other aliases.
Not sure if this helps localise the issue. The similar Combine_IP size across software versions is interesting but is higher if only two aliasses are combined. Processor load remains trivial.

@SteveITS said in Unexpected alias behaviour - two ranges:

some may overlap (laptop goes to an allowed public IP) so I'm wondering if one fails it is removing "both" IPs/entries?
I'm yet to test this as I have been first focussing on why wide spread alias problems have been occurring.

It is on my list of things to do as I have a white list alias (containing my home IP and laptops current IP). Losing home access when the laptop leaves home is not desirable but happened recently. I was thinking of using a host over ride to try and simulate this. But the fault could have been caused by something unrelated.

Edit
Added the requirement for each IP_set to include at least 1 FQDN

SteveITS

@Patch If you run "killall filterdns" and Status>Filter Reload do the tables populate?

Patch

@SteveITS is killall filterdns run from the command line or GUI menu option?

You do realise the IP_set alias consists almost exclusively of actual IPv4 addresses. I used a spreadsheet random number generator to construct addresses in the format
222.<random 1-255>. .<random 1-255> .<random 1-255>
The leading number changed for different aliases
However I don’t think the actual IP addresses make any difference.

The issue being the failure varies with number of hosts and number of aliases. The aliases now contain less than 2 FQDN in total now.

But will try later today

SteveITS

@Patch The command should run both places. It just ends the processes. I have run it in the GUI.

Yeah I'm aware of the difference, I'm just trying to connect dots. Yours may be a totally different issue than mine, but it started to sound similar.

stephenw10

Yes you should be able to run that in either place. Though I would run it on the real command line if possible in case it does something unexpected.

Patch

@SteveITS said in Unexpected alias behaviour - two ranges:

If you run "killall filterdns" and Status>Filter Reload do the tables populate?

No but I guess this is not the expected response
70 Killall filterdns.jpg

btw @stephenw10 what happens when you try to replicate the behaviour?

While I assume it makes no difference, I'm using a Proxmox VM with 2 GB ram (GUI shows 18% memory usage), a Host type processor (i5-1235U with 2 cores), Hard disk: 8GB SSD, Bios OVMS (UEFI), Machine q35.

Patch

@SteveITS @stephenw10
Oops.
My test description was wrong.
Each IP_set alias needs at least one FQDN for the fault to be shown.

Adding the FQDN results in the table for each IP_set alias being created / viewable
Removing all FQDN results in the Combined_IP being rapidly calculated.

Above post edited to include this requirement https://forum.netgate.com/post/1229337

SteveITS

@Patch filterdns processes are left running to monitor for updates in hostnames for Aliases/IPsec/etc, one thread per hostname. So, maybe unrelated to my observed problem.

But I’d expect some if you had FQDNs to resolve…?

Patch

@stephenw10 said in Unexpected alias behaviour - two ranges:

Do you see the full alias set shown in the Resolver logs when you add it?

More than showing in the alias tables

I can't be sure all entries are shown as display is limited to 2000 entries
IP_set3 table is empty however the log shows the actual 50 IP addresses are added but duplicates of "Adding Action: pf table: IP_set3 host:" but I think all 50 appear.
Similarly "Adding Action: pf table: IP_set2 host: " shows some duplicates. Not all actual IP addresses appear in the 2000 log entires. I was not able to readily tell if all 512 appear at least once in Adding Action: pf table: IP_set2 host:

As I have not looked at these logs in the past, I'm not sure what is normal

tinfoilmatt

I think I agree at this point in some of the most incoherent SQA-masquerading-as-troubleshooting I've ever witnessed that—

It's true. The ability to add IP addresses and/or IP ranges to "Host" type aliases should be removed completely (and vice versa) via validation. That this makes no sense whatsoever on its face notwithstanding, it clearly has more than mere potential to lead to all of the above confusion.

stephenw10

Mmm, you could be right. But that is going to hurt some users. And save some others. Potentially.