Pfctl: Bad address

[wcrowder]

When loading a large IP block lists in pfSense 2.2 I get this error. pfctl: Bad address on filter reload.

After investigating this problem I ran into this thread: https://forums.freebsd.org/viewtopic.php?t=45879

I believed that System->Advanced->Firewall / Nat->Firewall Maximum Table Entries corrected this in pfSense 2.1.4, this does not seem to be working in 2.2.

Has anyone else ran into this running Prime iBlock list on pfSense 2.2?

[jimp]

IIRC that knob was removed because pf on 10.x doesn't have an upper limit. Or it isn't supposed to. Might be a different case here.

[wcrowder]

Here is an example using:

2.2-ALPHA (amd64)
built on Tue Aug 26 15:41:42 CDT 2014
FreeBSD 10.0-STABLE

This is repeatable. Any list that's approximately over 200,000 addresses and pfctl will spit out a "Bad Address" error. Also pfSense will have "failed to allocate memory" "trap" while trying to update the list when the list is updated at the source.

Here is Bluetach Level 1 added and it's output as an example.

  pfB_Top
no changes.
  pfB_IBlock
39 addresses added.1039 addresses deleted.
  pfB_PRI1
28 addresses added.120 addresses deleted.
  pfB_PRI2
35 addresses added.529 addresses deleted.
  pfB_PRI3
769 addresses added.1483 addresses deleted.
  pfB_SEC1
81 addresses added.333 addresses deleted.
  pfB_TOR
15 addresses added.86 addresses deleted.
  pfB_BTLevel1
pfctl: Bad address.
  pfB_MVPS
1 addresses added.1 addresses deleted.

Alias Table IP Counts
-----------------------------
  375242 total
  230835 /usr/local/pkg/pfblocker/aliases/pfB_BTLevel1.txt
   60984 /usr/local/pkg/pfblocker/aliases/pfB_IBlock.txt
   ...

@jimp:

IIRC that knob was removed because pf on 10.x doesn't have an upper limit. Or it isn't supposed to. Might be a different case here.

[wcrowder]

Was playing with loading aliases by removing line 10,000 at a time, largest  file I can get to load on pfSense 2.2, (today's incarnation) using :

pfctl -t pfB_BTLevel1 -T replace -f /var/db/aliastables/pfB_BTLevel1.txt

At 149,405 addresses it worked, any more than 150,000 results in the “bad address” error. When I dropped from 150,000 to 140,000, I worked this out. At 149,405 I received this notice: 9405 addresses added so that = 149,405.

Largest "Block" list usable in pfSense is approximately 150,000 IP's, anything larger will fail regardless what is set in "Firewall Maximum Table Entries" That puts a bunch of publicly available and premium lists out of service.

[eri--]

Set the limit under system advanced by bumping it up.

[Harvy66]

@wcrowder:

Was playing with loading aliases by removing line 10,000 at a time, largest  file I can get to load on pfSense 2.2, (today's incarnation) using :

pfctl -t pfB_BTLevel1 -T replace -f /var/db/aliastables/pfB_BTLevel1.txt

At 149,405 addresses it worked, any more than 150,000 results in the “bad address” error. When I dropped from 150,000 to 140,000, I worked this out. At 149,405 I received this notice: 9405 addresses added so that = 149,405.

Largest "Block" list usable in pfSense is approximately 150,000 IP's, anything larger will fail regardless what is set in "Firewall Maximum Table Entries" That puts a bunch of publicly available and premium lists out of service.

Instead of removing 10k lines at a time, do a binary search. Remove 1/2 of all lines, see if it works, if it doesn't, remove another 1/2. If it does work, add back 1/2 of the 1/2 you removed. Rinse and repeat.

[wcrowder]

@ermal:

Set the limit under system advanced by bumping it up.

I am aware of this setting, it functions correctly on pfSense 2.1.4, does not on 2.2.

These test was both ran tonight with Firewall Maximum Table Entries: 2,000,000

See my update to Bug # 3854 on Redmine showing an easy way test this, it is repeatable. Bug #3854 should not be closed because the knob was not removed, the knob is still there but does not function on 2.2.

This is the same list being updated on 2.2 and then on 2.1.4 on identical VM's, as you can see 2.2 fails, 2.1.4 functions correctly.

[2.2-ALPHA][root@router.crowderfarm.local]/var/db/aliastables(9): pfctl -t pfB_BTLevel1 -T replace -f /var/db/aliastables/pfB_BTLevel1.txt
pfctl: Bad address.

[2.1.4-RELEASE][root@router.crowderfarm.local]/var/db/aliastables(6): pfctl -t pfBlockerP2P -T replace -f /var/db/aliastables/pfB_BTLevel1.txt
3702 addresses added.
33472 addresses deleted.

THIS IS A REPEATABLE REGRESSION…

[wcrowder]

Good idea Harvy66, How 'bout I just spend a week writing a btree in assembly, port that to library and feed that to pfctl in C?…  :o

@Harvy66:

@wcrowder:

Was playing with loading aliases by removing line 10,000 at a time, largest  file I can get to load on pfSense 2.2, (today's incarnation) using :

pfctl -t pfB_BTLevel1 -T replace -f /var/db/aliastables/pfB_BTLevel1.txt

At 149,405 addresses it worked, any more than 150,000 results in the “bad address” error. When I dropped from 150,000 to 140,000, I worked this out. At 149,405 I received this notice: 9405 addresses added so that = 149,405.

Largest "Block" list usable in pfSense is approximately 150,000 IP's, anything larger will fail regardless what is set in "Firewall Maximum Table Entries" That puts a bunch of publicly available and premium lists out of service.

Instead of removing 10k lines at a time, do a binary search. Remove 1/2 of all lines, see if it works, if it doesn't, remove another 1/2. If it does work, add back 1/2 of the 1/2 you removed. Rinse and repeat.