Good results with URL Table Aliases package
-
All,
I've been testing URL Table Aliases for about a week now and it is working well.
Thanks to jimp for putting this together.
FYI, as an example, one can only allow access in a rule to US IP blocks. In that case, a URL might be: http://www.countryipblocks.net/e_country_data/US_cidr.txt
The countryipblocks.net site is a valuable resource for this package. I am sure there are alternatives though.
In order to update these lists, you do need to install the cron package and put something like this in your cron list:
1 0 * * * root /etc/rc.update_urltables
IMPORTANT: It appears you cannot currently use this package with negation rules.
thanks,
Dave -
I like this package too but i've found that there is a limit on the amount of IP addresses :-( Would like to use the full Level1 and China IPs list but it goes over a limit that I think is hard set within pfconf…
Have you tried a really big list yet? Maybe its just my setup
-
It defaults to 100,000 I think, but it can be set higher in pf. I'd just need to figure out the best way to get that done. You just need to add the right command to the pf ruleset:
set limit table-entries 200000
That would set the limit to 200,000. There are some memory considerations for increasing the size, but the pf site (http://www.openbsd.org/faq/pf/options.html) implies that 100,000 is only needed as a default if the system has less than 100MB of RAM.
-
Memory shouldn't be an issue right now, other theb maybe a performance hit.. My box has 2gigs of memory. Is there a way to manually up that limit? I did a search for the pf.conf file and can't seem to find it. Would be a nice addition to the Advanced options under the System meny.. Being able to increase this limit would benefit this package and the IP-Blocklist package(If it gets added to the package manager)
thanks again!!
-
The pf ruleset is generated by the system automatically, so this setting would have to be coded into filter.inc somewhere during the ruleset generation.
If I bothered to do that though I may as well make a GUI field somewhere it could be tuned, though that may just be on 2.0
-
Something to add to the list for 2.0? The feature isn't there now, so I wouldn't mind waiting for 2.0 to have it and be stable. Thanks again!!
-
URL Tables are in 2.0 as-is, but the table size tweak is not (yet).
-
Hello,
Just tried, seems to work very well :)
What kind of list format is supported ?
Do you use some kind of regexp to find ip on each line, whatever the format used ?
Does list such as http://list.iblocklist.com/?list=cn works ?
Also, does list in compressed format (http://…./list.gz) works ?And please, where is the file to put 'set limit table-entries 200000', as rc.conf doesn't seems to be used in pfSense (?). Is it inc/filter.inc.
I don't think starting to modify the php code is a good idea, but it seems it's not possible to change this setting through the web configurator.Thank you.
-
It only supports plain text files with lists of IP addresses or CIDR masked subnets. No compressed files.
The setting to increase the table entries goes in the pf ruleset. I added that as a GUI option in 2.0 but haven't added that to the 1.2.x package yet.
-
Thanks for your reply, but how can i change the pf ruleset ?
Do i need to manually create a pf.conf file ? -
You cannot manually change it. You'd have to add the code to filter.inc.
-
after using pfsense for years now , this is a great addon, which I missed. I feel better to have a blocklist on the firewall beside moblock or iplist on the desktop.
I asked me about the supported Blocklistformat. Jimp mentioned above that only plain text files are supported-what a pitty! Most availible blocklist are compressed and in IP RANGE format with additional information such as nipfilter.dat.gz or level1.gz .
Has the list to be only in CIDR format or Ip range too?
I can download uncompress, convert the list manually and upload to pfsense box by sftp, but to which location?thanks
pete -
-
after using pfsense for years now , this is a great addon, which I missed. I feel better to have a blocklist on the firewall beside moblock or iplist on the desktop.
I asked me about the supported Blocklistformat. Jimp mentioned above that only plain text files are supported-what a pitty! Most availible blocklist are compressed and in IP RANGE format with additional information such as nipfilter.dat.gz or level1.gz .
Has the list to be only in CIDR format or Ip range too?
I can download uncompress, convert the list manually and upload to pfsense box by sftp, but to which location?thanks
peteSomeone else has another package ("ip blocklist") which can work on the level1 list and ranges and such. The author of that package converted it to pf (at my suggestion) and it may be viable for that now. I haven't used it myself yet.
The problem with ranges is they take a lot of time to calculate. IPs and CIDRs are what pf supports natively. Ranges and compressed files have to be processed before they can be used by pf, which increases the burden on the router.
-
@jimp: ok, i modified filter.inc the following way :
$rules .= "set limit table-entries 300000\n";
in the filter_configure_sync function
and it seems to work.
@pete : you can also store the updated list to a local web server, then use the corresponding url into the table alias.Where in the filter_configure_sync section did you put your statement in?
I was thinking of adding it before
$rules .= "\n";
$rules .= "set skip on pfsync0\n"; -
I notice that is package can not be uninstalled and I was wondering what were the reasons and could it possibly be detrimental in anyway.
-
If you uninstall it and leave a URL table alias configured, the behavior would be unpredictable and could result in the filter rules failing to load.
-
I installed it. I installed cron. I added a US CIDR txt url for allowing only us ips. I added the cron job specified on the first post. It does not seem to be working. Using a proxy to test, i could still gain access to pcs behind the firewall.
-
Did you use the alias in a firewall rule? Can you post a screencap of your firewall rules? If it got through, then it either matched a rule above it in the ruleset that passed it, it wasn't used in a rule properly, or the proxy you used was really in the US and not another country.
-
i havent had a chance to check yet, but just to mention I am running NanoBSD version. Does that matter?