PfBlocker

ccb056 · Dec 31, 2012, 7:50 PM

Upon investigating another error I was experiencing, see here:
http://forum.pfsense.org/index.php/topic,57113.0.html

I switched from Chrome to IE, in IE the 500 error is gone and the dashboard widget works as expected.

Maybe the pfBlocker and Traffic Graph bug are related….. both show errors in Chrome but work in IE

Supermule · Jan 4, 2013, 7:46 PM

I get this error

php: : There were error(s) loading the rules: /tmp/rules.debug:39: cannot define table pfBlockerEmerging_Block_IP: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [39]: table <pfblockeremerging_block_ip>persist file "/var/db/aliastables/pfBlockerEmerging_Block_IP.txt"

I have deleted the file, but it still shows up on the pfblocker widget on the frontpage of PFsense.

It keeps giving me errors of the list is gone, but before I deleted it, it was still there.

Very weird problem…</pfblockeremerging_block_ip>

Supermule · Jan 4, 2013, 8:18 PM

Cannot remove lists from pfblocker widget….

How to do?? They dont go away when deleted from the lists tab in the config GUI.

Supermule · Jan 4, 2013, 8:38 PM

Deleted and reinstalled. Lists are gone.

Supermule · Jan 4, 2013, 9:01 PM

Did a package delete and rebooted….

Jan 4 22:05:27 php: : There were error(s) loading the rules: /tmp/rules.debug:35: cannot define table pfBlockerTopSpammers: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [35]: table <pfblockertopspammers>persist file "/var/db/aliastables/pfBlockerTopSpammers.txt"

Jan 4 22:05:27 php: : New alert found: There were error(s) loading the rules: /tmp/rules.debug:35: cannot define table pfBlockerTopSpammers: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded The line in question reads [35]: table <pfblockertopspammers>persist file "/var/db/aliastables/pfBlockerTopSpammers.txt"

I think it has issues with 2.0.2 release…..</pfblockertopspammers></pfblockertopspammers>

Supermule · Jan 4, 2013, 9:13 PM

Did a manual delete in rules.debug to remove pfblocker aliases.

Now it says this…

Jan 4 22:16:46 php: : There were error(s) loading the rules: /tmp/rules.debug:35: cannot define table pfBlockerTopSpammers: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [35]: table <pfblockertopspammers>persist file "/var/db/aliastables/pfBlockerTopSpammers.txt"

Jan 4 22:16:46 php: : New alert found: There were error(s) loading the rules: /tmp/rules.debug:35: cannot define table pfBlockerTopSpammers: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded The line in question reads [35]: table <pfblockertopspammers>persist file "/var/db/aliastables/pfBlockerTopSpammers.txt"

It just wont go away!</pfblockertopspammers></pfblockertopspammers>

kilthro · Jan 5, 2013, 10:15 AM

Looks like memory allocation errors.

Did you go to system > Advanced > Firewall/Nat tab and increase the Firewall Maximum Table Entries?

I have had to increase mine quite a bit to run all my tables. I currently have mine set to 999999999 and no longer get any memory allocation errors.

Supermule · Jan 5, 2013, 11:10 AM

Its currently set at 200.000 table entries…and 100.000 tables...

kilthro · Jan 5, 2013, 11:16 AM

increase it way up. Thats most likely the problem. Its not going to affect the firewall negatively. You see where mine is set and all runs just fine and I dont get those errors anymore.

Supermule · Jan 5, 2013, 11:18 AM

Whats your maximum tables and table entries??

kilthro · Jan 5, 2013, 11:29 AM

Attached everything on that page. I didnt touch anything else. Left it at default.

Supermule · Jan 5, 2013, 11:59 AM

Thx!

francisuk22 · Jan 12, 2013, 10:26 PM

Love it! :)

LinuxTracker · Jan 21, 2013, 7:48 PM

I have pfBlocker loading a list from the following url:
http://www.dnsbl.manitu.net/download/nixspam-ip.dump.gz

I decompressed the .gz to look at it and found text in the following format:

2013-01-21T20:15+0100 78.39.223.180
2013-01-21T20:15+0100 49.202.135.218
2013-01-21T20:15+0100 213.108.75.31
...

I have 2 systems that handle this list fine.
One other system scrambles the file content - loads nonsense IPs the table - is why I peeked inside the .gz

The thing is, I thought pfBlocker had to have the IP addr/cidrs at the beginning of the line. Looks like I'm wrong.

In what format(s) does pfBlocker need it's IP data?

marcelloc · Jan 22, 2013, 12:40 AM

@LinuxTracker:

In what format(s) does pfBlocker need it's IP data?

It loks for ip addresses using preg_match, if you can edit it to remove ip date, it may help.

LinuxTracker · Jan 22, 2013, 8:48 PM

@marcelloc:

It looks for ip addresses using preg_match, if you can edit it to remove ip date, it may help.

The list loads fine on two systems; now I know why.

The system that scrambles the data has 27 lists loaded.
It's probably just reacting to all the abuse I heap upon it.

As always, thank you for your insight.

Gradius · Feb 1, 2013, 8:39 PM

The real main issue is even on last update 2.1-BETA1 (today), pfblocker still doesn't comes up as ENABLED.

Everytime after update I need to go there and manually enable myself. ::)

marcelloc · Feb 1, 2013, 8:49 PM

@Gradius:

Everytime after update I need to go there and manually enable myself. ::)

Pfblocker uninstall script will always disable the service to avoid aliases/rules issues.

It's a feature, not an issue ;)

asbirim · Feb 9, 2013, 10:20 PM

Hi, I added support to insert portlist file. this way peoples can customize installations with pre defined rules from their web sites.

you can deploy port list file with adding #PORT_LIST_PF text to begin of file.

i hope you will include this changes to next release and it will helpful to others.

example port list file: safeports.txt


#PORT_LIST_PF
21#ftp
22#ssh
25#smtp
53#dns
67#dhcp
80#http
110#pop
443#https
587#new smtp
1433#sql server
3306#mysql
3389#rdp

replace bellow code in your /usr/local/pkg/pfblocker.inc

add this function to after pfblocker_Range2CIDR function


function pfblocker_Range2Ports($port_min, $port_max) {
	#function called without any args
	if ($port_min == "" || $port_max == "")
		return "";
	#function called with same ip in min and max
	if ($port_min == $port_max)
		return $port_min;
	$bas=intval($port_min);
	$start_port+=1;//skip first value because its already processed
	$res="";
	$end_port=intval($port_max);
	for($i=$start_port;$i<=$end_port;$i++) {
		$res.="$i\n";
	}
	if($res!="") 
		return $res;
}

changes begins at "#print $list['aliasname'].$list['action']." ".$alias." ".$row['url']."
";"


#print $list['aliasname'].$list['action']." ".$alias." ".$row['url']."
";
			if ($alias != "pfBlocker" && $list['action'] != "" && $list['action'] != 'Disabled' && $pfblocker_enable == "on") {
				$isPortList=FALSE;
				#remove empty lists files if any
				if (is_array($list['row']))
				foreach ($list['row'] as $row) {
					#print $list['aliasname'].$list['action'].$list['cron']." ".$alias." ".$row['url']."$update_local
";
					if ($row['url'] != "") {
						$md5_url = md5($row['url']);
						if (file_exists($pfbdir."/".$md5_url.".txt")) {
							$f=file($pfbdir.'/'.$md5_url.'.txt');
							${$alias}.= file_get_contents($pfbdir.'/'.$md5_url.'.txt');
							if ($f[0]=="#PORT_LIST_PF\n") {
								$new_file=${$alias};
								$new_file=str_replace("#PORT_LIST_PF\n","",$new_file);
								$new_file=str_replace("#PORT_LIST_PF ","",$new_file);
								$isPortList=TRUE;
							}
						}
						else {
							if ($row['format'] == "gz")
								$url_list= gzfile($row['url']);
							else
								$url_list= file($row['url']);
							#extract range lists
							$new_file="";

							if (is_array($url_list)) {
								if ($url_list[0]=="#PORT_LIST_PF\n") {
									$isPortList=TRUE;
									$new_file="##PORT_LIST_PF\n";

									foreach ($url_list as $line) {

										# port format 80
										if (preg_match("/(\d+)/",$line,$matches)){
											${$alias}.= $matches[1]."\n";
											$new_file.= $matches[1]."\n";
										}

										# Port range 20-23

										if (preg_match("/(\d+)-(\d+)/",$line,$matches)){
											$plist = pfblocker_Range2Ports($matches[1],$matches[2]);
											if ($plist != ""){
												${$alias}.= $plist;
												$new_file.= $plist;
											}
										}
									}

								}
								else {
									foreach ($url_list as $line) {

										# CIDR format 192.168.0.0/16
										if (preg_match("/(\d+\.\d+\.\d+\.\d+\/\d+)/",$line,$matches)){
										${$alias}.= $matches[1]."\n";
										$new_file.= $matches[1]."\n";
										}
										# Single ip addresses 
										if (preg_match("/(\d+\.\d+\.\d+\.\d+)\s+/",$line,$matches)){
										${$alias}.= $matches[1]."/32\n";
										$new_file.= $matches[1]."/32\n";
										}
										# Network range 192.168.0.0-192.168.0.254
										if (preg_match("/(\d+\.\d+\.\d+\.\d+)-(\d+\.\d+\.\d+\.\d+)/",$line,$matches)){
											$cidr= pfblocker_Range2CIDR($matches[1],$matches[2]);
											if ($cidr != ""){
												${$alias}.= $cidr."\n";
												$new_file.= $cidr."\n";
											}
										}

									}
								}
							}
							if ($new_file != "")
								file_put_contents($pfbdir.'/'.$md5_url.'.txt',$new_file, LOCK_EX);
						}
					}
				}

				//if($isPortList==TRUE)
				//	echo "$alias port list $new_file";

				#check custom network list
				if (pfb_text_area_decode($list['custom']) != "")
					${$alias}.=pfb_text_area_decode($list['custom'])."\n";
				#save alias file if not empty
				if (${$alias} == ""){
					unlink_if_exists($pfb_alias_dir.'/'.$alias.'.txt');
					}
				else{
					file_put_contents($pfb_alias_dir.'/'.$alias.'.txt',${$alias}, LOCK_EX);
					file_put_contents($pfsense_alias_dir.'/'.$alias.'.txt',${$alias}, LOCK_EX);
					#create alias
					if($isPortList==TRUE) {
						$new_file=str_replace("#PORT_LIST_PF\n","",$new_file);
						$new_file=str_replace("#PORT_LIST_PF ","",$new_file);
						$new_file=str_replace("\n"," ",$new_file);
						$new_file=trim($new_file);
						$new_aliases_list[]=$alias;
						$new_aliases[]=array("name"=> $alias,
									 "url"=> "",
									 "updatefreq"=> "32",
									 "address"=>"$new_file",
									 "descr"=> "pfBlocker user list",
									 "type"=> "port",
									 "detail"=> "DO NOT EDIT THIS ALIAS");
					}
					else {
						$new_aliases_list[]=$alias;
						$new_aliases[]=array("name"=> $alias,
									 "url"=> $web_local.'?pfb='.$alias,
									 "updatefreq"=> "32",
									 "address"=>"",
									 "descr"=> "pfBlocker user list",
									 "type"=> "urltable",
									 "detail"=> "DO NOT EDIT THIS ALIAS");
						#Create rule if action permits
						switch($list['action']){
							case "Deny_Both":
								$rule = $base_rule;
								$rule["type"] = $deny_action_inbound;
								$rule["descr"]= "$alias auto rule";
								$rule["source"]= array("address"=> $alias);
								$rule["destination"]=array("any"=>"");
								if ($pfblocker_config['enable_log'])
									$rule["log"]="";
								$deny_inbound[]=$rule;
							case "Deny_Outbound":
								$rule = $base_rule;
								$rule["type"] = $deny_action_outbound;
								$rule["descr"]= "$alias auto rule";
								$rule["source"]=array("any"=>"");
								$rule["destination"]= array("address"=> $alias);
								if ($pfblocker_config['enable_log'])
									$rule["log"]="";
								$deny_outbound[]=$rule;					
								break;
							case "Deny_Inbound":
								$rule = $base_rule;
								$rule["type"] = $deny_action_inbound;
								$rule["descr"]= "$alias auto rule";
								$rule["source"]= array("address"=> $alias);
								$rule["destination"]=array("any"=>"");
								if ($pfblocker_config['enable_log'])
									$rule["log"]="";
								$deny_inbound[]=$rule;					
								break;
							case "Permit_Outbound":
								$rule = $base_rule;
								$rule["type"] = "pass";
								$rule["descr"]= "$alias auto rule";
								$rule["source"]=array("any"=>"");
								$rule["destination"]= array("address"=> $alias);
								if ($pfblocker_config['enable_log'])
									$rule["log"]="";
								$permit_outbound[]=$rule;					
								break;
							case "Permit_Inbound":
								$rule = $base_rule;
								$rule["type"] = "pass";
								$rule["descr"]= "$alias auto rule";
								$rule["source"]= array("address"=> $alias);
								$rule["destination"]=array("any"=>"");
								if ($pfblocker_config['enable_log'])
										$rule["log"]="";
								$permit_inbound[]=$rule;					
								break;
						}
					}
				}
			#mark pfctl aliastable for cleanup
			if (!in_array($alias, $aliases_list)) 
				$aliases_list[]=$alias;
			}
		else{
			#unlink previous pfblocker alias list if any
			unlink_if_exists($pfb_alias_dir.'/'.$alias.'.txt');
			}
		}
		#update pfsense alias table

end at "#update pfsense alias table"

Supermule · Feb 10, 2013, 5:23 AM

So that way Pfblocker only works on specified ports and not on all??