Unbound TCP buffer settings not sticky

bryan.paradis

I uninstalled/reinstalled the Unbound pkg last night at 9PM PST. Not sure if Wagonza's changes were a part of the pkg download. The PfSense server downloaded ver _21 again. Launched this morning after disabling the forwarder and all seems well except I couldn't log into forum.pfsense.org. I have in the DNS forwarder host overrides;
pfsense.com=66.219.34.171
files.pfsense.org=66.111.2.167

If you recall adding hostnames was recommended earlier in this thread as a way to recover Unbound from the PfSense server when Unbound won't start.

So I thought well maybe I need to add;
pfsense.org=192.207.126.26

I entered into the forwarder then disabled unbound then enabled the forwarder to populate the changes so Unbound could pick it up from the revised hosts file. Then I reversed the services never having both services enabled at the same time. It's at this time Unbound refused to start again and I had to revert back to the forwarder pointing to another Win box running Unbound.

Is this strike 3?

Why would you start the forwarder again? There has not be something telling us why unbound is not starting. You gave us zero information. ps aux while trying to start unbound? resolver.log? What does it say when you try to start unbound manually? Are you sure DNS Forwarder is off before trying to restart unbound?

markn62

As I mentioned, I started the Forwarder again to populate the hosts that Unbound recognizes. Restarting Unbound didn't pick up the host override entered in the Forwarder. The PfSense docs I read said that Unbound gets it's overrides from the Forwarder entries. Once Unbound quit I had to hurriedly get the system back online. No time to collect ps.aux or logs. I'll schedule this again for 4AM instead of 7AM when I can tolerate more downtime and get what you ask for and when I can reboot the router to perhaps clear port 53 in the event a disabled Forwarder doesn't release the port as it should. I underlined so you don't misunderstand that I know this was why Unbound didn't start. I don't know at this time.

I did say this was an online system right? I won't bother posting again unless I can fully document the results.
Sorry for the bother…

doktornotor

@markn62:

As I mentioned, I started the Forwarder again to populate the hosts that Unbound recognizes. Restarting Unbound didn't pick up the host override entered in the Forwarder.

No such thing needed. Frankly, with all your issues noone's able to reproduce, I'd recommend to flatten and rebuild the box from scratch.

bryan.paradis

@markn62:

As I mentioned, I started the Forwarder again to populate the hosts that Unbound recognizes. Restarting Unbound didn't pick up the host override entered in the Forwarder. The PfSense docs I read said that Unbound gets it's overrides from the Forwarder entries.

You do not need to turn on the DNS Forwarder service to get the host overrides to populate. Unbound reads them from the config.xml everytime you cause the unbound.conf to be generated.

Add overrides
Press save & apply changes
Go to services unbound -> save config and it will regenerate config and restart
Check unbound status tab to see your host overrides as local data records

I mean maybe wagonza can make it regen config on service restart would seem clearer. It has nothing to do with starting dnsmasq it just it shares the page with DNS Forwarder and is read by unbound. Below what I saw following steps above.

# DNSMasq Host overrides
local-data-ptr: "192.168.55.7 test.fudzilla.com"
local-data: "test.fudzilla.com IN A 192.168.55.7"

Pertinent code: Everything in hosts file has a function here and unbound does it when it generates its conf. no need for dnsmasq to be running.

function read_hosts() {

	// Open /etc/hosts and extract the only dhcpleases info
	$etc_hosts = array();
	foreach (file('/etc/hosts') as $line) {
		$d = preg_split('/\s/', $line, -1, PREG_SPLIT_NO_EMPTY);
		if (empty($d) || substr(reset($d), 0, 1) == "#")
			continue;
		if ($d[3] == "#") {
			$ip = array_shift($d);
			$fqdn = array_shift($d);
			$name = array_shift($d);
			if ($fqdn != "empty") {
				if ($name != "empty")
					array_push($etc_hosts, array(ipaddr => "$ip", fqdn => "$fqdn", name => "$name"));
				else
					array_push($etc_hosts, array(ipaddr => "$ip", fqdn => "$fqdn"));
			}
		}
	}
	return $etc_hosts;
}

/*
 * Setup /etc/hosts entries by overriding with local-data
 */
function unbound_add_host_entries() {
	global $config;

	/* XXX: break this out into a separate config file and make use of include */
	$unboundcfg = $config['installedpackages']['unbound']['config'][0];
	$syscfg = $config['system'];
	$dnsmasqcfg = $config['dnsmasq'];

	$unbound_entries = "local-zone: \"{$syscfg['domain']}\" transparent\n";
	// IPv4 entries
	$unbound_entries .= "local-data-ptr: \"127.0.0.1 localhost\"\n";
	$unbound_entries .= "local-data: \"localhost A 127.0.0.1\"\n";
	$unbound_entries .= "local-data: \"localhost.{$syscfg['domain']} A 127.0.0.1\"\n";
	// IPv6 entries
	if(function_exists("is_ipaddrv6")) {
		$unbound_entries .= "local-data-ptr: \"::1 localhost\"\n";
		$unbound_entries .= "local-data: \"localhost AAAA ::1\"\n";
		$unbound_entries .= "local-data: \"localhost.{$syscfg['domain']} AAAA ::1\"\n";
	}

	$added_item_v4 = array();
	$added_item_v6 = array();
	if ($config['interfaces']['lan']) {
		$current_host = $syscfg['hostname'].".".$syscfg['domain'];
		$cfgip = get_interface_ip("lan");
		if (is_ipaddr($cfgip)) {
			$unbound_entries .= "local-data-ptr: \"{$cfgip} {$current_host}\"\n";
			$unbound_entries .= "local-data: \"{$current_host} A {$cfgip}\"\n";
			$unbound_entries .= "local-data: \"{$syscfg['hostname']} A {$cfgip}\"\n";
			$added_item_v4[$current_host] = true;
		}
		$cfgip6 = get_interface_ipv6("lan");
		if (is_ipaddrv6($cfgip6)) {
			$unbound_entries .= "local-data-ptr: \"{$cfgip6} {$current_host}\"\n";
			$unbound_entries .= "local-data: \"{$current_host} AAAA {$cfgip6}\"\n";
			$unbound_entries .= "local-data: \"{$syscfg['hostname']} AAAA {$cfgip6}\"\n";
			$added_item_v6[$current_host] = true;
		}
	} else {
		$sysiflist = get_configured_interface_list();
		foreach ($sysiflist as $sysif) {
			if (!interface_has_gateway($sysif)) {
				$current_host = $syscfg['hostname'].".".$syscfg['domain'];
				$cfgip = get_interface_ip($sysif);
				if (is_ipaddr($cfgip)) {
					$unbound_entries .= "local-data-ptr: \"{$cfgip} {$current_host}\"\n";
					$unbound_entries .= "local-data: \"{$current_host} A {$cfgip}\"\n";
					$unbound_entries .= "local-data: \"{$syscfg['hostname']} A {$cfgip}\"\n";
					$added_item_v4[$current_host] = true;
				}
				$cfgip6 = get_interface_ipv6($sysif);
				if (is_ipaddr($cfgip6)) {
					$unbound_entries .= "local-data-ptr: \"{$cfgip6} {$current_host}\"\n";
					$unbound_entries .= "local-data: \"{$current_host} AAAA {$cfgip6}\"\n";
					$unbound_entries .= "local-data: \"{$syscfg['hostname']} AAAA {$cfgip6}\"\n";
					$added_item_v6[$current_host] = true;
				}
				if (is_ipaddr($cfgip) || is_ipaddr($cfgip6))
					break;
			}
		}
	}

	// DNSMasq entries static host entries
	if (isset($dnsmasqcfg['hosts'])) {
		$hosts = $dnsmasqcfg['hosts'];
		$host_entries = "# DNSMasq Host overrides\n";
		$added_item = array();
		foreach ($hosts as $host) {
			$current_host = ($host['host'] != "") ? $host['host'].".".$host['domain'] : $host['domain'];
			if (function_exists("is_ipaddrv6") && is_ipaddrv6($host['ip'])) {
				if (!$added_item_v6[$current_host]) {
					$host_entries .= "local-data-ptr: \"{$host['ip']} {$current_host}\"\n";
					$host_entries .= "local-data: \"{$current_host} IN AAAA {$host['ip']}\"\n";
					$added_item_v6[$current_host] = true;
				}
				if ($host['aliases']['item'] && is_array($host['aliases']['item']))
					foreach ($host['aliases']['item'] as $alias) {
						$current_alias = ($alias['host'] != "") ? $alias['host'].".".$alias['domain'] : $alias['domain'];
						if (!$added_item_v6[$current_alias]) {
							$host_entries .= "local-data: \"{$current_alias} IN AAAA {$host['ip']}\"\n";
							$added_item_v6[$current_alias] = true;
							if ((!$added_item[$current_alias]) && (!empty($alias['description'])) && ($unboundcfg['txtsupport'] == 'on')) {
								$host_entries .= "local-data: '{$current_alias} TXT \"".addslashes($alias['description'])."\"'\n";
								$added_item[$current_alias] = true;
							}
						}
					}
			} else {
				if (!$added_item_v4[$current_host]) {
					$host_entries .= "local-data-ptr: \"{$host['ip']} {$current_host}\"\n";
					$host_entries .= "local-data: \"{$current_host} IN A {$host['ip']}\"\n";
					$added_item_v4[$current_host] = true;
				}
				if ($host['aliases']['item'] && is_array($host['aliases']['item']))
					foreach ($host['aliases']['item'] as $alias) {
						$current_alias = ($alias['host'] != "") ? $alias['host'].".".$alias['domain'] : $alias['domain'];
						if (!$added_item_v4[$current_alias]) {
							$host_entries .= "local-data: \"{$current_alias} IN A {$host['ip']}\"\n";
							$added_item_v4[$current_alias] = true;
							if ((!$added_item[$current_alias]) && (!empty($alias['description'])) && ($unboundcfg['txtsupport'] == 'on')) {
								$host_entries .= "local-data: '{$current_alias} TXT \"".addslashes($alias['description'])."\"'\n";
								$added_item[$current_alias] = true;
							}
						}
					}
			}
			if ((!$added_item[$current_host]) && (!empty($host['descr'])) && ($unboundcfg['txtsupport'] == 'on')) {
				$host_entries .= "local-data: '{$current_host} TXT \"".addslashes($host['descr'])."\"'\n";
				$added_item[$current_host] = true;
			}
		}
		$unbound_entries .= $host_entries;
	}

	// Static DHCP entries
	$host_entries = "# DHCP Reservations\n";
	if (isset($unboundcfg['regdhcpstatic']) && is_array($config['dhcpd'])) {
		foreach ($config['dhcpd'] as $dhcpif => $dhcpifconf)
			if(is_array($dhcpifconf['staticmap']) && isset($dhcpifconf['enable']))
				foreach ($dhcpifconf['staticmap'] as $host)
					if ($host['ipaddr'] && $host['hostname']) {
						$current_host = $host['hostname'].".".$syscfg['domain'];
						if (!$added_item_v4[$current_host]) {
							$host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$current_host}\"\n";
							$host_entries .= "local-data: \"{$current_host} IN A {$host['ipaddr']}\"\n";
							$added_item_v4[$current_host] = true;
							if ((!$added_item[$current_host]) && (!empty($host['descr'])) && ($unboundcfg['txtsupport'] == 'on')) {
								$host_entries .= "local-data: '{$host['hostname']}.{$syscfg['domain']} TXT \"".addslashes($host['descr'])."\"'\n";
								$added_item[$current_host] = true;
							}
						}
					}
		$unbound_entries .= $host_entries;
	}

	// Static DHCPv6 entries
	$host_entries = "# DHCPv6 reservations\n";
	if (isset($unboundcfg['regdhcpstatic']) && is_array($config['dhcpdv6'])) {
		foreach ($config['dhcpdv6'] as $dhcpif => $dhcpifconf)
			if(is_array($dhcpifconf['staticmap']) && isset($dhcpifconf['enable']))
				foreach ($dhcpifconf['staticmap'] as $host)
					if ($host['ipaddrv6'] && $host['hostname']) {
						$current_host = $host['hostname'].".".$syscfg['domain'];
						if (!$added_item_v6[$current_host]) {
							$host_entries .= "local-data-ptr: \"{$host['ipaddrv6']} {$current_host}\"\n";
							$host_entries .= "local-data: \"{$current_host} IN AAAA {$host['ipaddrv6']}\"\n";
							$added_item_v6[$current_host] = true;
							if ((!$added_item[$current_host]) && (!empty($host['descr'])) && ($unboundcfg['txtsupport'] == 'on')) {
								$host_entries .= "local-data: '{$host['hostname']}.{$syscfg['domain']} TXT \"".addslashes($host['descr'])."\"'\n";
								$added_item[$current_host] = true;
							}
						}
					}
		$unbound_entries .= $host_entries;
	}

	// Handle DHCPLeases added host entries
	$dhcplcfg = read_hosts();
	$host_entries = "# /etc/hosts entries\n";
	if(is_array($dhcplcfg)) {
		foreach($dhcplcfg as $key=>$host) {
			$current_host = $host['fqdn'];
			if (function_exists("is_ipaddrv6") && is_ipaddrv6($host['ipaddr'])) {
				if (!$added_item_v6[$current_host]) {
					$host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$host['fqdn']}\"\n";
					$host_entries .= "local-data: \"{$host['fqdn']} IN AAAA {$host['ipaddr']}\"\n";
					$added_item_v6[$current_host] = true;
					if ((!empty($host['name'])) && (!$added_item_v6[$host['name']])) {
						$host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$host['name']}\"\n";
						$host_entries .= "local-data: \"{$host['name']} IN AAAA {$host['ipaddr']}\"\n";
						$added_item_v6[$host['name']] = true;
					}
				}
			} else {
				if (!$added_item_v4[$current_host]) {
					$host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$host['fqdn']}\"\n";
					$host_entries .= "local-data: \"{$host['fqdn']} IN A {$host['ipaddr']}\"\n";
					$added_item_v4[$current_host] = true;
					if ((!empty($host['name'])) && (!$added_item_v4[$host['name']])) {
						$host_entries .= "local-data-ptr: \"{$host['ipaddr']} {$host['name']}\"\n";
						$host_entries .= "local-data: \"{$host['name']} IN A {$host['ipaddr']}\"\n";
						$added_item_v4[$host['name']] = true;
					}
				}
			}
		}
		$unbound_entries .= $host_entries;
	}

	return $unbound_entries;
}

/* Setup any domain overrides that have been configured with stub-zone parameter
 */
function unbound_add_domain_overrides($pvt=false) {
	global $config;

	if (isset($config['dnsmasq']['domainoverrides'])) {
		$domains = $config['dnsmasq']['domainoverrides'];

		// Domain overrides that have multiple entries need multiple stub-addr: added
		$sorted_domains = msort($domains, "domain");
		$result = array();
		foreach($sorted_domains as $domain) {
			$domain_key = current($domain);
			if (!isset($result[$domain_key]))
				$result[$domain_key] = array();
			$result[$domain_key][] = $domain['ip'];
		}

		$domain_entries = "";
		foreach($result as $domain=>$ips) {
			if ($pvt == true) {
				if (strpos($domain, "in-addr.arpa") !== false)
					$domain_entries .= "local-zone: \"$domain\" transparent\n";
				else
					$domain_entries .= "private-domain: \"$domain\"\n";
				if (isset($config['installedpackages']['unbound']['config'][0]['dnssec_status']))
					$domain_entries .= "domain-insecure: \"$domain\"\n";
			} else {
				$domain_entries .= "stub-zone:\n";
				$domain_entries .= "\tname: \"$domain\"\n";
				foreach($ips as $ip)
					$domain_entries .= "\tstub-addr: $ip\n";
				$domain_entries .= "\tstub-prime: no\n";
			}
		}
		return $domain_entries;
	}
}

@markn62:

Once Unbound quit I had to hurriedly get the system back online. No time to collect ps.aux or logs. I'll schedule this again for 4AM instead of 7AM when I can tolerate more downtime and get what you ask for and when I can reboot the router to perhaps clear port 53 in the event a disabled Forwarder doesn't release the port as it should. I underlined so you don't misunderstand that I know this was why Unbound didn't start. I don't know at this time.

Yes it is probably the reason. I understand circumstances but having a second SSH open or third to launch a couple commands wouldn't hurt in the future as you are the one that needs the information after all ;)

@markn62:

I did say this was an online system right? I won't bother posting again unless I can fully document the results.
Sorry for the bother…

I understand production. No hostility meant. It isn't the clearest process or setup in the GUI ever. That is the problem with abstracting things into a GUI sometimes stuff gets meshed together and without looking at code you have no idea what is going on.

@doktornotor:

No such thing needed. Frankly, with all your issues noone's able to reproduce, I'd recommend to flatten and rebuild the box from scratch.

Don't think there is a need for this. I think it is just a misunderstanding. Think we have about everything covered now. Wagonza has fixed the conf generation clause order problem for custom options and probably this current problem as markn62 has noted that it was probably something to do with DNS Forwarder still running or something.

wagonza

@markn62:

I uninstalled/reinstalled the Unbound pkg last night at 9PM PST. Not sure if Wagonza's changes were a part of the pkg download. The PfSense server downloaded ver _21 again.

Nope not yet.

@markn62:

Launched this morning after disabling the forwarder and all seems well except I couldn't log into forum.pfsense.org. I have in the DNS forwarder host overrides;
pfsense.com=66.219.34.171
files.pfsense.org=66.111.2.167

If you recall adding hostnames was recommended earlier in this thread as a way to recover Unbound from the PfSense server when Unbound won't start.

So I thought well maybe I need to add;
pfsense.org=192.207.126.26

I entered into the forwarder then disabled unbound then enabled the forwarder to populate the changes so Unbound could pick it up from the revised hosts file. Then I reversed the services never having both services enabled at the same time. It's at this time Unbound refused to start again and I had to revert back to the forwarder pointing to another Win box running Unbound.

Is this strike 3?

You shouldn't need to do any of the above.

wagonza

@bryan.paradis:

I mean maybe wagonza can make it regen config on service restart would seem clearer. It has nothing to do with starting dnsmasq it just it shares the page with DNS Forwarder and is

Only gonna happen in pfSense2.2.

bryan.paradis

@wagonza:

@bryan.paradis:

I mean maybe wagonza can make it regen config on service restart would seem clearer. It has nothing to do with starting dnsmasq it just it shares the page with DNS Forwarder and is

Only gonna happen in pfSense2.2.

:) fair enough as it is getting integrated in.

markn62

Well I reinstalled unbound again @ 4AM. Ran fine initially so I tried the option command "outgoing-range: 900" and it killed the service. I thought Wagonza put this into the pkg. I removed that and normal function returned.

So I went back to bed until later this morning. Complaints of *.org, *.info and *.co resolution started rolling in around 9AM. Narrowed it down to the incoming/outgoing buffers. When set to zero, .com, .net resolves but the others mentioned above do not. So it appears these two parameters may not be doing what they're suppose to do, or doing more than is apparent in the GUI. Suppose to ignore TCP and only use UDP.

Doesn't make sense to me that with UDP only that DNS resolution would be selective by domain.

markn62

@wagonza:

@markn62 was this problem fixed for you? or is it still an issue?

Not much has changed. Seems that the package operates very much like it did since the question above was asked. The only change I noticed was a few more "do not's" that are important to adhere too for the package to work properly. I guess it is what it is.

bryan.paradis

@markn62:

@wagonza:

@markn62 was this problem fixed for you? or is it still an issue?

Not much has changed. Seems that the package operates very much like it did since the question above was asked. The only change I noticed was a few more "do not's" that are important to adhere too for the package to work properly. I guess it is what it is.

I can't reproduce any of your current issues at all with the package even using all your custom options. Something else must be going on or you have something else as an additional problem. Try saving your whole pfsense configuration and loading that config onto a VM ands testing to give us information to go on. Or even for yourself. The devs are replacing dnsmasq with unbound afterall. If you copy your unbound.conf on your seperate box to your pfsense and only restart the service does it work?