Unbound TCP buffer settings not sticky



  • I occasionally get the error "unbound [23056:0] error: read (in tcp:s): Operation timed out" which got me looking at this more closely.  I thought perhaps if I used UDP "only" this error wouldn't occur.

    What I found was the Unbound package GUI appears to have an issue with regard to incoming and outgoing TCP buffer settings. The GUI guidance is that if set to zero then no TCP queries will be sent to the authoritative server.  I set both to zero and the /usr/pbi/unbound-amd64/etc/unbound/unbound.conf still reports a default value of 10 on both even after restarting the Unbound service.

    Any reason the GUI settings don't modify the .conf file?





  • @markn62 was this problem fixed for you? or is it still an issue?



  • @markn62 - ive made some changes which should make these options stick. Let me know if it works for you.



  • Presume I would need to apply a snapshot.  Is the snapshot dated today?



  • Nope - just update your Unbound package.



  • Thanks I'll check and report back.


  • Banned

    I guess something got broken here on the way? On trying to save advanced settings I get

    
    The following input errors were detected:
    
    You must enter a valid number in 'TTL for lame delegation'.
    
    


  • Fixed. Reinstall :/


  • Banned

    @wagonza:

    Reinstall :/

    Hehe, I'll cron an hourly reinstall I guess. :D

    Fixed, thanks again!  8)



  • @doktornotor:

    Hehe, I'll cron an hourly reinstall I guess. :D

    Ha! Cool thanks for the feedback.



  • Upgraded the service watchdog package fine.  Then went onto the unbound upgrade and it failed.  Now I get "Unable to retrieve package info from www.pfsense.com. Cached data will be used" which doesn't get me anywhere.



  • First attempt at a package upgrade failed.  Had to reboot pfsense then the unbound version 1.4.21_3 package would install. Go figure.

    In the GUI I changed both "incoming TCP buffers" and "outgoing TCP buffers" from 10 to zero and /usr/pbi/unbound-amd64/etc/unbound/unbound.conf reports;

    outgoing-num-tcp: 10
    incoming-num-tcp: 10

    So no change that I can see.



  • Thanks for the feedback. Found the problem and fixed.



  • When do you plan to release the fix?



  • Its released so you just need to reinstall the package. I didnt bump the version.



  • Ahhh, didn't know I could PKG from the "installed packages" tab.  I'll report back by tomorrow.



  • After installing unbound 1.4.21_3 none of the custom options i am using seem to work anymore.  The error is

    php: config.inc: The command '/usr/pbi/unbound-amd64/sbin/unbound-control start' returned exit code '1', the output was '/usr/pbi/unbound-amd64/etc/unbound/unbound.conf:104: error: syntax error read /usr/pbi/unbound-amd64/etc/unbound/unbound.conf failed: 1 errors in configuration file [1393437916] unbound[79295:0] fatal error: Could not read config file: /usr/pbi/unbound-amd64/etc/unbound/unbound.conf'

    When i went into the file it starts on the first line of the custom options, if i remove all the options i have it will boot fine.

    "msg-cache-slabs: 1;rrset-cache-slabs: 1;infra-cache-slabs: 1;key-cache-slabs: 1;outgoing-range: 950;val-clean-additional: yes;harden-glue: yes;do-not-query-localhost: yes;do-ip6: no;use-caps-for-id: no;hide-identity: yes;hide-version: yes"

    the custom options i was using.  I even tried using them one at a time and it still give me the errors.



  • @shinzo:

    After installing unbound 1.4.21_3 none of the custom options i am using seem to work anymore.  The error is

    php: config.inc: The command '/usr/pbi/unbound-amd64/sbin/unbound-control start' returned exit code '1', the output was '/usr/pbi/unbound-amd64/etc/unbound/unbound.conf:104: error: syntax error read /usr/pbi/unbound-amd64/etc/unbound/unbound.conf failed: 1 errors in configuration file [1393437916] unbound[79295:0] fatal error: Could not read config file: /usr/pbi/unbound-amd64/etc/unbound/unbound.conf'

    When i went into the file it starts on the first line of the custom options, if i remove all the options i have it will boot fine.

    "msg-cache-slabs: 1;rrset-cache-slabs: 1;infra-cache-slabs: 1;key-cache-slabs: 1;outgoing-range: 950;val-clean-additional: yes;harden-glue: yes;do-not-query-localhost: yes;do-ip6: no;use-caps-for-id: no;hide-identity: yes;hide-version: yes"

    the custom options i was using.  I even tried using them one at a time and it still give me the errors.

    The options in the configuration file should be like below

    
    option: blah
    option: blah
    option: blah
    
    

    the gui should take the custom options then split them by the delimiter ; then it puts them on a new line for each option

    	# Handle custom options
    	if (!empty($adv_config['custom_options'])) {
    		$custom_options = explode(";", ($adv_config['custom_options']));
    		$unbound_conf .= "\n# Unbound Custom options\n";
    		foreach ($custom_options as $ent)
    			$unbound_conf .= $ent."\n";
    
    

    Is that the line you see exactly in your config file? What are you putting in the advanced options form in the gui?



  • @bryan.paradis:

    @shinzo:

    After installing unbound 1.4.21_3 none of the custom options i am using seem to work anymore.  The error is

    php: config.inc: The command '/usr/pbi/unbound-amd64/sbin/unbound-control start' returned exit code '1', the output was '/usr/pbi/unbound-amd64/etc/unbound/unbound.conf:104: error: syntax error read /usr/pbi/unbound-amd64/etc/unbound/unbound.conf failed: 1 errors in configuration file [1393437916] unbound[79295:0] fatal error: Could not read config file: /usr/pbi/unbound-amd64/etc/unbound/unbound.conf'

    When i went into the file it starts on the first line of the custom options, if i remove all the options i have it will boot fine.

    "msg-cache-slabs: 1;rrset-cache-slabs: 1;infra-cache-slabs: 1;key-cache-slabs: 1;outgoing-range: 950;val-clean-additional: yes;harden-glue: yes;do-not-query-localhost: yes;do-ip6: no;use-caps-for-id: no;hide-identity: yes;hide-version: yes"

    the custom options i was using.  I even tried using them one at a time and it still give me the errors.

    The options in the configuration file should be like below

    
    option: blah
    option: blah
    option: blah
    
    

    the gui should take the custom options then split them by the delimiter ; then it puts them on a new line for each option

    	# Handle custom options
    	if (!empty($adv_config['custom_options'])) {
    		$custom_options = explode(";", ($adv_config['custom_options']));
    		$unbound_conf .= "\n# Unbound Custom options\n";
    		foreach ($custom_options as $ent)
    			$unbound_conf .= $ent."\n";
    
    

    Is that the line you see exactly in your config file? What are you putting in the advanced options form in the gui?

    I looked over the code and it is the same.  In the advanced options in the gui i have

    "msg-cache-slabs: 1;rrset-cache-slabs: 1;infra-cache-slabs: 1;key-cache-slabs: 1;outgoing-range: 950;val-clean-additional: yes;harden-glue: yes;do-not-query-localhost: yes;do-ip6: no;use-caps-for-id: no;hide-identity: yes;hide-version: yes"  as the semi colon is used to make a new line.

    when i go into the config file, it is being set correctly.

    Unbound Custom options

    msg-cache-slabs: 1
    rrset-cache-slabs: 1
    infra-cache-slabs: 1
    key-cache-slabs: 1
    outgoing-range: 950
    val-clean-additional: yes
    harden-glue: yes
    do-not-query-localhost: yes
    do-ip6: no
    use-caps-for-id: no
    hide-identity: yes
    hide-version: yes



  • @Shinzo,
    What version package were you running prior?  I can tell you outgoing-range as a custom setting, along with a few select others, has never worked in 2.0 or 2.1, see my post @ https://forum.pfsense.org/index.php/topic,64897.msg352131.html#msg352131 .  I had to strip my custom options down to only forward zone and addresses.  No way to detune unbound.  So you cant really troubleshoot the package by probable trial'n'error if it acts quirky.  Unbound reboots by itself a few times a day on my 64-bit install.  And I've never read anywhere until today that the word "option:" needs to be placed before every custom option entry if I read this thread correctly.  Where is this documented?  I don't use them in my options and the ones I have entered show in the unbound.conf fine.  Can't say if, for example, "option: outgoing-range: 950" would be accepted by unbound but imagine it would get hung up on the pair of colon syntax, although I haven't tried it.



  • @markn62:

    @Shinzo,
    What version package were you running prior?  I can tell you outgoing-range as a custom setting, along with a few select others, has never worked in 2.0 or 2.1, see my post @ https://forum.pfsense.org/index.php/topic,64897.msg352131.html#msg352131 .  I had to strip my custom options down to only forward zone and addresses.  No way to detune unbound.  So you cant really troubleshoot the package by probable trial'n'error if it acts quirky.  Unbound reboots by itself a few times a day on my 64-bit install.  And I've never read anywhere until today that the word "option:" needs to be placed before every custom option entry if I read this thread correctly.  Where is this documented?  I don't use them in my options and the ones I have entered show in the unbound.conf fine.  Can't say if, for example, "option: outgoing-range: 950" would be accepted by unbound but imagine it would get hung up on the pair of colon syntax, although I haven't tried it.

    The "option: blah"  was used as a example of how the output should be.  As i continue to troubleshoot my issue, i removed the 4 previous lines which had to do with the forward-zones and got it up and running, il keep looking

    oh and i was using version 1.4.21_2 and it was on a 2.1.1 box



  • FILE FORMAT
          There  must be whitespace between keywords. Attribute keywords end with
          a colon ':'. An attribute is followed by its containing attributes,  or
          a value.

    Files  can be included using the include: directive. It can appear any-
          where, it accepts a single file name as argument.  Processing continues
          as  if  the text from the included file was copied into the config file
          at that point.  If also using chroot, using full path  names  for  the
          included files works, relative pathnames for the included names work if
          the directory where the daemon is  started  equals  its chroot/working
          directory.  Wildcards  can  be used  to  include  multiple files, see
          glob(7).

    Server Options
          These options are part of the server: clause.

    Looks like the configuration file should be laid out in clause sections full of options as below:

    
    server:
    option: blah
    
    remote-control:
    option: blah
    
    stub-zone:
    option: blah
    
    forward-zone:
    option: blah
    
    

    but it seems that the custom server options are being added after the forward-zone clause. I have moved the code in the unbound.inc file up above the foward-zone code and attached it. Try backing up the unbound.inc and placing the one I have attached in its place.

    unbound.inc.txt



  • #########################

    Unbound configuration

    #########################

    Server config

    server:
    chroot: ""
    username: "unbound"
    directory: "/usr/pbi/unbound-amd64/etc/unbound"
    pidfile: "/var/run/unbound.pid"
    harden-referral-path: no
    prefetch: no
    prefetch-key: no
    use-syslog: yes
    port: 53
    verbosity: 0
    do-ip4: yes
    do-ip6: yes
    do-udp: yes
    do-tcp: yes
    do-daemonize: yes
    module-config: "validator iterator"
    unwanted-reply-threshold: 5000000
    num-queries-per-thread: 512
    jostle-timeout: 200
    infra-host-ttl: 900
    prefetch: no
    prefetch-key: no
    infra-cache-numhosts: 15000
    outgoing-num-tcp: 10
    incoming-num-tcp: 10
    edns-buffer-size: 4096
    statistics-interval: 0
    extended-statistics: no
    statistics-cumulative: no
    cache-max-ttl: 86400
    cache-min-ttl: 0
    harden-dnssec-stripped: yes
    hide-identity: yes
    hide-version: yes
    harden-glue: yes
    num-threads: 4
    msg-cache-slabs: 4
    rrset-cache-slabs: 4
    infra-cache-slabs: 4
    key-cache-slabs: 4
    msg-cache-size: 10m
    rrset-cache-size: 20m
    outgoing-range: 8192
    #so-rcvbuf: 4m
    #so-sndbuf: 4m

    Interface IP(s) to bind to

    interface: 192.168.1.1

    Interfaces to query from

    outgoing-interface: 192.168.1.1

    auto-trust-anchor-file: /usr/pbi/unbound-amd64/etc/unbound/root-trust-anchor

    Access Control

    Local attached networks allowed to utilize service and any user added ACLs

    access-control: 127.0.0.0/8 allow
    access-control: ::1 allow
    access-control: 192.168.1.0/27 allow

    For DNS Rebinding prevention

    private-address: 10.0.0.0/8
    private-address: 172.16.0.0/12
    private-address: 192.168.0.0/16
    private-address: 169.254.0.0/16
    private-address: fd00::/8
    private-address: fe80::/10

    Set private domains in case authorative name server returns a RFC1918 IP address

    Host entries

    local-zone: "lan" transparent
    local-data-ptr: "127.0.0.1 localhost"
    local-data: "localhost A 127.0.0.1"
    local-data: "localhost.lan A 127.0.0.1"
    local-data-ptr: "::1 localhost"
    local-data: "localhost AAAA ::1"
    local-data: "localhost.lan AAAA ::1"
    local-data-ptr: "192.168.1.1 peanuts.lan"
    local-data: "peanuts.lan A 192.168.1.1"
    local-data: "peanuts A 192.168.1.1"

    DHCP Reservations

    /etc/hosts entries

    Domain overrides

    #forward-zone:
    #name: "."
    #forward-addr: 75.75.75.75
    #forward-addr: 75.75.76.76

    Unbound Custom options

    msg-cache-slabs: 1
    rrset-cache-slabs: 1
    infra-cache-slabs: 1
    key-cache-slabs: 1
    outgoing-range: 950
    val-clean-additional: yes
    harden-glue: yes
    do-not-query-localhost: yes
    do-ip6: no
    use-caps-for-id: no
    hide-identity: yes
    hide-version: yes

    Remote Control Config

    remote-control:
    control-enable: yes
    control-interface: 127.0.0.1
    control-port: 953
    server-key-file: "/usr/pbi/unbound-amd64/etc/unbound/unbound_server.key"
    server-cert-file: "/usr/pbi/unbound-amd64/etc/unbound/unbound_server.pem"
    control-key-file: "/usr/pbi/unbound-amd64/etc/unbound/unbound_control.key"
    control-cert-file: "/usr/pbi/unbound-amd64/etc/unbound/unbound_control.pem"

    thats how my file looks like when its working.  Some of those options are doubled because they weren't there before or i didn't want to manually change the values every time the package was updated.  If i put them in the custom options window, then it will overwrite the default value to what ever i wanted since the custom options are read last

    thats the output in the status tab when its running.  or do you want the config file?



  • @shinzo:

    #########################

    Unbound configuration

    #########################

    snip

    thats how my file looks like when its working.  Some of those options are doubled because they weren't there before or i didn't want to manually change the values every time the package was updated.  If i put them in the custom options window, then it will overwrite the default value to what ever i wanted since the custom options are read last

    Check my update attachement. I think it is because the custom options are getting put in after the server: clause has been finished because the forward-zones clause has started. Can you test it out to see if it works properly with my changes?



  • sure just need a minute



  • @bryan.paradis:

    @shinzo:

    #########################

    Unbound configuration

    #########################

    snip

    thats how my file looks like when its working.  Some of those options are doubled because they weren't there before or i didn't want to manually change the values every time the package was updated.  If i put them in the custom options window, then it will overwrite the default value to what ever i wanted since the custom options are read last

    Check my update attachement. I think it is because the custom options are getting put in after the server: clause has been finished because the forward-zones clause has started. Can you test it out to see if it works properly with my changes?

    yeah that did fix it.

    it looks like

    Unbound Custom options

    msg-cache-slabs: 1
    rrset-cache-slabs: 1
    infra-cache-slabs: 1
    key-cache-slabs: 1
    val-clean-additional: yes
    harden-glue: yes
    do-not-query-localhost: yes
    do-ip6: no
    use-caps-for-id: no
    hide-identity: yes
    hide-version: yes
    forward-zone:
    name: "."
    forward-addr: 75.75.75.75
    forward-addr: 75.75.76.76



  • So all the options work correctly now?



  • I've never had an issue with forward zones taking.  Bryan, where did you get the syntax document?  Is it for the unbound package or unbound stand-alone, two different animals.  And your work begs another question.  Do the custom options override the GUI settings or cause unbound to not start?

    Guess I'll have to do some testing myself.  Looks like forward zones no longer make it from the GUI to the config file.  If so, the GUI notes below the custom options box is in need of editing to clarify the syntax required.



  • @bryan.paradis:

    So all the options work correctly now?

    yes its working correctly, thank you



  • Yes that was a mistake. Thanks @bryan.paradis. Fix pushed.



  • I stand corrected, the old syntax is still putting forward zones in the config file.  Options are pushed way down with the recent expansion of reserve leases and hostnames.  Thanks for clarifying the syntax bryan.



  • @markn62:

    I've never had an issue with forward zones taking.  Bryan, where did you get the syntax document?  Is it for the unbound package or unbound stand-alone, two different animals.  And your work begs another question.  Do the custom options override the GUI settings or cause unbound to not start?

    Guess I'll have to do some testing myself.  Looks like forward zones no longer make it from the GUI to the config file.  If so, the GUI notes below the custom options box is in need of editing to clarify the syntax required.

    Forward zones would take but the custom options are being put in the forward zones clause area and not in the server clause area causing errors if you add any.

    @markn62:

    I stand corrected, the old syntax is still putting forward zones in the config file.  Options are pushed way down with the recent expansion of reserve leases and hostnames.  Thanks for clarifying the syntax bryan.

    GUI generates a config file in unbound.inc out of what you fill out which is then what unbound uses. Just like the standalone.

    @wagonza:

    Yes that was a mistake. Thanks @bryan.paradis. Fix pushed.

    :) Your welcome



  • Thanks for clarifying Bryan.  Wagonza, either it's not installing correctly or the unbound fix broke the service. When I install the package it always fails the first time and shows installed but won't run.  Reboot and reinstall pkg, then unbound starts but doesn't resolve 100%.  Hit'n'miss on the queries.  Some webpages load, via google links all connections would fail.  Had to point router to backup Unbound service on another box.

    When I finally get it to install it shows in GUI as 1.4.21_3 just as it does in the available packages list. When I uninstall it the script reports it's uninstalling 1.4.21_1.  Leaving remnants behind?  Also notice watchdog leaves unbound in it's list and fills the log.  Maybe its normal the uninstall wouldn't remove a watchdog entry.

    And I'm still not clear when it can't install the unbound pkg from pfsense server that it doesn't use a cached copy of the package from local drive like the pkg installer says it will do.  When unbound acts up its very hard to fix PfSense since all web updates are reliant on it.  I'm going to look into putting unbound on an embedded linux box and forget it as a PfSense package at least until such time as it gets a thorough looking over.

    On the bright side, outgoing-num-tcp & incoming-num-tcp GUI entries are now sticky.



  • Interesting…looks like you may have remnants of an old package lying around which might be the cause of your problems.
    The fact that Watchdog still has it in its list is a good indicator of this.

    Sorry you have had problems with the package its been kind(er) to others.
    Its going into 2.2 as a replacement for DNSMasq so no longer will it be a package. Might be a better experience for you then.


  • Banned

    Just noting that I reinstalled unbound multiple times on multiple boxes without any problems whatsoever after the latest batch of commits… So yeah, you probably have some state junk leftovers on your box.



  • States get reset on reboot, no?  Don't think it's states unless only a state reset will do.  So I should console ipkg list then ipkg remove package.name?  Or is more cleanup needed?



  • Check your config.xml for multiple entries for Unbound.
    Look between <installedpackages>xml tags.</installedpackages>



  • ./conf.default/config.xml or ./cf/conf/config.xml?



  • The latter one.


Log in to reply