No bogonsv6 in tables?


  • Rebel Alliance Global Moderator

    I don't show any entries in bogonsv6 table under diag?

    I have tried toggling the setting, just they are not listed?

    gitsync as of this morning, and reboot after gitsync

    2.1-DEVELOPMENT (i386)
    built on Fri Oct 21 12:51:56 EDT 2011
    FreeBSD 8.1-RELEASE-p6






  • I will need to verify. I do see we ship with a empty table but the file it fetches should hold some 50k networks.



  • i manually adding it a while ago and it hit a pf-filter limit. Can't remember what it was but its the same limit that IPBlocker and pfBlocker has when large list.



  • It could be that the import is going wrong. Is there contents i. /Etc/bogonsv6 ?


  • Rebel Alliance Global Moderator

    -rw-r–r--  1 root  wheel          0 Mar  6  2011 bogonsv6

    File is empty.



  • Arr. That is suspect. Since my install does have a filled file you need to check if your dns works.

    There is a script in /etc rc.update_bogons.sh that normally runs. Does that work?


  • Rebel Alliance Global Moderator

    I can assure you my dns works ;)

    ok I ran that script by hand, it first picked a random number of 53000 something – that is seconds? So it was going to sleep for 14 hours?  WTF

    Nov 24 13:03:23 root: rc.update_bogons.sh is sleeping for 53124

    So I killed it and ran it by hand with 10 seconds  And it clearly updated the files.

    [2.1-DEVELOPMENT][root@pfsense.local.lan]/etc(18): ls -la bogons*
    -rw-r–r--  1 root  wheel     132 Nov 24 13:10 bogons
    -rw-r--r--  1 root  wheel  761384 Nov 24 13:10 bogonsv6

    Nov 24 13:10:11 root: rc.update_bogons.sh is ending the update cycle.
    Nov 24 13:10:11 root: Bogons files downloaded: 49475 addresses added.
    Nov 24 13:10:10 root: Bogons file downloaded: no changes.
    Nov 24 13:10:08 root: rc.update_bogons.sh is beginning the update cycle.
    Nov 24 13:10:08 root: rc.update_bogons.sh is starting up.

    So something wrong with cron?  Why has that script not kicked off?  After I kicked it off by hand, the tables loads bogonsv6 -- and yeah its huge and does take a while to load the table ;)


  • Rebel Alliance Global Moderator

    ok did an update last night to
    2.1-DEVELOPMENT (i386)
    built on Fri Nov 25 17:45:38 EST 2011
    FreeBSD 8.1-RELEASE-p6

    and now bogonsv6 is empty again?
    No entries exist in this table.

    looked at /etc
    -rw-r–r--  1 root  wheel  132 Apr 28  2011 bogons
    -rw-r--r--  1 root  wheel    0 Mar  6  2011 bogonsv6

    So is there something wrong in the cron that should update these?

    So I manually ran script rc.update_bogons.sh 10
    Nov 30 14:50:29 admin: Bogons files downloaded: 49621 addresses added.

    [2.1-DEVELOPMENT][admin@pfsense.local.lan]/etc(19): ls -la bo*
    -rw-r–r--  1 root  wheel    132 Nov 30 14:50 bogons
    -rw-r--r--  1 root  wheel  763666 Nov 30 14:50 bogonsv6

    So what is suppose to update these, do you have to manually run the script - should something kick it off after an update?  On some sort of schedule?



  • I am running the IPv6 2.1-DEVELOPMENT (amd64) branch, and I see tons of these in the system log:

    Jan  3 09:03:07 pfs root: rc.update_bogons.sh is beginning the update cycle.
    Jan  3 09:03:09 pfs root: Could not download http://files.pfsense.org/mirrors/bogon-bn-nonagg.txt.md5 (md5 mismatch)
    Jan  3 09:03:09 pfs root: rc.update_bogons.sh is starting up.
    Jan  3 09:03:09 pfs root: rc.update_bogons.sh is sleeping for 35036
    Jan  3 09:03:09 pfs root: Could not download http://files.pfsense.org/mirrors/fullbogons-ipv6.txt.md5 (md5 mismatch)
    Jan  3 09:03:09 pfs root: rc.update_bogons.sh is ending the update cycle.
    Jan  3 09:03:09 pfs root: rc.update_bogons.sh is starting up.
    Jan  3 09:03:09 pfs root: rc.update_bogons.sh is sleeping for 37978
    Jan  3 09:03:52 pfs root: rc.update_bogons.sh is beginning the update cycle.

    I logged in via ssh and there are 64 "sh /etc/rc.update_bogons.sh" processes running.

    I could kill all the processes and manually download the file and put it in place but if the md5 sums don't match there is possibly a bigger problem than that?


  • Rebel Alliance Developer Netgate

    Looks fine from here:

    [jimp@files mirrors]$ md5 bogon-bn-nonagg.txt
    MD5 (bogon-bn-nonagg.txt) = 82bf1e457c3cbb64021067e2a2230051
    [jimp@files mirrors]$ cat bogon-bn-nonagg.txt.md5
    MD5 (bogon-bn-nonagg.txt) = 82bf1e457c3cbb64021067e2a2230051
    [jimp@files mirrors]$ md5 fullbogons-ipv6.txt
    MD5 (fullbogons-ipv6.txt) = 0c56fda817b9933477c576a36da0e5b6
    [jimp@files mirrors]$ cat fullbogons-ipv6.txt.md5 
    MD5 (fullbogons-ipv6.txt) = 0c56fda817b9933477c576a36da0e5b6
    
    


  • I'm not sure what was happening with the md5 sums not matching.  I killed all the sleeping processes (and all the sleep processes too) and i still couldn't run it manually.  I finally downloaded the http://files.pfsense.org/mirrors/fullbogons-ipv6.txt file (and removed the first and last comment lines) and put it into place at /etc/bogonsv6 but the web interface still showed the bogonsv6 table empty.

    I then went into the IPv4 bogons table and clicked the Download the latest bogon data button.  After that the bogonsv6 table was populated.

    I do notice one thing with the rc.update_bogons.sh script:

    in /etc/rc.update_bogons.sh there is a problem with the last part:

    BOGON_MD5=/usr/bin/fetch -q -o - "http://files.pfsense.org/mirrors/fullbogons-ipv6.txt.md5" | awk '{ print $4 }'
    ON_DISK_MD5=md5 /tmp/bogonsv6 | awk '{ print $4 }'
    if [ "$BOGON_MD5" = "$ON_DISK_MD5" ]; then
    egrep -v "^#" /tmp/bogonsv6 > /etc/bogonsv6
    /etc/rc.conf_mount_ro
    RESULT=/sbin/pfctl -t bogonsv6 -T replace -f /etc/bogonsv6 2>&1
    rm /tmp/bogons
    echo "Bogons files downloaded:  $RESULT" | logger
    else
    echo "Could not download http://files.pfsense.org/mirrors/fullbogons-ipv6.txt.md5 (md5 mismatch)" | logger

    Relaunch and sleep

    sh /etc/rc.update_bogons.sh &
    fi

    It writes the downloaded file (minus the '^#'  lines) to /tmp/bogonsv6 but after replacing /etc/bogonsv6 it deletes /tmp/bogons instead of /tmp/bogonsv6

    It is possible, i suppose, that if the script was run multiple times simultaneously the IPv6 part would delete the /tmp/bogons file while the other script was attempting to use it to update /etc/bogons


  • Rebel Alliance Developer Netgate

    I fixed that line in the script just now.

    Not sure how you could have had so many of them sitting there unfinished though, that is a bit perplexing.



  • Thanks jimp for fixing that!

    It is strange since my crontab has the rc.update_bogons.sh running once a day,

    …although each time the rc.update_bogons.sh script is run, it has the initial sleep plus each section has an additional relaunch and sleep in it for a total of 4 relaunch and 5 sleeps if it has major problems...
    Such as if the WAN interface is down, or some other problem such as md5 (weird i know).

    maybe an exit 1 should be called after the first relaunch so that it doesn't relaunch up to 4 times/script and start a relaunch cascade!


Locked