SIXXS-Aiccu and pfSense



  • As I wanted to setup an SIXXS AYIYA tunnel on my pfSense box and found this twitter conversation, I thought I would detail my pfSense + sixxs-aiccu findings here. SIXXS' nearest tunnel endpoint is located in my country(Brussels). The RTT to this POP is equal to ~20ms, which is 10ms faster than the closest HE tunnel endpoint in my area (Amsterdam). YMMV however. Configuring a non-aiccu tunnel is supported in pfSense's webinterface (cfr this guide); so you have to decide whether setting up a SIXXS aiccu tunnel is worth the hassle. Which solution is better for dynamic IPv4 endpoints is unknown to me, I'd speculate that the Aiccu software with its heartbeat mechanism has the edge here.

    One heads-up before we start: at the time of writing the aiccu tunnel interface's name is fixed to tun0 which means that it doesn't show up in the webinterface. One result is that you can't configure the firewall to differentiate according to the tun0 interface. See the bottom of this post for more info, any help is greatly appreciated!

    The installation/configuration is done via CLI (e.g. through SSH), to my knowledge there is no integration into the web-interface in pfSense 2.1-BETA.

    In short you can install the sixxs-aiccu package (http://www.freebsd.org/cgi/ports.cgi?query=sixxs-aiccu&stype=all) using pkg_add -r sixxs-aiccu from CLI.

    More elaborate guide:
    SSH to machine (SSH needs to be enabled via webinterface/serial port):
    ssh admin@ <pfsense-box>Mount filesystem as R/W:
    /etc/rc.conf_mount_rw
    Install package:
    pkg_add -r sixxs-aiccu
    You should see output similar to this:

    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/Latest/sixxs-aiccu.tbz… Done.
    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/p11-kit-0.11.tbz... Done.
    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/gmp-5.0.4.tbz... Done.
    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/nettle-2.4.tbz... Done.
    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/pkg-config-0.25_1.tbz... Done.
    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/libiconv-1.13.1_2.tbz... Done.
    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/gettext-0.18.1.1.tbz... Done.
    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/libgpg-error-1.10.tbz... Done.
    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/gnutls-2.12.18.tbz... Done.

    Aiccu created a configuration file under /usr/local/etc/aiccu.conf, the binary however looks for the config file at /etc/aiccu.conf so symlink it:
    ln -s /usr/local/etc/aiccu.conf /etc/aiccu.conf

    Update /etc/aiccu.conf according to the instructions on http://www.sixxs.net/wiki/Aiccu/InstallationOnFreeBSD#Configure_Aiccu. Note that creating the gif0 device isn't necessary, if you're using a AYIYA tunnel like me then aiccu will always created a tun0 interface. Unfortunately this seems to have some unwanted consequences, see the end of this post for more info.

    The sixxs-aiccu package also created a startup script at /usr/local/etc/rc.d/sixxs-aiccu. As the pfSense startup system differs a bit from FreeBSD (see NOTE on startup scripts at http://doc.pfsense.org/index.php/Installing_FreeBSD_Packages), we have to alter the script as follows:
    mv  /usr/local/etc/rc.d/sixxs-aiccu  /usr/local/etc/rc.d/sixxs-aiccu-start.sh
    chmod  +wx /usr/local/etc/rc.d/sixxs-aiccu-start.sh
    Apply the following patch:
    diff -puN sixxs-aiccu sixxs-aiccu-start.sh

    –- sixxs-aiccu 2013-05-09 12:34:42.000000000 +0200
    +++ sixxs-aiccu-start.sh 2013-05-09 12:12:43.000000000 +0200
    @@ -9,7 +9,7 @@

    Add the following lines to /etc/rc.conf to enable sixxs-aiccu:

    -#sixxs_aiccu_enable="YES"
    +sixxs_aiccu_enable="YES"

    . /etc/rc.subr
    @@ -37,4 +37,4 @@ license_cmd="${command} license"
    version_cmd="${command} version"
    extra_commands="brokers tunnels test autotest license version"

    -run_rc_command $1
    +run_rc_command start

    The resulting sixxs-aiccu-start.sh file should look like http://pastebin.com/mb9zKrhr.

    When you reboot there should now be an active tun0 interface (or gif0 in case of a non-AYIYA tunnel) and you should be able to reach IPv6 internet from your pfSense box, you can test this via ping6 ipv6.google.com.

    Now that you've successfully configured the tunnel you'll probably want your pfSense to act as an IPv6 Internet gateway for your home network. By default SIXXS provides one additional subnet that is routed over the tunnel that can be used inside your LAN. See the tunnelinfo page on sixxs.net for this subnet. You can setup a radvd server that distributes this Ipv6 prefix to all machines in your LAN or you can assign static addresses from the routed subnet to your LAN machines. I chose the latter approach and am now able to reach IPv6 Internet from my machines.

    That's it, enjoy IPv6 internet via SIXXS provided by a POP near to you. Any questions? Leave a reply.

    Issues:

    • sixxs-aiccu creates a tun0 interface and this interface isn't configurable in the webinterface.
      I've found the following issue: http://redmine.pfsense.org/issues/2211, the suggested workaround doesn't seem to work however. I've applied the work-around as follows and rebooted:

    diff /etc/inc/util.inc /nano/base/etc/inc/util.inc
    1563,1564c1563
    < if (preg_match("/^enc|^cua|^tap|^l2tp|^pptp|^ppp|^ovpn|^gif|^gre|^lagg|^bridge|vlan|_wlan/i", $ifcfg['if'])) { /* logion: enable tun interfaces in webinterface /
    < /
    if (preg_match("/^enc|^cua|^tun|^tap|^l2tp|^pptp|^ppp|^ovpn|^gif|^gre|^lagg|^bridge|vlan|_wlan/i", $ifcfg['if'])) { */
    –-

    if (preg_match("/^enc|^cua|^tun|^tap|^l2tp|^pptp|^ppp|^ovpn|^gif|^gre|^lagg|^bridge|vlan|_wlan/i", $ifcfg['if'])) {

    If anyone has some suggestions on how to fix this, please share them. You can still pass incoming IPv6 traffic to your LAN network, you just can't specify IPv6 rules specific to the AICCU tunnel (allthough the IPv6 subnet is fixed to the tunnel, so it's possible to differentiate on IP but not incoming/outgoing interface).

    References:



  • I managed to get the firewall working, assuming that the aiccu client creates a tunnel interface named tun0 you can apply the following patch to /usr/local/www/interfaces_assign.php:

    diff -pNu /usr/local/www/interfaces_assign.php.bak /usr/local/www/interfaces_assign.php
    –- ../../usr/local/www/interfaces_assign.php 2013-05-09 14:19:27.018264319 +0200
    +++ /home/fvdabeele/Downloads/interfaces_assign.php 2013-05-09 15:57:43.070257996 +0200
    @@ -143,6 +143,13 @@ if (is_array($config['ppps']['ppp']) &&
    }
    }

    +/* add tun0 interface */
    +                $portlist['tun0'] = array();
    +                $portlist['tun0']['if'] = 'tun0';
    +                $portlist['tun0']['istun'] = true;

    $ovpn_descrs = array();
    if (is_array($config['openvpn'])) {
    if (is_array($config['openvpn']['openvpn-server']))

    Then you have to create a new interface via the webinterface on  https://<pfsense>>/interfaces_assign.php. Edit the newly created interface and check 'Enable'. Next you can surf to Firewall > Rules and there you can now find a new tab for OPT1, here you can add rules for the IPv6 traffic that is tunneled via aiccu.
    Here is an example that allows SSH over IPv6 to all machines inside my subnet: http://imgur.com/KkgO1xU.

    Integrating all this in the pfSense web interface is feasible, but would be quite some work nonetheless. Apart from the php pages that would allow to configure the aiccu config, the aiccu package should be recompiled to use a separate tunnel interface named aiccu; which can then get a separate interface type (similar to the openvpn and gif tunnel types) inside pfSense.</pfsense>



  • Hi logion,

    There is support for aiccu in current stable releases?

    I am running pfsense 2.0.3 and I get  the following messages from package install:

    "Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.1-release/Latest/sixxs-aiccu.tbz: File unavailable (e.g., file not found, no access)
    pkg_add: unable to fetch 'ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.1-release/Latest/sixxs-aiccu.tbz' by URL"



  • Whether or not this guide will work in pfsense 2.0.3 I can't say (I'm running the Beta as you can see). But it would seem that the package repository structure has changed, a problem which should be more common to all 2.0.3 packages. Can you add a package via URL? You should install this package: http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/ports/i386/packages-8.1-release/Latest/sixxs-aiccu.tbz; it might be that the dependencies will give a 4.04 as well though.
    I'd suggest trying to install aiccu from the web interface: System -> Packages. If that doesn't work you might want to try changing package repository as mentioned here: http://doc.pfsense.org/index.php/Creating_Your_Own_Package_Repository#On_pfSense_2.0. I'd imagine the hostname would be ftp-archive.freebsd.org in this case.
    Here is an other suggestion: http://forum.pfsense.org/index.php/topic,8272.msg46662.html#msg46662.
    It really seems to be more pfSense related than aiccu related, can you install any other packages? pkg_add -r nano for instance?



  • I can add several packages and all work as expected. It appears to be aiccu package was intentionally not available to 2.0.x.

    I need so much to try ipv6, so I decided to move to 2.1-Beta. PfSense team is clear about ipv6 being on roadmap of 2.1 only.

    I will follow your guide and post my findings.

    Thanks!



  • Hi logion,

    I follow closely your guide, and I appeared to be successful. I can run

    sixxs-aiccu autotest /usr/local/etc/aiccu.conf
    
    

    and almost all tests passed. The only test unsuccessful was traceroute to remote IPv4 endpoint, but I can ping it. Some routers in path block traceroute's requests.

    But unfortunately I cannot reproduce 'sixxs-aiccu' behavior. I cannot ping none remote IPv6 address, nor the remote tunnel endpoint. Ping6 shows the error:

    
    [2.1-BETA1][root@fw.local]/root(21): ping6 noc.sixxs.net
    PING6(56=40+8+8 bytes) 2001:XXXX:YYYY:ZZZZ::2 --> 2001:838:1:1:210:dcff:fe20:7c7c
    ping6: sendmsg: Network is down
    ping6: wrote noc.sixxs.net 16 chars, ret=-1
    ping6: sendmsg: Network is down
    ping6: wrote noc.sixxs.net 16 chars, ret=-1
    ping6: sendmsg: Network is down
    ping6: wrote noc.sixxs.net 16 chars, ret=-1
    ^C
    --- noc.sixxs.net ping6 statistics ---
    3 packets transmitted, 0 packets received, 100.0% packet loss
    
    [2.1-BETA1][root@fw.local]/root(21): ping6 2001:XXXX:YYYY:ZZZZ::1
    PING6(56=40+8+8 bytes) 2001:XXXX:YYYY:ZZZZ::2 --> 2001:XXXX:YYYY:ZZZZ::1
    ping6: sendmsg: Network is down
    ping6: wrote noc.sixxs.net 16 chars, ret=-1
    ping6: sendmsg: Network is down
    ping6: wrote noc.sixxs.net 16 chars, ret=-1
    ping6: sendmsg: Network is down
    ping6: wrote noc.sixxs.net 16 chars, ret=-1
    ^C
    --- noc.sixxs.net ping6 statistics ---
    3 packets transmitted, 0 packets received, 100.0% packet loss
    
    

    If I run sixxs-aiccu autotest again the tests (and ping6 and traceroute6) work inside testing. At after sixxs-aiccu ends, it fails again.



  • The 'easy' part…  ;D

    I am not a programmer, but lastly I become very happy with my progress... ;)

    I figured out why network was offline: sixxs-aiccu is not running!!!

    
    /usr/local/etc/rc.d/sixxs-aiccu-start.sh start      
    Cannot 'start' sixxs_aiccu. Set sixxs_aiccu_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'.                                                                                               
    [2.1-BETA1][root@fw.local]/root(30): /usr/local/etc/rc.d/sixxs-aiccu-start.sh onestart   
    [2.1-BETA1][root@fw.local]/root(31): ping6 noc.sixxs.net                                 
    PING6(56=40+8+8 bytes) 2001:XXXX:YYYY:ZZZZ::2 --> 2001:838:1:1:210:dcff:fe20:7c7c                        
    16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=0 hlim=54 time=247.998 ms                      
    16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=1 hlim=54 time=248.104 ms                      
    16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=2 hlim=54 time=252.881 ms                      
    16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=3 hlim=54 time=248.184 ms                      
    16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=4 hlim=54 time=254.536 ms      
    
    

    I suspect pfsense do not use a '/etc/rc.conf' file to start services… so where is it?



  • logion,

    Please be more especific in 'patch' section… not all folks here deal with 'diff'...  ???



  • If you put daemonize=true in the aiccu config file (/etc/aiccu.conf), then you can just start aiccu from CMD line (/usr/local/sbin/sixxs-aiccu start) and it will start as a background process.
    If you want to integrate it with the rc.d system (i.e. setup the tunnel when pfSense boots), then look at the second part of my first post. Your /usr/local/etc/rc.d/sixxs-aiccu-start.sh shouldn't accept command line arguments, you have to uncomment sixxs_aiccu_enable="YES" and change the last line to "run_rc_command start". See lines 12 and 40 in this pastebin file: http://pastebin.com/mb9zKrhr. If you change these two lines, then the aiccu tunnel should come online after pfSense boots.

    The second patch file just adds these 4 lines at line 146 in the  /usr/local/www/interfaces_assign.php file (remove the plusses):
    +/* add tun0 interface */

    • $portlist['tun0'] = array();
    • $portlist['tun0']['if'] = 'tun0';
    • $portlist['tun0']['istun'] = true;


  • I put the entire patched /usr/local/www/interfaces_assign.php file online at http://pastebin.com/yxgAmtT0. Look at the contents near line 146



  • logion,

    Thank you very much. I am enjoying IPv6 now!  8) I believe your guide is enough for anyone wanting to try IPv6 until pFsense folks put it in the web interface. I am connected directly to a autonomous system (nearest PoP here is a AS one), with pings of marvelous 9ms for 1000km…  ;D

    However, I noticed some commands gone from sixxs-aiccu after running the patch. For instance, I can't do anymore 'sixxs-aiccu status'. There is no error message, nor output. Maybe it went to the system log?



  • There is only the onestatus command that you can run via the original /usr/local/etc/rc.d/sixxs-aiccu, it only seems to display the contents of /var/run/aiccu.pid though.
    Could you post how stable the tunnel is by the way? I've noticed that the tunnel stopped working eventhough the sixxs-aiccu daemon process was still running. Restarting the daemon fixed it…



  • @logion:

    Could you post how stable the tunnel is by the way? I've noticed that the tunnel stopped working eventhough the sixxs-aiccu daemon process was still running. Restarting the daemon fixed it…

    The first of first: I am using heartbeat here. I will let AYIYA only for NATed scenarios. The idea is my pFsense box handle tunnel and firewall only. My machines will get fixed IPs in the /64 network I got.

    That said, the tunnel is quite stable whenever your clock is synced via ntp and you allow ping6 from the remote side (very important). It go down silently if you do not allow v6 pings from remote side of tunnel after some time. Your side gets pinged every 60 seconds. Currently I could not open a rule to ICMPv6 ping since it is not on the protocols list to choose. I opened all protocols from remote IPv6 tunnel host (using single host or alias option), but it still is a issue. My pFsense box keeps blocking it. Maybe your case can be issues with delay, but I could not see it since my ISP is very good…  ;D My worst ping to remote side of tunnel was 12msec...

    Disabling firewall is not an option for me. Still seeking a solution.



  • NTP seems to be operating fine. I was dropping ICMPv6 traffic from the POP's endpoint however. Now that I've added a rule that should fix this, let's see how long the tunnel will last :). I know I should follow IETF's RFC 4890 on ICMP filtering, but has this per chance been applied to a pfSense setup before? I.e. can I use someone else's rules as a basis for ICMPv6 filtering? Maybe there should be a RFC 4890 compliant set of rules defined as a preset in pfSense?

    How do you know that the pings from the remote side aren't coming through by the way? SIXXS supplies this testing tool: http://www.sixxs.net/tools/traceroute/ but it seems to be down for the moment… You say: "Currently I could not open a rule to ICMPv6 ping since it is not on the protocols list to choose.", when choosing IPv6 as TCP/IP  Version and then choosing ICMP as protocol; I would assume this filters ICMPv6 traffic and not ICMPv4. I've added a rule like this from ALL sources and pnzone.net is able to ping me using ICMPv6, so I think it's working. Are you reloading the filter table after you create new rules? It should work...

    You can use AYIYA tunnels without NAT as well? At least that's what I'm doing and it seems to be working (although perhaps not so stable, the next few days should point this out).



  • I saw the ping packet being blocked by the 'default rule' in the firewall log. Check "show packets blocked by default rule" to see most of packets dropped. You will see the pings coming from the remote side of the tunnel being blocked. For sixxs tunnels it can't happen.

    ICMPv4 and ICMPv6 are (very) different protocols. There are several features of IPv6 wich depend of new things added to ICMPv6 (neighboor discovery, for example). I would not assume choosing IPv6 and ICMP being ICMPv6. It's make sense, but I cannot figure out why it has been dropped, so I guess better make no assumptions until someone from pFsense team can confirm.

    I believe allowing ICMPv6 from remote side is a thing more sixxs-related than needed by some RFC. They use it in their credit balance account (they mean this in the FAQ). SixXS is free at all, but has a interesting virtual credit system. If you make your tunnel stable, you gain credits to 'buy' more subnets, etc… ::) is like a sort of 'game'...

    I make a rule to allow IPV6 and ICMP, but the packets still get being dropped, and the drop is logged in firewall log. I don't know if this is a bug or if my interface is not choosen correctly in the rule. Firewall logs show WAN interface, not GIF (heartbeat is configured as GIF, not TUN). I did not apply your last patch to see the new GIF interface in webconfigurator yet, so I can't choose it in the rule setup.

    I made no testing with AYIYA yet. AYIYA have a bunch of complexity to help to connect, but I personally do not like when it fails, since there is a bunch of things to debug too... :(



  • You definitely need to allow icmpv6 traffic via the tunnel interface, not via the WAN interface (IPv6 traffic is encapsulated in IPv4 traffic when it enters via the WAN interface), therefore you need to create a OPT interface in the pfSense webinterface. This requires my patch to interfaces_assign.php if the tunnel is named tun0, if it is named gif (like in your case) it might not need to be patched; or you can replace tun with gif in my patch file ;)

    Just to demonstrate that it's working over here, my pfSense firewall rules for my tun0 interface look like this: https://www.dropbox.com/s/ldxqrx6li4frckf/Screenshot from 2013-05-13 22%3A47%3A15.png. The first rule allows SSH to the public IPv6 address of my pfSense box, the second address allows incoming ICMPv6 traffic from the tunnel POP's endpoint. Note that these addresses are in their own separate tunnel subnet (important)! I'm 100% sure that IPv6+ICMP results to ICMPv6. Executing "pfctl -sa | grep icmp" gives :
    pass in quick on tun0 inet6 proto ipv6-icmp from 2a02:578:xxxxx::1 to 2a02:578:xxxxx::2 keep state label "USER_RULE: Allow tunnel's POP endpoint to ping gatekeeper"
    Notice that proto says ipv6-icmp, while the interface is IPv6 + ICMP.

    Finally executing "tcpdump -n -i tun0 -vv icmp6" on pfSense shows ICMP6 echo requests/replies from the POP to my pfSense box every minute (as well as neighbor sollications for the tunnels POP from my pfSense box, strange that pfSense doesn't cache this for sending the echo reply packets :s).

    I am aware that ICMP6 differs from ICMP4 and that IPv6 relies heavily on ICMP6 for its workings (in regards to routers dropping packets that are larger than their link's MTUs allow, for instance). I also don't see any harm in allowing ICMP6 from the POP's endpoint to the pfSense box (they are conceptually in the same subnet after all). I was referring to a policy for incoming ICMPv6 traffic that will be forwarded to my LAN subnet (including my pfSense LAN interface, not really forwarded in this case though). In this case RFC 4890 might provide some good guidance. Guess I'll have to read up a bit more on this.



  • Hi logion,

    Thank you very much for your help. My pFsense box is correctly configured and my GIF interface is answering pings from remote side of tunnel.

    I will play with subnet now…  :P



  • I'm not quite sure why pfsense refuses to pass ICMPv6 from my POP to local End point on the tunnel if.
    IPv6 tunnel works. I've tried to allow everything from POP address to End point address but no, RRD graphs show that there's constant IPv6 in-block for 409b/s which seems to be ICMPv6.

    I've had this working earlier before i moved to routerboard, but had to come back to pfsense since ROS 6.1 managed to turn routerboard inaccessable… couple of times.



  • By the way, thanks for posting this - it helped me set up a tunnel on 2.1 really quick.

    Two problems left: the tunnel goes down after an undefined time, not passing any traffic. It seems to be realted to a tun-Problem in FreeBSD 8.0/8.1  . Should no longer present in 8.3, but sure show the same behaviour. Just upgraded to 2.1.1-PRERELEASE - let's see if that fixes it.

    Also, I recommend disabling the pfscrub Option in Advanced / Firewall/NAT due to a bug in FreeBSD:
    https://redmine.pfsense.org/issues/2762
    Even if they talk about an IPv6-Options allow rule, probably the fix mentioned in
    http://www.freebsd.org/cgi/query-pr.cgi?pr=172648
    is better. I did not want to disable rfc1323 (TCP window scaling).



  • Why you guys need a tun device for this.

    The SIXXS daemon is so damn simple that you just need a script to login and not need the daemon + the gif interface setup.

    Also it is very simple to integrate this to pfSense.

    If you do the php work on interfaces.php i will see to make this included in 2.2



  • Using AICCU is not required, but is throughly recommended. There are some requisites for keeping SIXXS tunnel up, as stated in https://www.sixxs.net/tools/heartbeat/, and AICCU takes care of them automatically.

    Based on logion's instructions and some research by myself, I made a kind of 'tutorial' for installing pfSense and use SIXXS without headaches, in portuguese language. You can take a look on it. It was made for Brazilian readers, but I added a 'Google translator' button to help readers around the world. Is not the best, but at least… it can be helpful.

    http://blog.luzemario.net.br/2013/12/ipv6-pfsense.html

    HTH



  • Hi guys

    I have the following problem with this configuration.
    I've setup AICCU a while ago (when 2.1 was still in Beta), everything worked fine for a while.
    Until recently, I don't know the exact cause, it might have been the update to 2.1.1 or 2.1.2.
    I now get this when I try to ping any external IPv6 address from the pfsense Box:

    
    ping6: sendmsg: No buffer space available
    ping6: wrote 2001:1620:f00:f::1 16 chars, ret=-1
    

    The Tunnel seems to be up:

    ./usr/local/sbin/sixxs-aiccu test
    
    Tunnel Information for xxxxxx:
    POP Id      : chzrh02
    IPv6 Local  : 2001:1620:f00:f::2/64
    IPv6 Remote : 2001:1620:f00:f::1/64
    Tunnel Type : ayiya
    Adminstate  : enabled
    Userstate   : enabled
    

    Also, the SixXs status page tells me the tunnel is up:

    Tunnel State: up 
    Tunnel Type: ayiya 
    Last Heartbeat: 2014-04-25 07:52:35 (1398412355; 0 days 00:00:24 ago) 
    Packet In: 2014-04-25 07:29:35 (1398410975; 0 days 00:23:24 ago) 
    Packet Out: 2014-04-25 07:52:17 (1398412337; 0 days 00:00:42 ago) 
    

    Upon reboot, I can briefly ping the remote endpoint.
    Firewall logs do not show any blocks.

    I also want to mention that I have a second pfSense installation with the same configuration working well (upgraded to 2.1.2 as well)
    I have tried re-installing the sixxs-aiccu package which did not help.

    Unfortunately, I'm not a FreeBSD guru…

    Any ideas on how I can troubleshoot that, or any helpful clues at all are very welcome :)



  • @Merlin: I also noticed this, unfortunately I haven't managed to fixed this (as it is probably related to OpenBSD), any update on the issue from your end?
    @ermal: can you clarify what you mean by the sixxs daemon? This is sixxs-aiccu, not? The reason for the tun device, is so that we can configure the firewall in the pfsense web configurator. When you say provide the php work, do you mean handling+processing forms for accepting sixxs-aiccu configuration input (like username, password, tunnel ID, …)? I might look into that in.
    @MichelZ: What does ifconfig say? What does traceroute6 ipv6.google.com say?

    Here the entire sixxs-aiccu package apparently got removed from my router when upgrading pfsense, I'm guessing only packages installed via the web interface might get restored after an upgrade?



  • @logion:

    ifconfig:

    tun0: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1280
            options=80000 <linkstate>inet6 fe80::250:56ff:fea1:7155%tun0 prefixlen 64 scopeid 0xb
            inet6 fe80::1420:f00:f:2%tun0 prefixlen 64 scopeid 0xb
            inet6 2001:1620:f00:f::2 --> 2001:1620:f00:f::1 prefixlen 128
            nd6 options=3 <performnud,accept_rtadv>Opened by PID 1050</performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast>
    

    tracert:

    traceroute6 to www.google.ch (2a00:1450:4001:c02::5e) from 2001:1620:f00:f::2, 64 hops max, 12 byte packets
    sendto: No buffer space available
     1 traceroute6: wrote www.google.ch 12 chars, ret=-1
    


  • Can you ping the sixxs' tunnel endpoint: ping6 2001:1620:f00:f::1 ? If so the tunnel is working and there might be an issue with your local IPv6 setup. Ifnot, then there is an issue with the tunnel.

    In case of the former:
    Do you have an IPv6 address from your tunnel subnet assigned to one of your pfsense box's interfaces? For example, in my case my tunnel prefix is 2a02:578:3ff:90ff::/64 (you can find the subnet prefix for your tunnel on the sixxs website, see https://www.sixxs.net/home/) and therefor I assigned 2a02:578:3ff:90ff:: as a static IPv6 address to my LAN interface (vr0 in my case). Note that 2a02:578:3ff:90ff/64 is not really my subnet prefix.
    If this is setup correctly there might be a firewall issue, this can be checked by disabling the firewall: pfctl -d, pinging and re-enabling the firewall (pfctl -e). If the pinging worked then you know that you should edit your firewall rules and reload the firewall.



  • @logion: I cannot ping the tunnel endpoint.
    SixXs Tunnel Info tells me that it still receives heartbeats from AICCU, but no packets are going in or out.
    Disabling PF does not help in that matter.

    I'll see if I can get some help in the SixXs forums as well, as it seems to be an issue with the tunnel itself.
    Thanks for the help so far



  • I have opened a forums post @ SixXs, if anyone's interested: https://www.sixxs.net/forum/?msg=setup-11524205

    Also, as far as the tunnel is "down" goes, a TCPDump tells me:

    0x02b0:  5069 6e67 6564 2062 7920 5369 7858 5321  Pinged.by.SixXS!
            0x02c0:  0a00 596f 7520 476f 7420 5069 6e67 6564  ..You.Got.Pinged
            0x02d0:  2062 7920 5369 7858 5321 0a00 596f 7520  .by.SixXS!..You.
            0x02e0:  476f 7420 5069 6e67 6564 2062 7920 5369  Got.Pinged.by.Si
            0x02f0:  7858 5321 0a00 596f 7520 476f 7420 5069  xXS!..You.Got.Pi
    

    So the tunnel does not seem to be "completely" down…



  • Weird. I tried to gather more info this morning, and the tunnel is currently UP :)
    I'll monitor it and see if it goes down again.

    Thanks for the help!




  • Moderator



  • To post a follow-up about this, I am still suffering from the issue where the tunnel stops working even though the sixxs-aiccu daemon is still running. I am 99% certain that this is due to me daily resetting the PPPoE session in pfsense (stupid ISP policy here that will disconnect PPPoE sessions older than 36 hours). I made a thread over at the sixxs forums as I thought one of the advantages of aiccu would be to survive this kind of scenario. I'm guessing it can (changing IP address), but there might be a bug in the daemon. Then again, the daemon is from 2007 so I don't expect much feedback from sixxs. Anyway, the thead is over here: https://www.sixxs.net/forum/?msg=general-12505873

    As a work-around I manually kill the sixxs-aiccu daemon every night when I reset my PPPoE session and restart it afterwards. This is done via a cron script in pfsense. I've put the script online at https://gist.github.com/fvdnabee/4defa281bcb7fe676b56. Be sure to heed the FAQ item Jeroen linked to though when using this script:

    I am blocked from TIC!

    Due to some people seeing a need for quering the TIC server every couple of seconds, as they most likely put AICCU or another TIC client in some sort of looping construct, eg by using daemontools/launchd/scripting/cron/etc, we have configured a ratelimit on the service to avoid it from being overburdened by misconfigured clients.

    If a client connects too frequently it will be blocked by the TIC server and a 500 error will be given pointing to this FAQ. If the client keeps on attempting to contact the TIC server even though it has been told that it is blocked, the block will be extended for a longer period of time. If we are able to determine the user causing this we will of course notify the user that this happened. It seems though that people even though informed rarely fix the problem and just keep on hammering.

    As in general you will not have to connect more than once this should not pose a problem to normal clients.

    To make it clear: do not run AICCU in a automatic restarting manner.

    If AICCU stops working there is a reason why it did that. Check the logs and the output of the program to check why and report problems to SixXS Staff or ask the forums on how to solve a problem.

    Both AYIYA and heartbeat have been designed for a variety of scenarios, eg frequently changing IP addresses there is thus no need to restart AICCU. Even if you have a mobile client you do not have to restart AICCU.

    As I only run the script once every 24 hours, I don't consider this a problem.