SIXXS-Aiccu and pfSense

Luzemario

I saw the ping packet being blocked by the 'default rule' in the firewall log. Check "show packets blocked by default rule" to see most of packets dropped. You will see the pings coming from the remote side of the tunnel being blocked. For sixxs tunnels it can't happen.

ICMPv4 and ICMPv6 are (very) different protocols. There are several features of IPv6 wich depend of new things added to ICMPv6 (neighboor discovery, for example). I would not assume choosing IPv6 and ICMP being ICMPv6. It's make sense, but I cannot figure out why it has been dropped, so I guess better make no assumptions until someone from pFsense team can confirm.

I believe allowing ICMPv6 from remote side is a thing more sixxs-related than needed by some RFC. They use it in their credit balance account (they mean this in the FAQ). SixXS is free at all, but has a interesting virtual credit system. If you make your tunnel stable, you gain credits to 'buy' more subnets, etc… ::) is like a sort of 'game'...

I make a rule to allow IPV6 and ICMP, but the packets still get being dropped, and the drop is logged in firewall log. I don't know if this is a bug or if my interface is not choosen correctly in the rule. Firewall logs show WAN interface, not GIF (heartbeat is configured as GIF, not TUN). I did not apply your last patch to see the new GIF interface in webconfigurator yet, so I can't choose it in the rule setup.

I made no testing with AYIYA yet. AYIYA have a bunch of complexity to help to connect, but I personally do not like when it fails, since there is a bunch of things to debug too... :(

logion

You definitely need to allow icmpv6 traffic via the tunnel interface, not via the WAN interface (IPv6 traffic is encapsulated in IPv4 traffic when it enters via the WAN interface), therefore you need to create a OPT interface in the pfSense webinterface. This requires my patch to interfaces_assign.php if the tunnel is named tun0, if it is named gif (like in your case) it might not need to be patched; or you can replace tun with gif in my patch file ;)

Just to demonstrate that it's working over here, my pfSense firewall rules for my tun0 interface look like this: https://www.dropbox.com/s/ldxqrx6li4frckf/Screenshot%20from%202013-05-13%2022%3A47%3A15.png. The first rule allows SSH to the public IPv6 address of my pfSense box, the second address allows incoming ICMPv6 traffic from the tunnel POP's endpoint. Note that these addresses are in their own separate tunnel subnet (important)! I'm 100% sure that IPv6+ICMP results to ICMPv6. Executing "pfctl -sa | grep icmp" gives :
pass in quick on tun0 inet6 proto ipv6-icmp from 2a02:578:xxxxx::1 to 2a02:578:xxxxx::2 keep state label "USER_RULE: Allow tunnel's POP endpoint to ping gatekeeper"
Notice that proto says ipv6-icmp, while the interface is IPv6 + ICMP.

Finally executing "tcpdump -n -i tun0 -vv icmp6" on pfSense shows ICMP6 echo requests/replies from the POP to my pfSense box every minute (as well as neighbor sollications for the tunnels POP from my pfSense box, strange that pfSense doesn't cache this for sending the echo reply packets :s).

I am aware that ICMP6 differs from ICMP4 and that IPv6 relies heavily on ICMP6 for its workings (in regards to routers dropping packets that are larger than their link's MTUs allow, for instance). I also don't see any harm in allowing ICMP6 from the POP's endpoint to the pfSense box (they are conceptually in the same subnet after all). I was referring to a policy for incoming ICMPv6 traffic that will be forwarded to my LAN subnet (including my pfSense LAN interface, not really forwarded in this case though). In this case RFC 4890 might provide some good guidance. Guess I'll have to read up a bit more on this.

Luzemario

Hi logion,

Thank you very much for your help. My pFsense box is correctly configured and my GIF interface is answering pings from remote side of tunnel.

I will play with subnet now… :P

s0lid

I'm not quite sure why pfsense refuses to pass ICMPv6 from my POP to local End point on the tunnel if.
IPv6 tunnel works. I've tried to allow everything from POP address to End point address but no, RRD graphs show that there's constant IPv6 in-block for 409b/s which seems to be ICMPv6.

I've had this working earlier before i moved to routerboard, but had to come back to pfsense since ROS 6.1 managed to turn routerboard inaccessable… couple of times.

MerlinWV

By the way, thanks for posting this - it helped me set up a tunnel on 2.1 really quick.

Two problems left: the tunnel goes down after an undefined time, not passing any traffic. It seems to be realted to a tun-Problem in FreeBSD 8.0/8.1 . Should no longer present in 8.3, but sure show the same behaviour. Just upgraded to 2.1.1-PRERELEASE - let's see if that fixes it.

Also, I recommend disabling the pfscrub Option in Advanced / Firewall/NAT due to a bug in FreeBSD:
https://redmine.pfsense.org/issues/2762
Even if they talk about an IPv6-Options allow rule, probably the fix mentioned in
http://www.freebsd.org/cgi/query-pr.cgi?pr=172648
is better. I did not want to disable rfc1323 (TCP window scaling).

eri--

Why you guys need a tun device for this.

The SIXXS daemon is so damn simple that you just need a script to login and not need the daemon + the gif interface setup.

Also it is very simple to integrate this to pfSense.

If you do the php work on interfaces.php i will see to make this included in 2.2

Luzemario

Using AICCU is not required, but is throughly recommended. There are some requisites for keeping SIXXS tunnel up, as stated in https://www.sixxs.net/tools/heartbeat/, and AICCU takes care of them automatically.

Based on logion's instructions and some research by myself, I made a kind of 'tutorial' for installing pfSense and use SIXXS without headaches, in portuguese language. You can take a look on it. It was made for Brazilian readers, but I added a 'Google translator' button to help readers around the world. Is not the best, but at least… it can be helpful.

http://blog.luzemario.net.br/2013/12/ipv6-pfsense.html

HTH

MichelZ

Hi guys

I have the following problem with this configuration.
I've setup AICCU a while ago (when 2.1 was still in Beta), everything worked fine for a while.
Until recently, I don't know the exact cause, it might have been the update to 2.1.1 or 2.1.2.
I now get this when I try to ping any external IPv6 address from the pfsense Box:


ping6: sendmsg: No buffer space available
ping6: wrote 2001:1620:f00:f::1 16 chars, ret=-1

The Tunnel seems to be up:

./usr/local/sbin/sixxs-aiccu test

Tunnel Information for xxxxxx:
POP Id      : chzrh02
IPv6 Local  : 2001:1620:f00:f::2/64
IPv6 Remote : 2001:1620:f00:f::1/64
Tunnel Type : ayiya
Adminstate  : enabled
Userstate   : enabled

Also, the SixXs status page tells me the tunnel is up:

Tunnel State: up 
Tunnel Type: ayiya 
Last Heartbeat: 2014-04-25 07:52:35 (1398412355; 0 days 00:00:24 ago) 
Packet In: 2014-04-25 07:29:35 (1398410975; 0 days 00:23:24 ago) 
Packet Out: 2014-04-25 07:52:17 (1398412337; 0 days 00:00:42 ago)

Upon reboot, I can briefly ping the remote endpoint.
Firewall logs do not show any blocks.

I also want to mention that I have a second pfSense installation with the same configuration working well (upgraded to 2.1.2 as well)
I have tried re-installing the sixxs-aiccu package which did not help.

Unfortunately, I'm not a FreeBSD guru…

Any ideas on how I can troubleshoot that, or any helpful clues at all are very welcome :)

logion

@Merlin: I also noticed this, unfortunately I haven't managed to fixed this (as it is probably related to OpenBSD), any update on the issue from your end?
@ermal: can you clarify what you mean by the sixxs daemon? This is sixxs-aiccu, not? The reason for the tun device, is so that we can configure the firewall in the pfsense web configurator. When you say provide the php work, do you mean handling+processing forms for accepting sixxs-aiccu configuration input (like username, password, tunnel ID, …)? I might look into that in.
@MichelZ: What does ifconfig say? What does traceroute6 ipv6.google.com say?

Here the entire sixxs-aiccu package apparently got removed from my router when upgrading pfsense, I'm guessing only packages installed via the web interface might get restored after an upgrade?

MichelZ

@logion:

ifconfig:

tun0: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1280
        options=80000 <linkstate>inet6 fe80::250:56ff:fea1:7155%tun0 prefixlen 64 scopeid 0xb
        inet6 fe80::1420:f00:f:2%tun0 prefixlen 64 scopeid 0xb
        inet6 2001:1620:f00:f::2 --> 2001:1620:f00:f::1 prefixlen 128
        nd6 options=3 <performnud,accept_rtadv>Opened by PID 1050</performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast>

tracert:

traceroute6 to www.google.ch (2a00:1450:4001:c02::5e) from 2001:1620:f00:f::2, 64 hops max, 12 byte packets
sendto: No buffer space available
 1 traceroute6: wrote www.google.ch 12 chars, ret=-1

logion

Can you ping the sixxs' tunnel endpoint: ping6 2001:1620:f00:f::1 ? If so the tunnel is working and there might be an issue with your local IPv6 setup. Ifnot, then there is an issue with the tunnel.

In case of the former:
Do you have an IPv6 address from your tunnel subnet assigned to one of your pfsense box's interfaces? For example, in my case my tunnel prefix is 2a02:578:3ff:90ff::/64 (you can find the subnet prefix for your tunnel on the sixxs website, see https://www.sixxs.net/home/) and therefor I assigned 2a02:578:3ff:90ff:: as a static IPv6 address to my LAN interface (vr0 in my case). Note that 2a02:578:3ff:90ff/64 is not really my subnet prefix.
If this is setup correctly there might be a firewall issue, this can be checked by disabling the firewall: pfctl -d, pinging and re-enabling the firewall (pfctl -e). If the pinging worked then you know that you should edit your firewall rules and reload the firewall.

MichelZ

@logion: I cannot ping the tunnel endpoint.
SixXs Tunnel Info tells me that it still receives heartbeats from AICCU, but no packets are going in or out.
Disabling PF does not help in that matter.

I'll see if I can get some help in the SixXs forums as well, as it seems to be an issue with the tunnel itself.
Thanks for the help so far

MichelZ

I have opened a forums post @ SixXs, if anyone's interested: https://www.sixxs.net/forum/?msg=setup-11524205

Also, as far as the tunnel is "down" goes, a TCPDump tells me:

0x02b0:  5069 6e67 6564 2062 7920 5369 7858 5321  Pinged.by.SixXS!
        0x02c0:  0a00 596f 7520 476f 7420 5069 6e67 6564  ..You.Got.Pinged
        0x02d0:  2062 7920 5369 7858 5321 0a00 596f 7520  .by.SixXS!..You.
        0x02e0:  476f 7420 5069 6e67 6564 2062 7920 5369  Got.Pinged.by.Si
        0x02f0:  7858 5321 0a00 596f 7520 476f 7420 5069  xXS!..You.Got.Pi

So the tunnel does not seem to be "completely" down…

MichelZ

Weird. I tried to gather more info this morning, and the tunnel is currently UP :)
I'll monitor it and see if it goes down again.

Thanks for the help!

Kiro

@logion:

Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/Latest/sixxs-aiccu.tbz… Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/p11-kit-0.11.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/gmp-5.0.4.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/nettle-2.4.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/pkg-config-0.25_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/libiconv-1.13.1_2.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/gettext-0.18.1.1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/libgpg-error-1.10.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8.3-release/All/gnutls-2.12.18.tbz... Done.

These links are down - is there a chance to install sixxs-aiccu under 2.1.5?

BBcan177

Take a look at this link:

https://forum.pfsense.org/index.php?topic=78935.msg431084#msg431084

logion

To post a follow-up about this, I am still suffering from the issue where the tunnel stops working even though the sixxs-aiccu daemon is still running. I am 99% certain that this is due to me daily resetting the PPPoE session in pfsense (stupid ISP policy here that will disconnect PPPoE sessions older than 36 hours). I made a thread over at the sixxs forums as I thought one of the advantages of aiccu would be to survive this kind of scenario. I'm guessing it can (changing IP address), but there might be a bug in the daemon. Then again, the daemon is from 2007 so I don't expect much feedback from sixxs. Anyway, the thead is over here: https://www.sixxs.net/forum/?msg=general-12505873

As a work-around I manually kill the sixxs-aiccu daemon every night when I reset my PPPoE session and restart it afterwards. This is done via a cron script in pfsense. I've put the script online at https://gist.github.com/fvdnabee/4defa281bcb7fe676b56. Be sure to heed the FAQ item Jeroen linked to though when using this script:

I am blocked from TIC!

Due to some people seeing a need for quering the TIC server every couple of seconds, as they most likely put AICCU or another TIC client in some sort of looping construct, eg by using daemontools/launchd/scripting/cron/etc, we have configured a ratelimit on the service to avoid it from being overburdened by misconfigured clients.

If a client connects too frequently it will be blocked by the TIC server and a 500 error will be given pointing to this FAQ. If the client keeps on attempting to contact the TIC server even though it has been told that it is blocked, the block will be extended for a longer period of time. If we are able to determine the user causing this we will of course notify the user that this happened. It seems though that people even though informed rarely fix the problem and just keep on hammering.

As in general you will not have to connect more than once this should not pose a problem to normal clients.

To make it clear: do not run AICCU in a automatic restarting manner.

If AICCU stops working there is a reason why it did that. Check the logs and the output of the program to check why and report problems to SixXS Staff or ask the forums on how to solve a problem.

Both AYIYA and heartbeat have been designed for a variety of scenarios, eg frequently changing IP addresses there is thus no need to restart AICCU. Even if you have a mobile client you do not have to restart AICCU.

As I only run the script once every 24 hours, I don't consider this a problem.