SIXXS-Aiccu and pfSense
-
Hi logion,
I follow closely your guide, and I appeared to be successful. I can run
sixxs-aiccu autotest /usr/local/etc/aiccu.confand almost all tests passed. The only test unsuccessful was traceroute to remote IPv4 endpoint, but I can ping it. Some routers in path block traceroute's requests.
But unfortunately I cannot reproduce 'sixxs-aiccu' behavior. I cannot ping none remote IPv6 address, nor the remote tunnel endpoint. Ping6 shows the error:
[2.1-BETA1][root@fw.local]/root(21): ping6 noc.sixxs.net PING6(56=40+8+8 bytes) 2001:XXXX:YYYY:ZZZZ::2 --> 2001:838:1:1:210:dcff:fe20:7c7c ping6: sendmsg: Network is down ping6: wrote noc.sixxs.net 16 chars, ret=-1 ping6: sendmsg: Network is down ping6: wrote noc.sixxs.net 16 chars, ret=-1 ping6: sendmsg: Network is down ping6: wrote noc.sixxs.net 16 chars, ret=-1 ^C --- noc.sixxs.net ping6 statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss [2.1-BETA1][root@fw.local]/root(21): ping6 2001:XXXX:YYYY:ZZZZ::1 PING6(56=40+8+8 bytes) 2001:XXXX:YYYY:ZZZZ::2 --> 2001:XXXX:YYYY:ZZZZ::1 ping6: sendmsg: Network is down ping6: wrote noc.sixxs.net 16 chars, ret=-1 ping6: sendmsg: Network is down ping6: wrote noc.sixxs.net 16 chars, ret=-1 ping6: sendmsg: Network is down ping6: wrote noc.sixxs.net 16 chars, ret=-1 ^C --- noc.sixxs.net ping6 statistics --- 3 packets transmitted, 0 packets received, 100.0% packet lossIf I run sixxs-aiccu autotest again the tests (and ping6 and traceroute6) work inside testing. At after sixxs-aiccu ends, it fails again.
-
The 'easy' part… ;D
I am not a programmer, but lastly I become very happy with my progress... ;)
I figured out why network was offline: sixxs-aiccu is not running!!!
/usr/local/etc/rc.d/sixxs-aiccu-start.sh start Cannot 'start' sixxs_aiccu. Set sixxs_aiccu_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'. [2.1-BETA1][root@fw.local]/root(30): /usr/local/etc/rc.d/sixxs-aiccu-start.sh onestart [2.1-BETA1][root@fw.local]/root(31): ping6 noc.sixxs.net PING6(56=40+8+8 bytes) 2001:XXXX:YYYY:ZZZZ::2 --> 2001:838:1:1:210:dcff:fe20:7c7c 16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=0 hlim=54 time=247.998 ms 16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=1 hlim=54 time=248.104 ms 16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=2 hlim=54 time=252.881 ms 16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=3 hlim=54 time=248.184 ms 16 bytes from 2001:838:1:1:210:dcff:fe20:7c7c, icmp_seq=4 hlim=54 time=254.536 msI suspect pfsense do not use a '/etc/rc.conf' file to start services… so where is it?
-
logion,
Please be more especific in 'patch' section… not all folks here deal with 'diff'... ???
-
If you put daemonize=true in the aiccu config file (/etc/aiccu.conf), then you can just start aiccu from CMD line (/usr/local/sbin/sixxs-aiccu start) and it will start as a background process.
If you want to integrate it with the rc.d system (i.e. setup the tunnel when pfSense boots), then look at the second part of my first post. Your /usr/local/etc/rc.d/sixxs-aiccu-start.sh shouldn't accept command line arguments, you have to uncomment sixxs_aiccu_enable="YES" and change the last line to "run_rc_command start". See lines 12 and 40 in this pastebin file: http://pastebin.com/mb9zKrhr. If you change these two lines, then the aiccu tunnel should come online after pfSense boots.The second patch file just adds these 4 lines at line 146 in the /usr/local/www/interfaces_assign.php file (remove the plusses):
+/* add tun0 interface */- $portlist['tun0'] = array();
- $portlist['tun0']['if'] = 'tun0';
- $portlist['tun0']['istun'] = true;
-
I put the entire patched /usr/local/www/interfaces_assign.php file online at http://pastebin.com/yxgAmtT0. Look at the contents near line 146
-
logion,
Thank you very much. I am enjoying IPv6 now! 8) I believe your guide is enough for anyone wanting to try IPv6 until pFsense folks put it in the web interface. I am connected directly to a autonomous system (nearest PoP here is a AS one), with pings of marvelous 9ms for 1000km… ;D
However, I noticed some commands gone from sixxs-aiccu after running the patch. For instance, I can't do anymore 'sixxs-aiccu status'. There is no error message, nor output. Maybe it went to the system log?
-
There is only the onestatus command that you can run via the original /usr/local/etc/rc.d/sixxs-aiccu, it only seems to display the contents of /var/run/aiccu.pid though.
Could you post how stable the tunnel is by the way? I've noticed that the tunnel stopped working eventhough the sixxs-aiccu daemon process was still running. Restarting the daemon fixed it… -
Could you post how stable the tunnel is by the way? I've noticed that the tunnel stopped working eventhough the sixxs-aiccu daemon process was still running. Restarting the daemon fixed it…
The first of first: I am using heartbeat here. I will let AYIYA only for NATed scenarios. The idea is my pFsense box handle tunnel and firewall only. My machines will get fixed IPs in the /64 network I got.
That said, the tunnel is quite stable whenever your clock is synced via ntp and you allow ping6 from the remote side (very important). It go down silently if you do not allow v6 pings from remote side of tunnel after some time. Your side gets pinged every 60 seconds. Currently I could not open a rule to ICMPv6 ping since it is not on the protocols list to choose. I opened all protocols from remote IPv6 tunnel host (using single host or alias option), but it still is a issue. My pFsense box keeps blocking it. Maybe your case can be issues with delay, but I could not see it since my ISP is very good… ;D My worst ping to remote side of tunnel was 12msec...
Disabling firewall is not an option for me. Still seeking a solution.
-
NTP seems to be operating fine. I was dropping ICMPv6 traffic from the POP's endpoint however. Now that I've added a rule that should fix this, let's see how long the tunnel will last :). I know I should follow IETF's RFC 4890 on ICMP filtering, but has this per chance been applied to a pfSense setup before? I.e. can I use someone else's rules as a basis for ICMPv6 filtering? Maybe there should be a RFC 4890 compliant set of rules defined as a preset in pfSense?
How do you know that the pings from the remote side aren't coming through by the way? SIXXS supplies this testing tool: http://www.sixxs.net/tools/traceroute/ but it seems to be down for the moment… You say: "Currently I could not open a rule to ICMPv6 ping since it is not on the protocols list to choose.", when choosing IPv6 as TCP/IP Version and then choosing ICMP as protocol; I would assume this filters ICMPv6 traffic and not ICMPv4. I've added a rule like this from ALL sources and pnzone.net is able to ping me using ICMPv6, so I think it's working. Are you reloading the filter table after you create new rules? It should work...
You can use AYIYA tunnels without NAT as well? At least that's what I'm doing and it seems to be working (although perhaps not so stable, the next few days should point this out).
-
I saw the ping packet being blocked by the 'default rule' in the firewall log. Check "show packets blocked by default rule" to see most of packets dropped. You will see the pings coming from the remote side of the tunnel being blocked. For sixxs tunnels it can't happen.
ICMPv4 and ICMPv6 are (very) different protocols. There are several features of IPv6 wich depend of new things added to ICMPv6 (neighboor discovery, for example). I would not assume choosing IPv6 and ICMP being ICMPv6. It's make sense, but I cannot figure out why it has been dropped, so I guess better make no assumptions until someone from pFsense team can confirm.
I believe allowing ICMPv6 from remote side is a thing more sixxs-related than needed by some RFC. They use it in their credit balance account (they mean this in the FAQ). SixXS is free at all, but has a interesting virtual credit system. If you make your tunnel stable, you gain credits to 'buy' more subnets, etc… ::) is like a sort of 'game'...
I make a rule to allow IPV6 and ICMP, but the packets still get being dropped, and the drop is logged in firewall log. I don't know if this is a bug or if my interface is not choosen correctly in the rule. Firewall logs show WAN interface, not GIF (heartbeat is configured as GIF, not TUN). I did not apply your last patch to see the new GIF interface in webconfigurator yet, so I can't choose it in the rule setup.
I made no testing with AYIYA yet. AYIYA have a bunch of complexity to help to connect, but I personally do not like when it fails, since there is a bunch of things to debug too... :(
-
You definitely need to allow icmpv6 traffic via the tunnel interface, not via the WAN interface (IPv6 traffic is encapsulated in IPv4 traffic when it enters via the WAN interface), therefore you need to create a OPT interface in the pfSense webinterface. This requires my patch to interfaces_assign.php if the tunnel is named tun0, if it is named gif (like in your case) it might not need to be patched; or you can replace tun with gif in my patch file ;)
Just to demonstrate that it's working over here, my pfSense firewall rules for my tun0 interface look like this: https://www.dropbox.com/s/ldxqrx6li4frckf/Screenshot%20from%202013-05-13%2022%3A47%3A15.png. The first rule allows SSH to the public IPv6 address of my pfSense box, the second address allows incoming ICMPv6 traffic from the tunnel POP's endpoint. Note that these addresses are in their own separate tunnel subnet (important)! I'm 100% sure that IPv6+ICMP results to ICMPv6. Executing "pfctl -sa | grep icmp" gives :
pass in quick on tun0 inet6 proto ipv6-icmp from 2a02:578:xxxxx::1 to 2a02:578:xxxxx::2 keep state label "USER_RULE: Allow tunnel's POP endpoint to ping gatekeeper"
Notice that proto says ipv6-icmp, while the interface is IPv6 + ICMP.Finally executing "tcpdump -n -i tun0 -vv icmp6" on pfSense shows ICMP6 echo requests/replies from the POP to my pfSense box every minute (as well as neighbor sollications for the tunnels POP from my pfSense box, strange that pfSense doesn't cache this for sending the echo reply packets :s).
I am aware that ICMP6 differs from ICMP4 and that IPv6 relies heavily on ICMP6 for its workings (in regards to routers dropping packets that are larger than their link's MTUs allow, for instance). I also don't see any harm in allowing ICMP6 from the POP's endpoint to the pfSense box (they are conceptually in the same subnet after all). I was referring to a policy for incoming ICMPv6 traffic that will be forwarded to my LAN subnet (including my pfSense LAN interface, not really forwarded in this case though). In this case RFC 4890 might provide some good guidance. Guess I'll have to read up a bit more on this.
-
Hi logion,
Thank you very much for your help. My pFsense box is correctly configured and my GIF interface is answering pings from remote side of tunnel.
I will play with subnet now… :P
-
I'm not quite sure why pfsense refuses to pass ICMPv6 from my POP to local End point on the tunnel if.
IPv6 tunnel works. I've tried to allow everything from POP address to End point address but no, RRD graphs show that there's constant IPv6 in-block for 409b/s which seems to be ICMPv6.I've had this working earlier before i moved to routerboard, but had to come back to pfsense since ROS 6.1 managed to turn routerboard inaccessable… couple of times.
-
By the way, thanks for posting this - it helped me set up a tunnel on 2.1 really quick.
Two problems left: the tunnel goes down after an undefined time, not passing any traffic. It seems to be realted to a tun-Problem in FreeBSD 8.0/8.1 . Should no longer present in 8.3, but sure show the same behaviour. Just upgraded to 2.1.1-PRERELEASE - let's see if that fixes it.
Also, I recommend disabling the pfscrub Option in Advanced / Firewall/NAT due to a bug in FreeBSD:
https://redmine.pfsense.org/issues/2762
Even if they talk about an IPv6-Options allow rule, probably the fix mentioned in
http://www.freebsd.org/cgi/query-pr.cgi?pr=172648
is better. I did not want to disable rfc1323 (TCP window scaling). -
Why you guys need a tun device for this.
The SIXXS daemon is so damn simple that you just need a script to login and not need the daemon + the gif interface setup.
Also it is very simple to integrate this to pfSense.
If you do the php work on interfaces.php i will see to make this included in 2.2
-
Using AICCU is not required, but is throughly recommended. There are some requisites for keeping SIXXS tunnel up, as stated in https://www.sixxs.net/tools/heartbeat/, and AICCU takes care of them automatically.
Based on logion's instructions and some research by myself, I made a kind of 'tutorial' for installing pfSense and use SIXXS without headaches, in portuguese language. You can take a look on it. It was made for Brazilian readers, but I added a 'Google translator' button to help readers around the world. Is not the best, but at least… it can be helpful.
http://blog.luzemario.net.br/2013/12/ipv6-pfsense.html
HTH
-
Hi guys
I have the following problem with this configuration.
I've setup AICCU a while ago (when 2.1 was still in Beta), everything worked fine for a while.
Until recently, I don't know the exact cause, it might have been the update to 2.1.1 or 2.1.2.
I now get this when I try to ping any external IPv6 address from the pfsense Box:ping6: sendmsg: No buffer space available ping6: wrote 2001:1620:f00:f::1 16 chars, ret=-1The Tunnel seems to be up:
./usr/local/sbin/sixxs-aiccu test Tunnel Information for xxxxxx: POP Id : chzrh02 IPv6 Local : 2001:1620:f00:f::2/64 IPv6 Remote : 2001:1620:f00:f::1/64 Tunnel Type : ayiya Adminstate : enabled Userstate : enabledAlso, the SixXs status page tells me the tunnel is up:
Tunnel State: up Tunnel Type: ayiya Last Heartbeat: 2014-04-25 07:52:35 (1398412355; 0 days 00:00:24 ago) Packet In: 2014-04-25 07:29:35 (1398410975; 0 days 00:23:24 ago) Packet Out: 2014-04-25 07:52:17 (1398412337; 0 days 00:00:42 ago)Upon reboot, I can briefly ping the remote endpoint.
Firewall logs do not show any blocks.I also want to mention that I have a second pfSense installation with the same configuration working well (upgraded to 2.1.2 as well)
I have tried re-installing the sixxs-aiccu package which did not help.Unfortunately, I'm not a FreeBSD guru…
Any ideas on how I can troubleshoot that, or any helpful clues at all are very welcome :)
-
@Merlin: I also noticed this, unfortunately I haven't managed to fixed this (as it is probably related to OpenBSD), any update on the issue from your end?
@ermal: can you clarify what you mean by the sixxs daemon? This is sixxs-aiccu, not? The reason for the tun device, is so that we can configure the firewall in the pfsense web configurator. When you say provide the php work, do you mean handling+processing forms for accepting sixxs-aiccu configuration input (like username, password, tunnel ID, …)? I might look into that in.
@MichelZ: What does ifconfig say? What does traceroute6 ipv6.google.com say?Here the entire sixxs-aiccu package apparently got removed from my router when upgrading pfsense, I'm guessing only packages installed via the web interface might get restored after an upgrade?
-
ifconfig:
tun0: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1280 options=80000 <linkstate>inet6 fe80::250:56ff:fea1:7155%tun0 prefixlen 64 scopeid 0xb inet6 fe80::1420:f00:f:2%tun0 prefixlen 64 scopeid 0xb inet6 2001:1620:f00:f::2 --> 2001:1620:f00:f::1 prefixlen 128 nd6 options=3 <performnud,accept_rtadv>Opened by PID 1050</performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast>tracert:
traceroute6 to www.google.ch (2a00:1450:4001:c02::5e) from 2001:1620:f00:f::2, 64 hops max, 12 byte packets sendto: No buffer space available 1 traceroute6: wrote www.google.ch 12 chars, ret=-1 -
Can you ping the sixxs' tunnel endpoint: ping6 2001:1620:f00:f::1 ? If so the tunnel is working and there might be an issue with your local IPv6 setup. Ifnot, then there is an issue with the tunnel.
In case of the former:
Do you have an IPv6 address from your tunnel subnet assigned to one of your pfsense box's interfaces? For example, in my case my tunnel prefix is 2a02:578:3ff:90ff::/64 (you can find the subnet prefix for your tunnel on the sixxs website, see https://www.sixxs.net/home/) and therefor I assigned 2a02:578:3ff:90ff:: as a static IPv6 address to my LAN interface (vr0 in my case). Note that 2a02:578:3ff:90ff/64 is not really my subnet prefix.
If this is setup correctly there might be a firewall issue, this can be checked by disabling the firewall: pfctl -d, pinging and re-enabling the firewall (pfctl -e). If the pinging worked then you know that you should edit your firewall rules and reload the firewall.