IPv6 Native with Telstra, Australia

Larrikin · Mar 2, 2019, 1:02 AM

@bigmaccius said in IPv6 Native with Telstra, Australia:

@larrikin had a quick look at your pcap and the first DHCPv6 packets are the "CONFIRM" type. This is what you'd see after a SOLICIT, ADVERTISE and REQUEST.

I would suggest that perhaps the capture is missing whatever happened before it was started.

Have a read of my workflow. Impossible.

Just to add, I have packet captured ubuntu two completely different ways. One using TCPDUMP on the box, another using wireshark using a HUB on the same WAN interface.

Both packet captures start before any connectivity exists between Ubuntu and Telstra.

So this is my very point. It's natural to assume I've missed it in the packet capture. But I've done it four times with a very precise workflow to ensure the packet capture starts before ANY connectivity. The fact it is missing is exactly the point. It's got nothing to do with my workflow on capturing. I think its pertinent on how IPv6 is working with Telstra.

Larrikin · Mar 2, 2019, 3:58 AM

@bigmaccius said in IPv6 Native with Telstra, Australia:

@larrikin had a quick look at your pcap and the first DHCPv6 packets are the "CONFIRM" type. This is what you'd see after a SOLICIT, ADVERTISE and REQUEST.

I would suggest that perhaps the capture is missing whatever happened before it was started.

@Derelict

Further to my above post, I just did another packet capture of the Telstra router.

I have a new working theory which goes way beyond the T1 and T2 posts above.

Here is my theory:

ICMPv6 has everything to do with setting up IPv6 for success with Telstra.

With Ubuntu, there is not even a DHCPv6 solicit packet. It just gets an IP address.

I think with Ubuntu, because its not a router, and its just a client, its behaving slightly differently with Telstra as to how it gets its IPv6 compared to a router, but that said, there are very strong similarities with the Telstra router and what happens to get a valid IPv6 from Telstra. In fact, I think that is the key to this entire thing. Let me continue...

With the Telstra router, there is something very consistent with Ubuntu going on - there is a lot of ICMPv6 traffic occurring first which seems to setup the path for DHCPv6 to work with Telstra. It's to do with the neighbor solicitation. Exactly the same thing that happens with Ubuntu.

With pfsense, this does not happen as per the packet captures. There is no successful neighbor solicitation. pfsense is just ignoring the ICMPv6 stuff.

I think this is where the key of all of this lies.

On all of the successful IPv6 captures I've done, ICMPv6 neighbor solicitation is successful before it proceeds to DHCPv6 (the router goes to DHCPv6 but Ubuntu does not - it seems to get a single IP - not a PD - through ICMPv6).

On the unsuccessful IPv6 captures I'd done (which is pfsense), ICMPv6 neighbor solicitation is unsuccessful, and then when going to DHCPv6, it fails.

That's where I am thinking the problem lies. We have been focussing on just the one DHCPv6 solicit packet, but not looking at the bigger picture of the stuff that precedes that which is all the ICMPv6 stuff. If I were a betting person, I'd bet that the ICMPv6 solicitation is the key requirement for DHCPv6 to work.

Lastly, with the very little correspondence I've had with my Telstra contact to date on this topic, he made a comment that one of the things he believes is causing the failure of pfsense, is the neighbor solicitation at the ICMPv6 level.

edit Ununtu uses ISC DHCP6 and freebsd uses WIDE DHCP6. Just tried a freebsd box (not pfsense) and it also doesn't connect to Telstra. Maybe this is a freebsd issue?

Larrikin · Mar 2, 2019, 3:17 AM

BTW, I'm going to bow out spending any more time on this for now. I've been working on it for 10 days and have failed to get it working.

I've left an email with my Telstra contact basically asking for a very detailed description on IPv6 Telstra implementation and how to get it to work – what are the requirements?

Without that, I'm spent on this. It's too difficult with too many variables, guess work, lack of knowledge of IPv6 all around with everyone I engage with (including my own very limited knowledge).

So farewell all – I will come back if Telstra will provide me with the info required – and there is hope, because my contact who is currently away said he would come back to me early next week. If that evolves into something, I'll come back. If it doesn't, I'm out for the reasons above.

Good luck all if you choose to continue to work on it. I'll keep an eye on this thread to see if one of you manages to get it to work.

Larrikin · Mar 2, 2019, 1:29 PM

@derelict Want some good news? @Bigmaccius got it working. I'm currently writing the how to guide and will post it shortly. I can attest it fully works with Telstra IPv6. I think this is one of the most complex pfsense setups for IPv6 we've seen to date. Thanks for your help derelict - we got there!

Larrikin · Mar 2, 2019, 2:47 PM

@derelict

Well folks, this has been 10.5 days worth of work, and has involved a number of people to get us here.

Special thanks go to:

derelict at netgate

Anon Telstra contact I won't name my contact as I don't think he'd appreciate it – which is one of the nice things about him – he works quietly behind the scenes and makes life easier for all of us. You know who you are, so a big thank you to you! He gave me some pertinent info last week that even at the time, I didn't realise how important that info ended up being to help solve this.

@Bigmaccius a massive shout out to him as he did most of the heavy lifting and also for cracking the final part of getting this to work – he got us across the line and just kept at it relentlessly – you can see above, I had given up.

Dean at Byte Foundry.

This is the Wiki to get it working.

https://whirlpool.net.au/wiki/pfsense_ipv6_telstra

Larrikin · Mar 2, 2019, 2:21 PM

@derelict When you read the wiki, you'll see something interesting in the system tuneables page.

What is interesting is that the switch that we're turning back on here to get this working - net.inet6.icmp6.nd6_onlink_ns_rfc4861

Seems to be related to a vulnerability from 2008 and turning this switch on is effectively re-enabling functionality that was changed to fix the problem!

https://www.freebsd.org/security/advisories/FreeBSD-SA-08:10.nd6.asc

Not sure what this means yet, but seems low-ish risk limited to ability to spoof packets on local link with ISP. Be interested if you have a view on this.

Larrikin · Mar 2, 2019, 3:06 PM

@derelict Quick question - I am guessing there are a couple of firewall rules I need to put in place to deal with the filtered result as per this? Would you mind letting me know what they are?

Thanks.

Derelict · Mar 2, 2019, 4:32 PM

Glad you got it working.

I don't know. It depends on what they are testing. There's a little ? on the right of each line. Maybe that says what they are looking for.

You have to pass whatever that is. Probably ICMPv6 echo requests.

I assume for hostname they expect you to have DNS. If they do then make it so to pass that test.

Gertjan · Mar 2, 2019, 6:31 PM

@larrikin said in IPv6 Native with Telstra, Australia:

Quick question -

Quick answer.

Glad it finally worked for you.

edit :

Forget about the hostname, except if you want your LAN device to be 'seen' on the Internet.
My only motivation to make that one 'ok' was to have a 20/20 (stupid reason, I know).

dugeem · Mar 2, 2019, 10:40 PM

@Larrikin Well done.

For reference can you please post a capture of successful pfSense IPv6 initialisation on Telstra?

randomaustralian · Mar 2, 2019, 10:43 PM

I got pointed here asking the exact same question on Netgate last night. I have been at this on and off for like 2 years and i'd always give up after a week or so.

@Larrikin i am impressed you got such a good contact from Tesltra, every time i have asked them for this information they refuse to give me any help as it is "unsupported" and when i have said i will support it myself I just need the configuration information they just flat out refused to give it to me or more recently told me to sign up for Telstra Platinum and even they told me there is no point in subscribing because they aren't allowed to give me that information.

I was considering paying for a Netgate support subscription to get the results i was after.

thank you guys for the hard your you have put into this and releasing the information to the public.

next step now is to see if i can get it working myself.

Larrikin · Mar 2, 2019, 11:31 PM

@derelict said in IPv6 Native with Telstra, Australia:

Glad you got it working.

I don't know. It depends on what they are testing. There's a little ? on the right of each line. Maybe that says what they are looking for.

You have to pass whatever that is. Probably ICMPv6 echo requests.

I assume for hostname they expect you to have DNS. If they do then make it so to pass that test.

I looked into it more deeply and it is simply that they can't reach my local laptop internally in my lan for ICMP ping. I am happy that they can't do that - I have no intention of opening that up :).

On another note, when you read the wiki, you'll see something interesting in the system tuneables page.

What is interesting is that the switch that we're turning back on here to get this working - net.inet6.icmp6.nd6_onlink_ns_rfc4861

Seems to be related to a vulnerability from 2008 and turning this switch on is effectively re-enabling functionality that was changed to fix the problem!

https://www.freebsd.org/security/advisories/FreeBSD-SA-08:10.nd6.asc

Not sure what this means yet, but seems low-ish risk limited to ability to spoof packets on local link with ISP.

Am interested if you have a view on this.

Larrikin · Mar 3, 2019, 12:30 AM

@dugeem said in IPv6 Native with Telstra, Australia:

@Larrikin Well done.

For reference can you please post a capture of successful pfSense IPv6 initialisation on Telstra?

I'm torn on that. The problem with that is that will publicly give away all my MAC addressing and IP addressing - not something I really want to do. I am all for sharing as much as I can but on this one, however I am a bit nervous of giving away my personal IP and mac details.

Larrikin · Mar 2, 2019, 11:53 PM

@randomaustralian said in IPv6 Native with Telstra, Australia:

next step now is to see if i can get it working myself.

You should be fine - just follow the wiki. If you run into any issues, just WHIM me on whirlpool and I'll help.

randomaustralian · Mar 3, 2019, 12:03 AM

so i followed your guide and the only thing that was different to my existing settings was the system tunable lines of steps 19 and 20.

i do get IPv6 internally rout-able addresses like last time but i still cant seem to pass any traffic which has been my standing problem for a while now.

Edit:

i'd like to add i am receiving IPv6 traffic because i never have had snort report an alert with an IPv6 address. i don't know how to or what a WHIM is on whirlpool.

Larrikin · Mar 3, 2019, 12:04 AM

@randomaustralian said in IPv6 Native with Telstra, Australia:

so i followed your guide and the only thing that was different to my existing settings was the system tunable lines of steps 19 and 20.

i do get IPv6 internally rout-able addresses like last time but i still cant seem to pass any traffic which has been my standing problem for a while now.

Double check steps 1 to 6.

Show screen shots of System, Routing, Gateways and your firewall ruleset on the LAN.

Larrikin · Mar 3, 2019, 12:32 AM

@randomaustralian said in IPv6 Native with Telstra, Australia:

Edit:

i'd like to add i am receiving IPv6 traffic because i never have had snort report an alert with an IPv6 address. i don't know how to or what a WHIM is on whirlpool.

That's fine. Just direct msg me here instead. I frequent both forums. I'm sure we'll get you working. I'm willing to bet its an old setting you've forgotten about when you've played with this that you've assumed aligns with the how to guide, but probably doesn't. We'll find it, and fix it :)

randomaustralian · Mar 3, 2019, 12:34 AM

@larrikin Actually i'm confident your settings are working fine. I can ping IPv6 addresses from my desktop.

In fact. I tried to ping Cloudflares IPv6 DNS server 2606:4700:4700::1111 and realized i had not re-added Cloudflares IPv6 DNS addresses back into my pfSense configuration.

Addresses re-added. Rebooted. http://ipv6-test.com/ reports i have a working IPv6 stack.

Larrikin · Mar 3, 2019, 12:34 AM

@randomaustralian said in IPv6 Native with Telstra, Australia:

@larrikin Actually i'm confident your settings are working fine. I can ping IPv6 addresses from my desktop.

In fact. I tried to ping Cloudflares IPv6 DNS server 2606:4700:4700::1111 and realized i had not re-added Cloudflares IPv6 DNS addresses back into my pfSense configuration.

Addresses re-added. Rebooted. http://ipv6-test.com/ reports i have a working IPv6 stack.

Yep - you are good. It's working. You may not have rebooted before after making the tunable changes which is key for this to work. There you go. Enjoy IPv6!

Derelict · Mar 3, 2019, 12:35 AM

@randomaustralian said in IPv6 Native with Telstra, Australia:

I was considering paying for a Netgate support subscription to get the results i was after.

With an uncooperative ISP who needs special sauce there is probably not a lot we could have done. Paying a local consultant who is familiar with Telstra would have probably been a better bet.