IPv6 Native with Telstra, Australia
-
@derelict said in IPv6 Native with Telstra, Australia:
@larrikin said in IPv6 Native with Telstra, Australia:
They will not change their DHCPv6 config for us unfortunately.
So I suggest submitting a detailed feature request at https://redmine.pfsense.org/ to ask the developers/maintainers to incur all the technical debt for making pfSense accommodate all of the ISPs in the world who choose to disobey accepted standards.
BTW, are you serious when you ask me to write this up there? Hard to know if you are being sarcastic or not. I do appreciate your point, which is why, as I said above, I created this post (actually before we even started this journey):
https://forum.netgate.com/topic/140938/commercial-opportunity-for-netgate-ipv6
-
Yes, I am serious that you write it up there. I only write things up I can personally test and prove. And I am not going to write something up asking the devs to code something so it works with a broken ISP.
Let me ask you this: If Telstra were to mangle something fundamental like the TCP 3-way handshake so you could only make TCP connections when using their routers, would it be pfSense's burden to code around that, too? Where does it end? What does Telstra have to do to get the finger pointed at THEM?
I am still not convinced this has anything to do with T1/T2.
It would be nice to have confirmation from Telstra as to what they require - and why.
What is being logged by their system when the Solicit is received and why is there no response?
-
@derelict said in IPv6 Native with Telstra, Australia:
Yes, I am serious that you write it up there. I only write things up I can personally test and prove. And I am not going to write something up asking the devs to code something so it works with a broken ISP.
Oh, I certainly wouldn't expect you to write it up. I'd do it if I am going to. But I don't want to write anything up yet until Telstra get back to me next week so I have a much better understanding from their perspective on their implementation of IPv6, and also get their views as to why their DHCPv6 server is not responding (my contact said he'd check the logs against my packet captures to find out). Without that info, I don't want to write anything up as what I have today is speculation, not pure facts.
Let me ask you this: If Telstra were to mangle something fundamental like the TCP 3-way handshake so you could only make TCP connections when using their routers, would it be pfSense's burden to code around that, too? Where does it end? What does Telstra have to do to get the finger pointed at THEM?
Understood so let me get more facts before we do anything else so we know what the actual facts are.
I am still not convinced this has anything to do with T1/T2.
Yeah, I hear you. It is speculation on my part, and until I have further info from Telstra, I say we put ourselves into a holding pattern.
It would be nice to have confirmation from Telstra as to what they require - and why.
What is being logged by their system when the Solicit is received and why is there no response?
Yep - fully agree.
-
@derelict I have captured Ubuntu successfully connecting to Telstra IPv6 and I cannot see a DHCPv6 solicit packet in terms of how it got it.
So I have a theory of how this all works - but I’ll caveat it that I don’t really know this stuff as well as you do.
- Unless I am blind, I cannot see a UDP DHCPv6 solicit packet at all. This is consistent with a number of packet captures I did. I have not missed it by starting the capture late. I followed a very precise workflow and packet captured two ways. one using TCPdump and one using a WAN hub so plugging into the WAN interface and capturing the packets. At every time, the capture started before Ubuntu had ANY connectivity to the NTD.
- ICMPv6 is being used to setup the entire IPv6 delegation.
- ICMPv6 is using neighbor solicitation
- Telstra’s original email to me (which is posted in my original post) also states that one of the reasons they think its failing (pfsense) is that neighbor solicitation on ICMPv6 is failing
- DHCPv6 is the FINAL stage where its issued an IPv6 address and PD, but all the hard work to get there is done via ICMPv6 (and DHCPv6 is not being used to solicit).
ICMPv6 is SLAAC is it not? Are we witnessing a combination of SLAAC and DHCP at work?
Attached is a capture this morning. See attached screenshot for proof that it got its IPv6. The capture is using tcpdump on the Ubuntu box itself.
I booted up Ubuntu without it plugged into anything.
Started the capture.
Plugged the connection in directly to my NTD.
Took the screen shot attached to prove its getting a valid IPv6 address.
Then stopped the capture.
Interested in your thoughts.
0_1551487552781_ubuntu telstra.cap.zip
-
@larrikin had a quick look at your pcap and the first DHCPv6 packets are the "CONFIRM" type. This is what you'd see after a SOLICIT, ADVERTISE and REQUEST.
I would suggest that perhaps the capture is missing whatever happened before it was started.
-
@bigmaccius said in IPv6 Native with Telstra, Australia:
@larrikin had a quick look at your pcap and the first DHCPv6 packets are the "CONFIRM" type. This is what you'd see after a SOLICIT, ADVERTISE and REQUEST.
I would suggest that perhaps the capture is missing whatever happened before it was started.
Have a read of my workflow. Impossible.
Just to add, I have packet captured ubuntu two completely different ways. One using TCPDUMP on the box, another using wireshark using a HUB on the same WAN interface.
Both packet captures start before any connectivity exists between Ubuntu and Telstra.
So this is my very point. It's natural to assume I've missed it in the packet capture. But I've done it four times with a very precise workflow to ensure the packet capture starts before ANY connectivity. The fact it is missing is exactly the point. It's got nothing to do with my workflow on capturing. I think its pertinent on how IPv6 is working with Telstra.
-
@bigmaccius said in IPv6 Native with Telstra, Australia:
@larrikin had a quick look at your pcap and the first DHCPv6 packets are the "CONFIRM" type. This is what you'd see after a SOLICIT, ADVERTISE and REQUEST.
I would suggest that perhaps the capture is missing whatever happened before it was started.
Further to my above post, I just did another packet capture of the Telstra router.
I have a new working theory which goes way beyond the T1 and T2 posts above.
Here is my theory:
ICMPv6 has everything to do with setting up IPv6 for success with Telstra.
With Ubuntu, there is not even a DHCPv6 solicit packet. It just gets an IP address.
I think with Ubuntu, because its not a router, and its just a client, its behaving slightly differently with Telstra as to how it gets its IPv6 compared to a router, but that said, there are very strong similarities with the Telstra router and what happens to get a valid IPv6 from Telstra. In fact, I think that is the key to this entire thing. Let me continue...
With the Telstra router, there is something very consistent with Ubuntu going on - there is a lot of ICMPv6 traffic occurring first which seems to setup the path for DHCPv6 to work with Telstra. It's to do with the neighbor solicitation. Exactly the same thing that happens with Ubuntu.
With pfsense, this does not happen as per the packet captures. There is no successful neighbor solicitation. pfsense is just ignoring the ICMPv6 stuff.
I think this is where the key of all of this lies.
On all of the successful IPv6 captures I've done, ICMPv6 neighbor solicitation is successful before it proceeds to DHCPv6 (the router goes to DHCPv6 but Ubuntu does not - it seems to get a single IP - not a PD - through ICMPv6).
On the unsuccessful IPv6 captures I'd done (which is pfsense), ICMPv6 neighbor solicitation is unsuccessful, and then when going to DHCPv6, it fails.
That's where I am thinking the problem lies. We have been focussing on just the one DHCPv6 solicit packet, but not looking at the bigger picture of the stuff that precedes that which is all the ICMPv6 stuff. If I were a betting person, I'd bet that the ICMPv6 solicitation is the key requirement for DHCPv6 to work.
Lastly, with the very little correspondence I've had with my Telstra contact to date on this topic, he made a comment that one of the things he believes is causing the failure of pfsense, is the neighbor solicitation at the ICMPv6 level.
edit Ununtu uses ISC DHCP6 and freebsd uses WIDE DHCP6. Just tried a freebsd box (not pfsense) and it also doesn't connect to Telstra. Maybe this is a freebsd issue?
-
BTW, I'm going to bow out spending any more time on this for now. I've been working on it for 10 days and have failed to get it working.
I've left an email with my Telstra contact basically asking for a very detailed description on IPv6 Telstra implementation and how to get it to work – what are the requirements?
Without that, I'm spent on this. It's too difficult with too many variables, guess work, lack of knowledge of IPv6 all around with everyone I engage with (including my own very limited knowledge).
So farewell all – I will come back if Telstra will provide me with the info required – and there is hope, because my contact who is currently away said he would come back to me early next week. If that evolves into something, I'll come back. If it doesn't, I'm out for the reasons above.
Good luck all if you choose to continue to work on it. I'll keep an eye on this thread to see if one of you manages to get it to work.
-
@derelict Want some good news? @Bigmaccius got it working. I'm currently writing the how to guide and will post it shortly. I can attest it fully works with Telstra IPv6. I think this is one of the most complex pfsense setups for IPv6 we've seen to date. Thanks for your help derelict - we got there!
-
Well folks, this has been 10.5 days worth of work, and has involved a number of people to get us here.
Special thanks go to:
derelict at netgate
Anon Telstra contact I won't name my contact as I don't think he'd appreciate it – which is one of the nice things about him – he works quietly behind the scenes and makes life easier for all of us. You know who you are, so a big thank you to you! He gave me some pertinent info last week that even at the time, I didn't realise how important that info ended up being to help solve this.
@Bigmaccius a massive shout out to him as he did most of the heavy lifting and also for cracking the final part of getting this to work – he got us across the line and just kept at it relentlessly – you can see above, I had given up.
Dean at Byte Foundry.
This is the Wiki to get it working.
https://whirlpool.net.au/wiki/pfsense_ipv6_telstra
-
@derelict When you read the wiki, you'll see something interesting in the system tuneables page.
What is interesting is that the switch that we're turning back on here to get this working - net.inet6.icmp6.nd6_onlink_ns_rfc4861
Seems to be related to a vulnerability from 2008 and turning this switch on is effectively re-enabling functionality that was changed to fix the problem!
https://www.freebsd.org/security/advisories/FreeBSD-SA-08:10.nd6.asc
Not sure what this means yet, but seems low-ish risk limited to ability to spoof packets on local link with ISP. Be interested if you have a view on this.
-
@derelict Quick question - I am guessing there are a couple of firewall rules I need to put in place to deal with the filtered result as per this? Would you mind letting me know what they are?
Thanks.
-
Glad you got it working.
I don't know. It depends on what they are testing. There's a little
?on the right of each line. Maybe that says what they are looking for.You have to pass whatever that is. Probably ICMPv6 echo requests.
I assume for hostname they expect you to have DNS. If they do then make it so to pass that test.
-
@larrikin said in IPv6 Native with Telstra, Australia:
Quick question -
Glad it finally worked for you.
edit :
Forget about the hostname, except if you want your LAN device to be 'seen' on the Internet.
My only motivation to make that one 'ok' was to have a 20/20 (stupid reason, I know). -
@Larrikin Well done.
For reference can you please post a capture of successful pfSense IPv6 initialisation on Telstra?
-
I got pointed here asking the exact same question on Netgate last night. I have been at this on and off for like 2 years and i'd always give up after a week or so.
@Larrikin i am impressed you got such a good contact from Tesltra, every time i have asked them for this information they refuse to give me any help as it is "unsupported" and when i have said i will support it myself I just need the configuration information they just flat out refused to give it to me or more recently told me to sign up for Telstra Platinum and even they told me there is no point in subscribing because they aren't allowed to give me that information.
I was considering paying for a Netgate support subscription to get the results i was after.
thank you guys for the hard your you have put into this and releasing the information to the public.
next step now is to see if i can get it working myself.
2 x UP board, 4GB RAM + 64 GB eMMC w/ vesa case (http://up-shop.org/)
1x UP^2 Pentium Quad Core, 8GB RAM, 128GB eMMC w/ vesa case (pfSense)
1x UP Core Plus E3950, 8GB RAM, 64GB EMMC+ Net Plus i210-IT
1x Dell Power Edge R510
2x Dell Power Edge R610 -
@derelict said in IPv6 Native with Telstra, Australia:
Glad you got it working.
I don't know. It depends on what they are testing. There's a little
?on the right of each line. Maybe that says what they are looking for.You have to pass whatever that is. Probably ICMPv6 echo requests.
I assume for hostname they expect you to have DNS. If they do then make it so to pass that test.
I looked into it more deeply and it is simply that they can't reach my local laptop internally in my lan for ICMP ping. I am happy that they can't do that - I have no intention of opening that up :).
On another note, when you read the wiki, you'll see something interesting in the system tuneables page.
What is interesting is that the switch that we're turning back on here to get this working - net.inet6.icmp6.nd6_onlink_ns_rfc4861
Seems to be related to a vulnerability from 2008 and turning this switch on is effectively re-enabling functionality that was changed to fix the problem!
https://www.freebsd.org/security/advisories/FreeBSD-SA-08:10.nd6.asc
Not sure what this means yet, but seems low-ish risk limited to ability to spoof packets on local link with ISP.
Am interested if you have a view on this.
-
@dugeem said in IPv6 Native with Telstra, Australia:
@Larrikin Well done.
For reference can you please post a capture of successful pfSense IPv6 initialisation on Telstra?
I'm torn on that. The problem with that is that will publicly give away all my MAC addressing and IP addressing - not something I really want to do. I am all for sharing as much as I can but on this one, however I am a bit nervous of giving away my personal IP and mac details.
-
@randomaustralian said in IPv6 Native with Telstra, Australia:
next step now is to see if i can get it working myself.
You should be fine - just follow the wiki. If you run into any issues, just WHIM me on whirlpool and I'll help.
-
so i followed your guide and the only thing that was different to my existing settings was the system tunable lines of steps 19 and 20.
i do get IPv6 internally rout-able addresses like last time but i still cant seem to pass any traffic which has been my standing problem for a while now.
Edit:
i'd like to add i am receiving IPv6 traffic because i never have had snort report an alert with an IPv6 address. i don't know how to or what a WHIM is on whirlpool.
