Setup NAT64 in pfSense

chrcoluk

It seems some vendors are treating fragments differently, I dont know the specifics.

Also I dont own an xbox one. So dont know what addresses are been made available, the guy who posted on the tech forum said was just one address showing in the xbox interface.

JKnott

@chrcoluk said in Setup NAT64 in pfSense:

seems some vendors are treating fragments differently, I dont know the specifics.

There shouldn't ever be any fragments for some vendors to treat differently. In IPv6 routers are not allowed to fragment, ever. In IPv4, routers originally would fragment when sending over a link with a MTU too small to handle the packet. With IPv6, [path MTU discovery](Path MTU Discovery) is used to prevent sending packets that are too large for the smallest MTU along the path. This is mandatory on IPv6 and IPv4 is moving to it too. On Linux PMTUD is used for everything over IPv4 and Windows uses it for TCP. IPv6 doesn't even have the IP flags to support fragmentation, so where are those fragments coming from?

chrcoluk

You never heard of vendors not following guidelines before? It happens.

Also why do cloudflare even test for fragment support on this page if it never ever happens? http://icmpcheckv6.popcount.org/

I think I remember reading about reports of broken dnssec related to fragments on ipv6.

JKnott

@chrcoluk

I tried that test, but didn't see any ICMP too big messages. Also, is the spec supposed to be changed because some vendors don't follow it? As I mentioned, IPv6 doesn't even support fragmentation and IPv4 has the do not fragment flag that tells routers not to fragment. So, if routers fragment with either, then those routers are defective.

chrcoluk

I dont know about the spec, but most people just care if the internet works. They dont care about specification's if their internet appliance isnt working, and if you are a large isp, then ultimately thats all you going to care about as well, just making the internet work well enough to keep the complaints down and sales coming in.

I just reported on here what I was told in regards to this large isp, they have had problems discovered that were related to fragments as well as icmp issues.

As an example, ipv6 everyone ideally should be using 1280 mtu, which is compatible with pppoe, pppoa, ethernet, openvpn etc. In this imagined world everyone is happy without mtu discovery even doing anything at all, but we already have various entities including cloudflare deciding to use different mtu sizes. But as you said icmp discovery is mandatory so that shouldnt be a problem right? yet we have devices blocking icmp discovery on ipv6. Its just how it is.

I had to enable icmp in my windows firewall to get ipv6 icmp working, windows is the most popular OS on the planet and doesnt even comply.

JKnott

@chrcoluk

In IPv4, routers use the fragment offset and more fragments flag to handle fragments. Those do not exist in IPv6, so how is a router supposed to fragment IPv6? Any router that fragments IPv6 or IPv4 with the do not fragment flag set is defective. There is no two ways about it and those routers shouldn't be allowed anywhere near the Internet. The only fragmentation allowed with IPv6 is that which may be done by the source and never by any router. Here is some info on the source fragmenting a packet. Please note that requires an extra header and is used only for end to end fragmentation.

As an example, ipv6 everyone ideally should be using 1280 mtu

That would be the same as everyone having to use 576 bytes on IPv4. By doing that, you decrease usable bandwidth. On the other hand some networks now use 9K jumbo frames, to improve performance.

chrcoluk

By accident when looking on other information related to PF I came across this old mailing list post.

https://lists.freebsd.org/pipermail/freebsd-pf/2014-December/007532.html

Its in regards to PF in FreeBSD mishandling fragment headers, its EDNS0 so again dnssec related.

Now of course supposedly fragments are not a thing in IPv6 so how can a device be mishandling something that's not part of the spec?

I looked into it a bit more, and I think fragments are allowed to be sent by the sender, but the difference from IPv4 is intermediate routers should only pass on the fragments and do no reassembly. Not block them entirely. The recipient can reassemble or block.

I have gone back to my friend on this, as this is above my head and experience. He has way more qualifications than me, and I will see what he tells me and post back here.

I also came across this

https://melkfl.es/article/2018/07/edns/

So IPv6 is adding new challenges to people related to fragments, that guy figured it out as he is affluent in internet tech, but your average joe bloggs broadband subscriber wouldn't.

JKnott

@chrcoluk said in Setup NAT64 in pfSense:

I looked into it a bit more, and I think fragments are allowed to be sent by the sender, but the difference from IPv4 is intermediate routers should only pass on the fragments and do no reassembly.

I mentioned that only the source can fragment. Also, routers never reassemble fragments, even with IPv4. Reassembly only occurs at the destination.

Not block them entirely.

Are they being blocked? Or is there some other issue?

FWIW, I just tried pinging google.com with 2000 byte pings, but it fails. I can see the fragments going out, but nothing comes back. I then tried from my notebook computer, connected to another port on my modem and pinged my desktop computer. Wireshark on the desktop shows both the ping and reply, but Wireshark on the notebook and also Packet Capture on pfSense WAN show only the ping, but not the reply. So, pfSense is allowing the fragmented ping in, but not letting the fragmented reply out. I'll have to look into this some more.

JKnott

@JKnott

I just did some more testing. I connected my notebook computer to the 2nd port on my cable modem, so that it gets it's own address, separate from my LAN. When I ping my desktop system from my notebook, I receive the requests and replies are sent, but not received. Packet Capture on the WAN port does not show replies. When I ping from my desktop to the notebook, I get replies. It appears pfSense is blocking replies from a computer on the LAN to another out on the WAN, but not from WAN to LAN. All pings were sent with the payload set to 2000 bytes.

This appears to be a bug in pfSense or, more likely, in the BSD it runs on. How does this bug get "officially" reported?

chrcoluk

@JKnott said in Setup NAT64 in pfSense:

@chrcoluk said in Setup NAT64 in pfSense:

I looked into it a bit more, and I think fragments are allowed to be sent by the sender, but the difference from IPv4 is intermediate routers should only pass on the fragments and do no reassembly.

I mentioned that only the source can fragment. Also, routers never reassemble fragments, even with IPv4. Reassembly only occurs at the destination.

Not block them entirely.

Are they being blocked? Or is there some other issue?

FWIW, I just tried pinging google.com with 2000 byte pings, but it fails. I can see the fragments going out, but nothing comes back. I then tried from my notebook computer, connected to another port on my modem and pinged my desktop computer. Wireshark on the desktop shows both the ping and reply, but Wireshark on the notebook and also Packet Capture on pfSense WAN show only the ping, but not the reply. So, pfSense is allowing the fragmented ping in, but not letting the fragmented reply out. I'll have to look into this some more.

I expect its a bug, and also that url is very old, that particular issue might even be fixed now.

It be good if you look into it tho. :)

Just seen your newer reply also, if you think its a bug, and its in the underlying PF/BSD code, then my opinion is it should be reported to FreeBSD developers. Then if/when they fix it, that fix would make it into pfSense.

JKnott

@chrcoluk said in Setup NAT64 in pfSense:

I expect its a bug, and also that url is very old, that particular issue might even be fixed now.
It be good if you look into it tho. :)

I did look into it and it appears there is a bug with pfSense, as I mentioned above. This is with the current version of pfSense.

mrsunfire

Any news on this? I need NAT64. Is it possible in any way with pfSense?

bert64

@johnpoz said in Setup NAT64 in pfSense:

They are running out of IPv4 rfc1918 space because they choose to do so.. Plain and simple!

Sure some devices might need to talk to each other.. Not ALL of them!! And if need be they can nat, etc. etc.. Sorry but they are touting their move to ipv6 like they are doing something innovative.. And they are using it as marketing.. they don't NEED to move to it..

Which is GREAT... But don't tell me you "have to" because your out of rfc1918 space.

And to be honest here is the big problem with the eventual migration... Is once you move part of the network to IPv6.. That frees up lots of IPv4 that can be used now..

For example could their management vlans on Ipv6, they could put their storage vlans on IPv6, they could put xyz on IPv6, etc etc.. This frees up LOTS of address space to use where its needed, etc.

They don't need to move to IPv6, they could continue working around the shortcomings of legacy ip for years to come. Just like you could keep repairing and modifying a rusty 1980s car. Microsoft have clearly decided that it's more cost effective, easier and more secure to move to IPv6.

Yes not everything needs to talk to each other, but inevitably some things will.
With an IPv6 network where everything is addressable, you add the necessary allow rules and job done.

With legacy ip, you might have overlapping address space so you need nat or even double nat, which then means you need to waste address space with the translated addresses too. Their offices in redmond and finland might both use 192.168.1.x, so 192.168.1.10 (finland) cant talk to 192.168.1.10 (redmond) as the stack will route the traffic locally. Instead you need to create a virtual network 192.168.2.10 (virtual) which 192.168.1.10(redmond) talks to, which then translates the traffic before forwarding it to 192.168.1.10(finland), so both sides are actually talking to 192.168.2.10.
Then you have inconsistent addresses, depending on where you're locate you might need to connect to 192.168.1.10 or 192.168.2.10, so you need to setup split dns etc too.
Then you consider logging/security, as the traffic will have different src/dst addresses depending where on the network it is, you have to correlate multiple log sources. If your sat in the SOC and you see suspicious traffic from 192.168.1.10 did it originate in finland or redmond? You now have extra work to find out... If you see traffic from an ipv6 address it's unique and you know it correlates to a single device.

The idea that the number of employees correlates to address usage also makes no sense... Assuming every employee has at least a desktop and a mobile device, some employees are going to have a lot more - for instance software developers will have clusters of machines performing builds, machines for testing etc. Microsoft also support their products for several years after release so they are going to have build/test networks for each major version going back several years.
Then you have address wastage due to the nat kludges above...
Not to mention the wastage of addresses each time you create a legacy ip subnet - network address, broadcast address, minimum of 1 address for the router possibly 3 if you use vrrp/hsrp/etc.
Then every time you create a subnet, you make it bigger than strictly necessary to allow room for expansion - because otherwise having to readdress everything is extremely painful.

Github isn't the last company microsoft are going to acquire either, sooner or later they are going to acquire more and it will be the same integration headaches all over again.

So yes, Microsoft could kick the can down the road and continue struggling with legacy ip for a few more years, spending a lot of money dealing with the headaches and security implications before having to implement IPv6 at some point in the future anyway.
Or they can implement ipv6 now, then its done and doesn't need to be done again. They gain a network which is simpler, easier to manage, easier to monitor, more secure and easier to expand in future. They made the smart move.

JKnott

@bert64 said in Setup NAT64 in pfSense:

They don't need to move to IPv6, they could continue working around the shortcomings of legacy ip for years to come. Just like you could keep repairing and modifying a rusty 1980s car. Microsoft have clearly decided that it's more cost effective, easier and more secure to move to IPv6.
Yes not everything needs to talk to each other, but inevitably some things will.
With an IPv6 network where everything is addressable, you add the necessary allow rules and job done.
With legacy ip, you might have overlapping address space so you need nat or even double nat, which then means you need to waste address space with the translated addresses too.

Yep, IPv4 hasn't been adequate since the day it became necessary to use NAT. Now, we have hacks upon hacks to get around the address shortage. Of course, this is before we get to the fact that many people are behind carrier grade NAT, which means they have no means of accessing their home network with a VPN etc..

IPv6 is where the world is moving and refusing to move with it is head in the sand stupidity. The longer people refuse to move, the longer some people will be behind CG NAT.

dabombnl

Just tried this and it works great. Here is what I did:

Download FreeBSD 11.3 (or whatever version your pfSense is based on), copy /boot/kernel/ipfw_nat64.ko to your pfsense install.
Load the IPFW module: 'kldload ipfw_nat64'
Enable IPFW: 'sysrc firewall_enable=YES' and 'service ipfw start'
Enter the nat64lsn rules you want, like in OP.
Make sure you are allowing the traffic in both PF and IPFW firewalls.

How do we go about getting this integrated?

IsaacFL

@dabombnl said in Setup NAT64 in pfSense:

Just tried this and it works great. Here is what I did:

Download FreeBSD 11.3 (or whatever version your pfSense is based on), copy /boot/kernel/ipfw_nat64.ko to your pfsense install.

Load the IPFW module: 'kldload ipfw_nat64'

Enable IPFW: 'sysrc firewall_enable=YES' and 'service ipfw start'

Enter the nat64lsn rules you want, like in OP.

Make sure you are allowing the traffic in both PF and IPFW firewalls.

How do we go about getting this integrated?

There is very old feature request, but the developers haven't seemed to be working on it.

https://redmine.pfsense.org/issues/2358

Maybe you could add this comment to the end of the feature request and see if it will bump it.

I do know that the unbound resolver has added a feature to turn on the DNS64 support in 2.5 roadmap.

dabombnl

@IsaacFL

Just went ahead and implemented this.

https://github.com/pfsense/pfsense/pull/4405

Have never tried to integrate code into pfSense before. Will see how it goes.

Napsterbater

@dabombnl

Thanks for this. I am hoping your work will make it into pfsense.

I would love to contribute, but coding and such is just not my forte, so even for submitting what you have you have my thanks whatever the outcome.

JeGr

Just FYI: the issue https://redmine.pfsense.org/issues/2358 got that pull request #4405 so it should go into review now.

mfld

@jegr said in Setup NAT64 in pfSense:

Just FYI: the issue https://redmine.pfsense.org/issues/2358 got that pull request #4405 so it should go into review now.

Unfortunately this didn't make it.

We will have to start all over.