Pfsense dropping traffic on 169.254.0.0/16 network

pedymaster

Hi,
We use pfsesne as a gateway for our network and we have ipsec tunnel setup to Amazon VPC. In the tunnel setup, there are two Phase II configurations. One is in subnet of 169.254.0.0/16 and the other is for our internal routing (subnet of 10.0.0.0/8). The first one is used in amazon for communication between pfsense and their routing system. To be specific, we have openbgp and this subnet was used for its communication. It all works nicely on 2.1.5 version but stopped with 2.2 and 2.2.1 where the IPsec was established successfully, I was able to ping the remote 169.254.x.x address but the communication of BGP was dropped by firewall. From changelog and little googling I found that this might be the issue

block IPv4 link-local/APIPA 169.254.0.0/16. Per RFC 3927, hosts "MUST NOT send the packet to any router for forwarding", and "any network device receiving such a packet MUST NOT forward it". FreeBSD won't route it (route-to can override in some circumstances), so it can't be in use as a real network anywhere with the possible exception of local-only networks. Unlikely any such situation exists anywhere #2073

Is this really the issue? Is there a way around it? Of course I will provide any additional information you might need.

Thank you for the help

doktornotor

Do not use APIPA in whacky ways. End of story.

ptt

^
Exactly !

https://forum.pfsense.org/index.php?topic=82472.msg451305#msg451305

pedymaster

Well, I have no choice, this setting is from Amazon VPC and I have no word in that - they decide this.

antillie

Then you are boned. The 169.254.0.0/16 range is not routable. Period. Amazon needs to give you a real IP range to use.

I set up IPSec VPN tunnels to AWS instances all the time (although not from pfSense) and I have never seen Amazon try to use the 169.254.0.0/16 range. If Amazon won't give you a routable IP range then you need to find a cloud provider that doesn't suck.

pedymaster

That's useful. Thank you very much for the clarification.

cmb

Amazon really shouldn't use 169.254 for this purpose, rather CGN space would be preferable, but there are a number of systems out there using VPC VPNs where this block will break things. I added a hidden config option to disable this for 2.2.2 and newer. Setting <no_apipa_block>between <system>and</system> in the config will omit those blocking rules. Can add via PHP Execute in Diag>Command with:

$config['system']['no_apipa_block'] = true;
write_config();
```</no_apipa_block>

pedymaster

Thank you!

MV1908

Hi,

Sorry to reply to this old post, but i'm facing this problem too.

I am using pfSense as a frontend VM in the OVH public cloud (based on OpenStack). This pfSense has an IP on the public internet and an IP in a private VLAN where other VMs are.
The problem is that openstack (like Amazon EC2) is using 169.254.169.254 to give the VMs their initial parameters (private IPs, public keys, …) with cloud-init. So 169.254.169.254 is on the WAN side of pfSense.

I applied the no_apipa_block in the config like described, and added a firewall rule that corresponds, but i still cant ping the IP. The firewall log marks it as pass. I can ping it from the pfSense itself, but not from the private lan behind.

The pfSense version is
2.3.4-RELEASE (amd64)
built on Wed May 03 15:13:29 CDT 2017
FreeBSD 10.3-RELEASE-p19

Is this hack still valid for this version ?
Am i missing something ?

Thanks !

JKnott

Not much has changed in 2 years. The 169.254.0.0/16 addresses are still NOT routable. They are intended to be used only on the local LAN.

https://en.wikipedia.org/wiki/Link-local_address

kpa

There is no point in trying to route the traffic because it's purely intended to be used within a single network segment from host to host without the assistance of a router. IPv6 has the same kind of link-local addresses and they are explicitly not routable because they are scoped by interface.

On top of that IPv4 link-local addresses are not a completely official standard, implementations tend to do what they please with them and as with pfSense they are ignored and blocked completely.

Harvy66

I assume you can do transparent firewalling with pfSense with 169.254.0.0/16. I assume the issue is when using NAT that causes the packets to be "routed", which you can't do with 169.254.0.0/16.

JKnott

@kpa:

On top of that IPv4 link-local addresses are not a completely official standard, implementations tend to do what they please with them and as with pfSense they are ignored and blocked completely.

Actually, there is RFC 3927.
https://tools.ietf.org/html/rfc3927

cthomas

https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_VPN.html#VPNTunnels

They are using APIPA addresses inside of the VPN tunnel(s). Per the document, you can specify a size /30 CIDR block from the 169.254.0.0/16 range. The CIDR block must be unique across all VPN connections that use the same virtual private gateway.

The following CIDR blocks are reserved and cannot be used:

169.254.0.0/30

169.254.1.0/30

169.254.2.0/30

169.254.3.0/30

169.254.4.0/30

169.254.5.0/30

169.254.169.252/30

Additional it looks like they are using it for metadata server(s);

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

…ct

johnpoz

@JKnott:

Actually, there is RFC 3927.
https://tools.ietf.org/html/rfc3927

Since when did a RFC mean that some company couldn't do what they want how they want ;)

While yes its nice when everyone adheres to the RFC.. RFC is not gospel that is for sure..

For that matter.. Just look at all the people that run servers on the internet that don't bother to follow the rfc's - listing them was the whole point of rfc ignorant which has been replaced by http://rfc-clueless.org/

But yeah that RFC for sure calls out that 169.254 is local, not to route - its ttl is suppose to be 1, etc. So it not suppose to go anywhere.. ie you can not give a device an IP with that and expect it to be able to talk to anything outside that network from that address.

So its use a tunnel network should work - between 2 boxes sure… Not going to work as some road warrior IP in a vpn.. But as a tunnel sure..

kpa

Yeah, there's a great big gray area in IPv4 link-local addresses and that's the co-existence with the proper routable addresses (be it public or RFC1918 but still routable). Right now the only use for them is when a system fails to get a dynamic DHCP address from a DHCP server to have some kind of connectivity in the immediate network segment. As far as I know none of the implementations even try to use them as alias addresses alongside with the real addresses the same way IPv6 does.

kraduk

Quite old but also still relevant as it hasnt changed from the cloud providers pint of view and the thread is misleading from a modern context.

The above thread has a misconception of routable vs link local. Routable means that the ip block is propagated via routing protocols or made reachable via the router itself to anything connected to it. ie a client on a lan segment being able to reach a 169.254.0.0/16 address via the router not its own broadcast discovery over its local interface/

This is not the same as having a 169.254.0.0 /30 subnet on a vti interface. Only the two routers in the tunnel could see these addresses so it is therefore arguably compatible with rfc3927 as it is entirely on link. After all a vti interface is just a NIC, its just a virtual one over a tunnel vs a physical one.

More still it is a better address space to use than the CGNAT netblocks as these can still clash on internal networks especially when multiple organizations are at play, where as rfc3927 addresses could not clash unless the same address spaces were used on the same router. This would obviously be trivial to fix and totally preventable