DHCP v6 - no route to host

wallabybob · Mar 6, 2011, 5:56 AM

I remember reading somewhere that FreeBSD deliberately does not assign a link local address to a bridge interface because it is not a physical interface.

On thinking about this issue I wondered whether it would make a difference if the bridge had a link local address. In the absence of a link local address the send code for a bridge might decide that it needed to look up the routing table to find a way to get to the destination.

I added to the bridge a link local IPv6 address I calculated from the the bridge's assigned mac address (but I misread 'b' in the bridge MAC address as '6'):

ifconfig bridge0 inet6 add fe80::a242:06ff:fe44:fff2/64

ifconfig bridge0

bridge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
ether a2:42:0b:44:ff:f2
inet 192.168.211.173 netmask 0xffffff80 broadcast 192.168.211.255
inet6 2001:470:1f05:14b3::1 prefixlen 64
inet6 fe80::a242:6ff:fe44:fff2%bridge0 prefixlen 64 scopeid 0xe
nd6 options=3 <performnud,accept_rtadv>id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: vr0 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 3 priority 128 path cost 200000
member: ath0_wlan0 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 13 priority 128 path cost 370370
#</learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></performnud,accept_rtadv></up,broadcast,running,promisc,simplex,multicast>

Soon after the DHCP log started indicating the DHCP response completed successfully:

Mar 6 00:39:55 dhcpd: Solicit message from fe80::ec89:c46a:8e2f:4ad port 546, transaction ID 0x514AC200
Mar 6 00:39:55 dhcpd: Picking pool address 2001:470:1f05:14b3::16
Mar 6 00:39:55 dhcpd: Sending Advertise to fe80::ec89:c46a:8e2f:4ad port 546
Mar 6 00:39:55 dhcpd: send_packet6: No route to host
Mar 6 00:39:55 dhcpd: dhcpv6: send_packet6() sent -1 of 104 bytes
Mar 6 00:41:05 dhcpd: Solicit message from fe80::212:7bff:fe46:e765 port 546, transaction ID 0x1921C200
Mar 6 00:41:05 dhcpd: Picking pool address 2001:470:1f05:14b3::16
Mar 6 00:41:05 dhcpd: Sending Advertise to fe80::212:7bff:fe46:e765 port 546
Mar 6 00:41:05 dhcpd: Request message from fe80::212:7bff:fe46:e765 port 546, transaction ID 0x453A6600
Mar 6 00:41:05 dhcpd: Sending Reply to fe80::212:7bff:fe46:e765 port 546
Mar 6 00:41:23 dhcpd: DHCPREQUEST for 192.168.211.231 from 00:1c:bf:b8:96:1d via bridge0
Mar 6 00:41:23 dhcpd: DHCPACK on 192.168.211.231 to 00:1c:bf:b8:96:1d via bridge0
Mar 6 00:43:50 dhcpd: DHCPREQUEST for 192.168.211.191 from 00:03:47:81:cd:f7 via bridge0
Mar 6 00:43:50 dhcpd: DHCPACK on 192.168.211.191 to 00:03:47:81:cd:f7 via bridge0
Mar 6 00:46:00 dhcpd: Solicit message from fe80::ec89:c46a:8e2f:4ad port 546, transaction ID 0xC41D500
Mar 6 00:46:00 dhcpd: Picking pool address 2001:470:1f05:14b3::14
Mar 6 00:46:00 dhcpd: Sending Advertise to fe80::ec89:c46a:8e2f:4ad port 546
Mar 6 00:46:01 dhcpd: Request message from fe80::ec89:c46a:8e2f:4ad port 546, transaction ID 0xC41D500
Mar 6 00:46:01 dhcpd: Sending Reply to fe80::ec89:c46a:8e2f:4ad port 546

and my Linux netbook suddenly showed an address matching one of the DHCP assigned IPv6 addresses:

root@kogan:~# ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:12:7b:46:e7:65
inet addr:192.168.211.240 Bcast:192.168.211.255 Mask:255.255.255.128
inet6 addr: 2001:470:1f05:14b3:212:7bff:fe46:e765/64 Scope:Global
inet6 addr: fe80::212:7bff:fe46:e765/64 Scope:Link
inet6 addr: 2001:470:1f05:14b3::16/64 Scope:Global
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1202584 errors:0 dropped:0 overruns:0 frame:0
TX packets:1097997 errors:7478 dropped:7478 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1080459299 (1.0 GB) TX bytes:264067254 (264.0 MB)

There is an ifconfig option (eui64) that looks as if it might be useful in computing the link local address from the bridge MAC address but the man page doesn't give a usage example. I'll explore that and report back later.

databeestje · Mar 6, 2011, 3:26 PM

that should be easy enough to fix then. Before activating the dhcp server for v6 I could just activate that.

rjairam · Mar 8, 2011, 4:31 AM

Found it, this is why you don't have link local by default on bridges:

http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/netinet6/in6_ifattach.c

wallabybob · Mar 8, 2011, 5:15 AM

Sorry, I couldn't find an explanation in the revision history in the web page you linked to. Did you mean to link to a particular revision of that source file?

wallabybob · Mar 8, 2011, 5:41 AM

@wallabybob:

There is an ifconfig option (eui64) that looks as if it might be useful in computing the link local address from the bridge MAC address but the man page doesn't give a usage example. I'll explore that and report back later.

On reboot, my previously assigned link local address disappeared so there was an opportunity to try out the eui64 option:

# ifconfig bridge0
bridge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
	ether 86:8f:1f:f9:d3:c6
	inet 192.168.211.173 netmask 0xffffff80 broadcast 192.168.211.255
	inet6 2001:470:1f05:14b3::1 prefixlen 64 
	nd6 options=3 <performnud,accept_rtadv>id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: vr0 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 3 priority 128 path cost 200000
	member: ath0_wlan0 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 13 priority 128 path cost 370370
# ifconfig bridge0 inet6 add fe80::0/64 eui64
ifconfig: could not determine link local address
# ifconfig bridge0 inet6 add fe80::868f:1fff:fef9:d3c6/64
# ifconfig bridge0 
bridge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
	ether 86:8f:1f:f9:d3:c6
	inet 192.168.211.173 netmask 0xffffff80 broadcast 192.168.211.255
	inet6 2001:470:1f05:14b3::1 prefixlen 64 
	inet6 fe80::868f:1fff:fef9:d3c6%bridge0 prefixlen 64 scopeid 0xf 
	nd6 options=3 <performnud,accept_rtadv>id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: vr0 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 3 priority 128 path cost 200000
	member: ath0_wlan0 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 13 priority 128 path cost 370370
#</learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></performnud,accept_rtadv></up,broadcast,running,promisc,simplex,multicast></learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></performnud,accept_rtadv></up,broadcast,running,promisc,simplex,multicast>

so, looks as if the eui64 option is no use to generate a link local address from the "MAC" address of a bridge.

rjairam · Mar 8, 2011, 1:59 PM

@wallabybob:

Sorry, I couldn't find an explanation in the revision history in the web page you linked to. Did you mean to link to a particular revision of that source file?

http://lists.freebsd.org/pipermail/freebsd-net/2007-March/013835.html

rev 1.28
Add support for multicast to the bridge and allow inet6 addresses to be
assigned to the interface.

IPv6 auto-configuration is disabled. An IPv6 link-local address has a
link-local scope within one link, the spec is unclear for the bridge case and
it may cause scope violation.

An address can be assigned in the usual way;
ifconfig bridge0 inet6 xxxx:…

wallabybob · Mar 8, 2011, 2:43 PM

Thanks.

wallabybob · Mar 20, 2011, 9:06 PM

After updating to 2.0-RC1-IPv6 (i386)
built on Sun Mar 20 02:20:38 EDT 2011 I noticed in /var/log/system.log:

Mar 21 06:29:42 pfsense php: : The command ' bridge0 inet6 fe80::/64 eui64' returned exit code '127', the output was ''

and

Mar 21 06:41:53 pfsense dhcpd: send_packet6: No route to host
Mar 21 06:41:53 pfsense dhcpd: dhcpv6: send_packet6() sent -1 of 84 bytes

so I guess the eui64 option doesn't work on bridges. (I have a wired LAN and wireless LAN bridged together and DHCP server on the bridge.)

databeestje · Mar 20, 2011, 9:11 PM

That means I need to write a php function to perform that calculation. Sometime this century then.

Attempted fix committed.

rjairam · Mar 20, 2011, 11:38 PM

How is the MAC address of a bridge calculated anyway? Mine seems to change on every boot. Is it some random value?

wallabybob · Mar 20, 2011, 11:51 PM

@rjairam:

How is the MAC address of a bridge calculated anyway? Mine seems to change on every boot. Is it some random value?

From the FreeBSD bridge man page:

The if_bridge interface randomly chooses a link (MAC) address in the range reserved for locally administered addresses when it is created. This address is guaranteed to be unique only across all if_bridge interfaces on the local machine. Thus you can theoretically have two bridges on the different machines with the same link addresses. The address can be changed by assigning the desired link address using ifconfig(8 ).

rjairam · Mar 21, 2011, 2:33 AM

Ah.

There is a utility called ipv6calc that can be used to generate EUI 64 link local addresses.

http://www.tunnelbroker.net/forums/index.php?topic=751.0

In reality I've been able to get away with any random nonsense and it seems to work. It may not be right but so is assigning a link local to a bridge.

databeestje · Mar 21, 2011, 8:10 AM

Calculating the eui64 from the existing hardware address isn't too bad with a bad of calculus and string replacement. I just have not tried reproducing it yet.

MrKoen · May 2, 2011, 9:36 AM

@wallabybob:

I was feeling brave and enabled DHCP6 on my pfSense. After some futzing around with firewall rules and client software I finally got a DHCP Solicit to pfSense dhcpd which wasn't able to process the solicit.

I still can not get the DHCPv6 service to work on my pfSense box. Can't figure out why. All it lists in the logs is:

May 2 11:23:38 dhcpd: Sending on Socket/14/em0/2001:470:xxx:51::/64
May 2 11:23:38 dhcpd: Listening on Socket/14/em0/2001:470:xxx:51::/64
May 2 11:23:38 dhcpd: Bound to *:547
May 2 11:23:38 dhcpd: Wrote 0 leases to leases file.
May 2 11:23:38 dhcpd: For info, please visit https://www.isc.org/software/dhcp/
May 2 11:23:38 dhcpd: All rights reserved.
May 2 11:23:38 dhcpd: Copyright 2004-2010 Internet Systems Consortium.
May 2 11:23:38 dhcpd: Internet Systems Consortium DHCP Server 4.1.1-P1

DHCPv4 works fine, however DHCPv6 requests never arrive at or get logged by pfSense. Since you mention in your post you had to play with firewall rules to get it to work, I was wondering what exactly did you set in your rules? Possibly that's the issue here as well.

MrKoen · May 3, 2011, 11:48 AM

One step further.. it was indeed a firewall rule that blocked the DHCPv6 requests. I have an alias which specifies the IPv4 and IPv6 ranges on my local network which are allowed to send traffic through pfSense. From the logs posted here I noticed the DHCPv6 requests come from fe80::/64 addresses. So I added that to my alias specifying my local network addresses and the DHCP logs started to report the DHCPv6 requests and give out leases.

The problem I'm facing now is that my client does get an IPv6 lease from pfSense, but it doesn't request the gateway and it assumes the DHCPv6 address to have a CIDR of 128. So it can not connect to the outside world and can not reach other IPv6 hosts within the same /64 address space.

MrKoen · May 3, 2011, 12:52 PM

I think I have found the problem.

It is caused by this change about a week ago:
https://rcs.pfsense.org/projects/pfsense/repos/pfSense-smos/commits/753bd64dffb8352ea273c1bf1f00aa099ec38634

The raflags for the rtadvd router discovery were changed from a numeric format to a letter in the /etc/inc/services.inc file. The translation for the "assist" mode wasn't correct though. The 192 number has been replaced by an O while this should be a M and an O. This will cause my Windows 7 client to discover the pfSense router first and get the assignment from it to request an IPv6 address via a DHCP broadcast resulting in both a gateway and an IPv6 lease. Currently since it only gives the O assignment, it only uses DHCPv6 to discover DNS services on the network, but not retrieve a DHCPv6 address. This causes it to generate its own address based on the address space the gateway is in.

Documentary supporting this thought: http://blogs.technet.com/b/teamdhcp/archive/2009/03/03/dhcpv6-understanding-of-address-configuration-in-automatic-mode-and-installation-of-dhcpv6-server.aspx from which I qoute:

Both M and O Flags are Set to 1. DHCPv6 is used for both addresses and other configuration settings. This combination is known as DHCPv6 stateful, in which DHCPv6 is assigning stateful addresses to IPv6 hosts.

And actually the /etc/inc/services.inc file also contains a comment supporting this:

/* raflags, other o, managed=64 m, stateful=128, both=192 */

@Databeestje, could you have a look at this and change it in the code if you confirm my suspicion to be correct?

I changed my DHCPv6 operating mode to Managed /services_dhcpv6.php to see what that would do and now my pfSense installation seems to have gotten corrupt. I'm getting loads of the following messages in my logs:

dhcpd: /var/db/dhcpd6.leases line 834: corrupt lease file; expecting ia_na contents, got ' '

When starting the dhcpd deamon it actually kills the whole webinterface and will eventually result in the following message in my browser:

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 24 bytes) in /etc/inc/util.inc on line 905

When I run killall dhcpd through the console, the webinterface comes back up. Even trying to set it back to operating mode Assisted will not get it working anymore. Also the mentioned file /var/db/dhcpd6.leases file does not exist so I have no idea why it tries to read line 834 from it.

Does anyone have a clue how I can fix this again?

MrKoen · May 3, 2011, 1:06 PM

I got the dhcpd deamon back up again. I simply went to Diagnostics -> Edit file -> /cf/conf/config.xml -> Load and replaced the <lan>contents within <dhcpdv6>with an empty set of nodes copied from a NIC on which DHCPv6 was not enabled. Now the dhcpd deamon starts again. However, when I try to enable the DHCPv6 service again in any operation mode, the problem returns. So there is a serious bug in there somewhere which once triggered will eliminate the use of the DHCPv6 service.</dhcpdv6></lan>

databeestje · May 3, 2011, 4:32 PM

Now I know how to reproduce the out of memory issue. I'll have a look later this evening.

MrKoen · May 3, 2011, 9:03 PM

I saw you have adjusted the raflags accordingly. Just gitsynced and ran some tests and it works like a charm now! Only thing I can't get to work yet is the DHCPv6 reservations. Reservations are simply ignored and an address from the address pool is provided. Is this a thing you're still working on or should it work and do I need to play with it some more to try to get it to work?

MrKoen · May 3, 2011, 9:13 PM

Actually what I can not do anymore with the latest version is let pfSense send out the advertisements through rtadvd and let my Windows DHCPv6 hand out the IPv6 addresses as it used to do (since I have the IPv6 reservations working on my Windows server). For it to work I need pfSense to advertise itself using the MO flags, but when I configure pfSense in Assisted mode, it automatically also enables its own DHCPv6 service thus causing a conflict on the network with pfSense which is also giving out addresses. Ideally it should be that I can always choose an operating mode on the DHCPv6 page in pfSense, but only when I check the box to enable the DHCPv6 server, it will also enable the DHCPv6 service. Sounds rather logical too if you think about it. Is this a thing you agree on Databeestje and can realize? I can change the Javascript on the page to allow this if you want.