Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Seeing Kea DHCP Issues after upgrade to 24.11

    Scheduled Pinned Locked Moved DHCP and DNS
    25 Posts 7 Posters 1.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 3
      3aandl
      last edited by

      Decided to give Kea another shot after the upgrade to 24.11 on our 1541 but after doing so while things seemed ok last night, today I am seeing the DHCP logs flooded with the following and I am unable to pull a DHCP address on this network (VLAN). If I switch back to ISC everything works just fine without issue. This looks similar to the following bug in Redmine: https://redmine.pfsense.org/issues/15735 and https://redmine.pfsense.org/issues/15328 which states it was fixed in 24.11. Has anyone seen this or have any ideas? We literally just switched back to ISC with no other changes and everything works without issue again so definitely seems like a KEA issue.

      2024-11-26_18-37-48.png

      Nov 26 18:28:02 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 2c:aa:8e:63:dd:a3], cid=[01:2c:aa:8e:63:dd:a3], tid=0x18106178: Failed to allocate an IPv4 address for client with classes: ALL, VENDOR_CLASS_udhcp 1.33.1, pool_lan_0, pool_opt3_0, pool_opt2_0, pool_opt4_0, UNKNOWN
      Nov 26 18:28:02 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 2c:aa:8e:63:dd:a3], cid=[01:2c:aa:8e:63:dd:a3], tid=0x18106178: failed to allocate an IPv4 address after 101 attempt(s)
      Nov 26 18:28:02 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_SUBNET [hwtype=1 2c:aa:8e:63:dd:a3], cid=[01:2c:aa:8e:63:dd:a3], tid=0x18106178: failed to allocate an IPv4 lease in the subnet 192.168.3.0/24, subnet-id 2, shared network (none)
      Nov 26 18:28:02 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 ec:81:50:aa:a4:cb], cid=[01:ec:81:50:aa:a4:cb], tid=0x866909c8: Failed to allocate an IPv4 address for client with classes: ALL, pool_lan_0, pool_opt3_0, pool_opt2_0, pool_opt4_0, UNKNOWN
      Nov 26 18:28:02 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 ec:81:50:aa:a4:cb], cid=[01:ec:81:50:aa:a4:cb], tid=0x866909c8: failed to allocate an IPv4 address after 101 attempt(s)
      Nov 26 18:28:02 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_SUBNET [hwtype=1 ec:81:50:aa:a4:cb], cid=[01:ec:81:50:aa:a4:cb], tid=0x866909c8: failed to allocate an IPv4 lease in the subnet 192.168.3.0/24, subnet-id 2, shared network (none)
      Nov 26 18:28:00 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 e8:68:e7:1f:9c:e0], cid=[no info], tid=0x207bdfb4: Failed to allocate an IPv4 address for client with classes: ALL, pool_lan_0, pool_opt3_0, pool_opt2_0, pool_opt4_0, UNKNOWN
      Nov 26 18:28:00 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 e8:68:e7:1f:9c:e0], cid=[no info], tid=0x207bdfb4: failed to allocate an IPv4 address after 101 attempt(s)
      Nov 26 18:28:00 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_SUBNET [hwtype=1 e8:68:e7:1f:9c:e0], cid=[no info], tid=0x207bdfb4: failed to allocate an IPv4 lease in the subnet 192.168.3.0/24, subnet-id 2, shared network (none)
      Nov 26 18:28:00 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 a4:e5:7c:c5:57:70], cid=[no info], tid=0x961dc84e: Failed to allocate an IPv4 address for client with classes: ALL, pool_lan_0, pool_opt3_0, pool_opt2_0, pool_opt4_0, UNKNOWN
      Nov 26 18:28:00 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 a4:e5:7c:c5:57:70], cid=[no info], tid=0x961dc84e: failed to allocate an IPv4 address after 101 attempt(s)
      Nov 26 18:28:00 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_SUBNET [hwtype=1 a4:e5:7c:c5:57:70], cid=[no info], tid=0x961dc84e: failed to allocate an IPv4 lease in the subnet 192.168.3.0/24, subnet-id 2, shared network (none)
      Nov 26 18:28:00 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 70:31:7f:c3:6c:6b], cid=[01:70:31:7f:c3:6c:6b], tid=0x80049ece: Failed to allocate an IPv4 address for client with classes: ALL, pool_lan_0, pool_opt3_0, pool_opt2_0, pool_opt4_0, UNKNOWN
      Nov 26 18:28:00 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 70:31:7f:c3:6c:6b], cid=[01:70:31:7f:c3:6c:6b], tid=0x80049ece: failed to allocate an IPv4 address after 101 attempt(s)
      Nov 26 18:28:00 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_SUBNET [hwtype=1 70:31:7f:c3:6c:6b], cid=[01:70:31:7f:c3:6c:6b], tid=0x80049ece: failed to allocate an IPv4 lease in the subnet 192.168.3.0/24, subnet-id 2, shared network (none)
      Nov 26 18:27:59 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 2c:aa:8e:63:dd:a3], cid=[01:2c:aa:8e:63:dd:a3], tid=0x84d1813e: Failed to allocate an IPv4 address for client with classes: ALL, VENDOR_CLASS_udhcp 1.33.1, pool_lan_0, pool_opt3_0, pool_opt2_0, pool_opt4_0, UNKNOWN
      Nov 26 18:27:59 kea-dhcp4 18175 WARN [kea-dhcp4.alloc-engine.0x1aefa3e15f00] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 2c:aa:8e:63:dd:a3], cid=[01:2c:aa:8e:63:dd:a3], tid=0x84d1813e: failed to allocate an IPv4 address after 101 attempt(s)

      cmcdonaldC 1 Reply Last reply Reply Quote 1
      • cmcdonaldC
        cmcdonald Netgate Developer @3aandl
        last edited by

        Can you try deleting all leases and trying again?

        Need help fast? https://www.netgate.com/support

        3 1 Reply Last reply Reply Quote 0
        • 3
          3aandl @cmcdonald
          last edited by

          @cmcdonald OK, went in a deleted everything and it seemed ok for a very short amount of time and then right back to flooding the logs again with the same errors and unable to pull an IP address. I also checked and there are plenty of IP's available in the pool.

          2024-11-26_23-18-14.png

          cmcdonaldC 2 Replies Last reply Reply Quote 0
          • cmcdonaldC
            cmcdonald Netgate Developer @3aandl
            last edited by cmcdonald

            @3aandl can you run at Diagnostics > Command Prompt

            echo '{"command":"config-get"}' | nc -U /var/run/kea4-ctrl-socket | jq

            and

            echo '{"command":"lease4-get-all"}' | nc -U /var/run/kea4-ctrl-socket | jq

            Need help fast? https://www.netgate.com/support

            1 Reply Last reply Reply Quote 0
            • cmcdonaldC
              cmcdonald Netgate Developer @3aandl
              last edited by

              @3aandl This sort of seems like there is a misbehaving client on the network that is exhausting the pool, resulting in an unintentional (or intentional....) DoS attack on DHCP services. When Kea receives a DISCOVER it will soft-allocate a lease and wait for the client to send a REQUEST before transitioning to a hard allocation. However, during this window those addresses are unavailable. There is likely some tuning that we can do to mitigate that, but it would also be helpful to confirm that this is indeed was is happening.

              Need help fast? https://www.netgate.com/support

              3 1 Reply Last reply Reply Quote 0
              • 3
                3aandl @cmcdonald
                last edited by

                @cmcdonald I will reenable and run the command as soon as I’m back onsite. Do you feel it is just a client that doesn’t like the way Kea hands the addresses out? I tested with laptops and two iPhones (8 & 15 ProMax) and they would not pull an address but literally just going in and ticking ISC and applying and everything works as normal?

                3 1 Reply Last reply Reply Quote 0
                • 3
                  3aandl @3aandl
                  last edited by

                  @3aandl Reenabled and ran the command. I have the output but did lnow if I should attach the entire thing as they are both quite long.....here is a new snippet that shows it runs ok for a bit after restarting but then starts throwing errors and can't pull an IP. In this case testing with an Ipad. Can pull from one network but move over on wireless to another and no go. Again, turn off KEA and turn on ISC and all is good again with no issues. I also tested this with multiple client etc. and there are still at least 25+ IP's left in the pool to hand out.

                  2024-11-27_19-36-49.png

                  M 1 Reply Last reply Reply Quote 0
                  • M
                    marcosm Netgate @3aandl
                    last edited by

                    @3aandl Yes, please include the output of the commands. You may upload a text file as well.

                    3 1 Reply Last reply Reply Quote 0
                    • 3
                      3aandl @marcosm
                      last edited by

                      @marcosm pfkea1.txt pfkea2.txt

                      1 Reply Last reply Reply Quote 0
                      • Y
                        ysam
                        last edited by

                        Hello,

                        We have the exact same issue. After upgrading to 24.11 dhcp on a specific vlan refuses to allocate an IP. Even though we specifically assigned an IP to a mac address, still same issue.

                        Dec 3 15:10:43	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b25d8700] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: Failed to allocate an IPv4 address for client with classes: ALL, VENDOR_CLASS_MSFT 5.0, pool_opt4_0, pool_opt9_0, pool_opt3_0, pool_opt1_0, pool_opt6_0, pool_opt8_0, pool_lan_0, pool_opt5_0, pool_opt2_0, pool_opt7_0, pool_opt11_0, pool_opt10_0, UNKNOWN
Dec 3 15:10:43	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b25d8700] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: failed to allocate an IPv4 address after 2 attempt(s)
Dec 3 15:10:43	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b25d8700] ALLOC_ENGINE_V4_ALLOC_FAIL_SUBNET [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: failed to allocate an IPv4 lease in the subnet 192.168.0.0/24, subnet-id 11, shared network (none)
Dec 3 15:10:43	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b2418200] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: Failed to allocate an IPv4 address for client with classes: ALL, VENDOR_CLASS_MSFT 5.0, pool_opt4_0, pool_opt9_0, pool_opt3_0, pool_opt1_0, pool_opt6_0, pool_opt8_0, pool_lan_0, pool_opt5_0, pool_opt2_0, pool_opt7_0, pool_opt11_0, pool_opt10_0, UNKNOWN
Dec 3 15:10:43	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b2418200] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: failed to allocate an IPv4 address after 2 attempt(s)
Dec 3 15:10:43	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b2418200] ALLOC_ENGINE_V4_ALLOC_FAIL_SUBNET [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: failed to allocate an IPv4 lease in the subnet 192.168.0.0/24, subnet-id 11, shared network (none)
Dec 3 15:10:27	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b2417b00] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: Failed to allocate an IPv4 address for client with classes: ALL, VENDOR_CLASS_MSFT 5.0, pool_opt4_0, pool_opt9_0, pool_opt3_0, pool_opt1_0, pool_opt6_0, pool_opt8_0, pool_lan_0, pool_opt5_0, pool_opt2_0, pool_opt7_0, pool_opt11_0, pool_opt10_0, UNKNOWN
Dec 3 15:10:27	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b2417b00] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: failed to allocate an IPv4 address after 2 attempt(s)
Dec 3 15:10:27	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b2417b00] ALLOC_ENGINE_V4_ALLOC_FAIL_SUBNET [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: failed to allocate an IPv4 lease in the subnet 192.168.0.0/24, subnet-id 11, shared network (none)
Dec 3 15:10:27	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b2417b00] ALLOC_ENGINE_V4_ALLOC_FAIL_CLASSES [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: Failed to allocate an IPv4 address for client with classes: ALL, VENDOR_CLASS_MSFT 5.0, pool_opt4_0, pool_opt9_0, pool_opt3_0, pool_opt1_0, pool_opt6_0, pool_opt8_0, pool_lan_0, pool_opt5_0, pool_opt2_0, pool_opt7_0, pool_opt11_0, pool_opt10_0, UNKNOWN
Dec 3 15:10:27	kea-dhcp4	69034	WARN [kea-dhcp4.alloc-engine.0x289b2417b00] ALLOC_ENGINE_V4_ALLOC_FAIL [hwtype=1 06:91:00:80:e0:85], cid=[01:06:91:00:80:e0:85], tid=0x61e91a5e: failed to allocate an IPv4 address after 2 attempt(s)

                        kea-config.txt

                        3 1 Reply Last reply Reply Quote 0
                        • 3
                          3aandl @ysam
                          last edited by

                          @ysam Yeah, we switched back to ISC and everything runs just fine with no issues. We have messed with this on and off on several firewalls over the past week and we consistently see the same issues. Clear out the leases, things run ok for a bit and then the logs are flooded with what you are seeing and a number of devices quit pulling DHCP. We merely switch back to ISC and everything works right away and continues working with no errors and/or issues. Seems like there are still some issues on the implementation of KEA

                          C 1 Reply Last reply Reply Quote 0
                          • C
                            Cylosoft
                            last edited by

                            I came here to post a Kea issue. Seems to be along these lines. I don't have good info and can't get good info because it caused problems and we had to get back to ISC asap.

                            DHCP ran great for a couple of days. Had a VM with a static IP assigned by MAC that rebooted and when it did it was assigned a new IP from the pool. In the DHCP Leases Status page it showed the same MAC on the assigned IP and the new IP from the pool. We rebooted the VM several more times and it kept assigning the next IP on each reboot. So this MAC had 4 IPs. We went back to ISC and the issue went away.

                            1 Reply Last reply Reply Quote 0
                            • C
                              Cylosoft @3aandl
                              last edited by

                              @3aandl Yeah we had planned to switch all devices to Kea. Got 6 in and ended up rolling 4 of them back to ISC so we stopped the project.

                              Biggest complaint I got was that it was re-IPing the entire network. It doesn't respect leases given out by ISC already. Then we ran into the static IP not being respected and we had to hit the brakes.

                              GertjanG 3 2 Replies Last reply Reply Quote 0
                              • Y
                                ysam
                                last edited by

                                Problem is we cannot switch to ISC without huge pain!
                                We have 11 Vlans all with /24 dhcp space... so we will suffer a lot to do that.

                                Any luck for a fix?

                                3 1 Reply Last reply Reply Quote 0
                                • GertjanG
                                  Gertjan @Cylosoft
                                  last edited by

                                  @Cylosoft said in Seeing Kea DHCP Issues after upgrade to 24.11:

                                  Then we ran into the static IP not being respected

                                  I have about 50 of these :

                                  4f42c65d-6466-4b2a-ad86-1156a4ae0aaa-image.png

                                  and all my device still get the same IP.

                                  @Cylosoft said in Seeing Kea DHCP Issues after upgrade to 24.11:

                                  Biggest complaint I got was that it was re-IPing the entire network. It doesn't respect leases given out by ISC already.

                                  AFAIK, kea doesn't use the dhcpleases storage file ISC created.
                                  But, as the IP pool is probably (right ?) the same, and say a device using a lease like IP 192.168.1.10 right now and wants to renew 192.168.1.10,- knowing that that at that moment, 192.168.1.10 hasn't been given away to some one (as kea just started), it will give 192.168.1.10 to 192.168.1.10

                                  No "help me" PM's please. Use the forum, the community will thank you.
                                  Edit : and where are the logs ??

                                  cmcdonaldC C 2 Replies Last reply Reply Quote 0
                                  • cmcdonaldC
                                    cmcdonald Netgate Developer @Gertjan
                                    last edited by

                                    @Gertjan said in Seeing Kea DHCP Issues after upgrade to 24.11:

                                    AFAIK, kea doesn't use the dhcpleases storage file ISC created.

                                    dhcpleases will die along with ISC DHCPD. It is not used with Kea.

                                    Need help fast? https://www.netgate.com/support

                                    3 1 Reply Last reply Reply Quote 0
                                    • C
                                      Cylosoft @Gertjan
                                      last edited by

                                      @Gertjan Yeah static mappings work. Except when they don't. This particular network has 91 static mappings. 90 of them had no issue. The VM with an issue rebooted once and got it's correct static IP back, then the next reboot it got one from the pool. Then I forced several more reboots and it kept pulling the next pool IP. I actually deleted the static mapping and recreated, rebooted and it pulled the static, then I rebooted again and it got one from the pool. I should have taken a screenshot with the same MAC having a bunch of IPs consumed, but it was causing issues so no time for that.

                                      After switching from ISC to Kea I fully expected Kea to give out the same lease when the client did a renewal request. On the 6 networks we did it was always that every renewal request started from the first available on the pool and went up. So on every network we had issues with duplicate IPs. Then we had to cycle switches and APs to get that quickly cleaned up. Then you get a few people doing things by IP that shouldn't be "but the IP never has changed in years". So complaints about the entire network being re-IP'd.

                                      I agree it should be if we had 192.168.1.10 to 192.168.1.100 as a pool and the client requests 192.168.1.80 as a renewal it gets 192.168.1.80 if that's not already used. But Kea forces it down to 192.168.1.10. So then you get a duplicate IP issue for a bit.

                                      It happened twice and I actually told my guy he must be wrong because I was sure ISC would have given out the requested IP again and no way would Kea be setup to not do that. I switched 4 of the networks myself and saw it every time.

                                      1 Reply Last reply Reply Quote 0
                                      • 3
                                        3aandl @Cylosoft
                                        last edited by

                                        @Cylosoft Same here, as we dug in, we saw several times where leases that were assigned elsewhere were given to another device etc. The crazy part is there were still IP's available in the pool when it started flooding the logs and no longer handing out addresses. In most cases KEA seemed to hand out the next IP in the pool to a device and if it is rebooted etc. it would pull the next one so it seemed like one MAC may have 4-5 IP's etc. tied to it.

                                        1 Reply Last reply Reply Quote 0
                                        • 3
                                          3aandl @cmcdonald
                                          last edited by

                                          @cmcdonald Don't they use "Host Reservations" in KEA which from what I understand are basically the same thing or am I not understanding this correctly?

                                          https://kb.isc.org/docs/what-are-host-reservations-how-to-use-them

                                          GertjanG 1 Reply Last reply Reply Quote 0
                                          • GertjanG
                                            Gertjan @3aandl
                                            last edited by

                                            @3aandl said in Seeing Kea DHCP Issues after upgrade to 24.11:

                                            https://kb.isc.org/docs/what-are-host-reservations-how-to-use-them

                                            That's what my /usr/local/etc/kea/kea-dhcp4.conf shows :

                                                            "reservations": [
                    {
                        "hw-address": "00:4e:01:ca:ca:9c",
                        "ip-address": "192.168.1.2",
                        "hostname": "bureau2"
                    },
                    {
                        "hw-address": "ac:15:a2:42:b0:0b",
                        "ip-address": "192.168.1.3",
                        "hostname": "TL-SG108E"
                    },
                    {
                        "hw-address": "00:15:71:f6:ce:77",
                        "ip-address": "192.168.1.4",
                        "hostname": "poweredget310"
                    },
.......

                                            No "help me" PM's please. Use the forum, the community will thank you.
                                            Edit : and where are the logs ??

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.