DHCP Server fails after sometime- Filesystem full
-
I am running pfsense 2.0.2 with Captive Portal and FreeRadius2 using embedded 2G image on CF card. After several hours with no problems the device stops providing dhcp addresses. What i discover is that it's telling me that the filesystem is full. My DHCP lease is 86400 and the range is 172.16.0.10-172.16.255.250/16 subnet.
Please see the logs below. When i restart the system everything goes back to normal and all is working again. Is it a way to increase the database size or reduce the logs or what is written? I'm not sure why this is getting filled.
$ df -i Filesystem 512-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/ufs/pfsense0 1859358 689885 1020725 40% 16635 102499 14% / devfs 2 2 0 100% 0 0 100% /dev /dev/md0 78812 1424 71084 2% 136 5238 3% /tmp /dev/md1 118492 118388 -9372 109% 149 7785 2% /var /dev/ufs/cf 101055 2226 90745 2% 21 6505 0% /cf devfs 2 2 0 100% 0 0 100% /var/dhcpd/dev
Jan 22 09:43:05 kernel: pid 12591 (dhcpd), uid 1002 inumber 22 on /var: filesystem full Jan 22 09:43:08 kernel: pid 12591 (dhcpd), uid 1002 inumber 23 on /var: filesystem full Jan 22 09:43:16 kernel: pid 12591 (dhcpd), uid 1002 inumber 22 on /var: filesystem full Jan 22 09:43:31 kernel: pid 12591 (dhcpd), uid 1002 inumber 23 on /var: filesystem full Jan 22 09:44:24 kernel: pid 12591 (dhcpd), uid 1002 inumber 22 on /var: filesystem full Jan 22 09:48:53 kernel: pid 12591 (dhcpd), uid 1002 inumber 23 on /var: filesystem full Jan 22 09:48:58 kernel: pid 12591 (dhcpd), uid 1002 inumber 22 on /var: filesystem full Jan 22 09:49:06 kernel: pid 12591 (dhcpd), uid 1002 inumber 23 on /var: filesystem full Jan 22 09:49:23 kernel: pid 12591 (dhcpd), uid 1002 inumber 22 on /var: filesystem full Jan 22 09:54:38 kernel: pid 12591 (dhcpd), uid 1002 inumber 23 on /var: filesystem full Jan 22 09:54:42 kernel: pid 12591 (dhcpd), uid 1002 inumber 22 on /var: filesystem full Jan 22 09:54:49 kernel: pid 12591 (dhcpd), uid 1002 inumber 23 on /var: filesystem full Jan 22 09:55:05 kernel: pid 12591 (dhcpd), uid 1002 inumber 22 on /var: filesystem full
Jan 22 09:54:37 dhcpd: DHCPDISCOVER from 00:26:55:57:4b:14 (CGASWS9001) via em1 Jan 22 09:54:38 dhcpd: unexpected ICMP Echo Reply from 10.1.3.254 Jan 22 09:54:38 dhcpd: DHCPOFFER on 172.16.134.175 to 00:26:55:57:4b:14 (CGASWS9001) via em1 Jan 22 09:54:38 dhcpd: write_lease: unable to write lease 172.16.0.16 Jan 22 09:54:38 dhcpd: DHCPREQUEST for 172.16.134.175 (172.16.0.1) from 00:26:55:57:4b:14 (CGASWS9001) via em1: database update failed Jan 22 09:54:42 dhcpd: write_lease: unable to write lease 172.16.0.16 Jan 22 09:54:42 dhcpd: DHCPREQUEST for 172.16.134.175 (172.16.0.1) from 00:26:55:57:4b:14 (CGASWS9001) via em1: database update failed Jan 22 09:54:49 dhcpd: write_lease: unable to write lease 172.16.0.16 Jan 22 09:54:49 dhcpd: DHCPREQUEST for 172.16.134.175 (172.16.0.1) from 00:26:55:57:4b:14 (CGASWS9001) via em1: database update failed Jan 22 09:55:05 dhcpd: write_lease: unable to write lease 172.16.0.16 Jan 22 09:55:05 dhcpd: DHCPREQUEST for 172.16.134.175 (172.16.0.1) from 00:26:55:57:4b:14 (CGASWS9001) via em1: database update failed
-
The /var file system where DHCP leases are being written is clearly full. On an embedded system /var is a "RAM disk".
Your a parameters appear to allow a large number of concurrent leases. This MIGHT be beyond the capacity of the embedded system unless you are prepared to tweak it to significantly increase the size of the RAM disk used for /var.
A number of system logs are also written to /var, so it not just DHCP leases causing the problem.
-
You could try searching from the CLI for files larger than x, e.g. :
find /var -size +800k -print
-
i found out that it was the FR2 log daily log files that were filling up the space. I changed the logging form the radius.log files to system logs and this should resolve the issue. Also i increased the size of the /var filesystem in the rc.embedded to 700MB since i'm running 2 GB of RAM on the system.
I am going to replace the embedded image i have with a full pfsense image. i'm just waiting on the new industrial CF cards to arrive so i don't have to worry about my R/W limits. :)