Add users to FreeRadius USERS file without restart



  • I am trying to do have the usernames for freeradius package generated from a proprietary software that we use and updated into the pfsense box automatically. Our software can insert the file into the pfsense firewall with STP connection and replace the USERS file which freeradius uses. Now what I want to know is if this is the best method? and if there is a way to make the freeradius server update the USERS file without needing to restart the service after each modification.

    This is for a hotel hotspot and so each time we sell a code it will need to be restarted and knock the connected users off. I was trying to see a better way of doing this.

    Any feedback will be appreciated.



  • If you change something on any freeradius file it will not take effect until freeradius service was restartet.
    So you need to restart the service or it will not work.

    If you problem is that the GUI will overwrite the file you copied on pfsense then you can modify the freeradius.inc file:
    edit /usr/local/pkg/freeradius.inc on line 571
    The is a line called:

    file_put_contents($filename, $conf);
    

    Remove this line or uncomment it. This is the part which writes the file.

    edit /usr/local/pkg/freeradius.inc on line 576
    The is a line called:

    restart_service('radiusd');
    

    Remove this line or uncomment it. This is the command which restarts the service. But remember if you remove it there you need to do it somewhere else. There is now way around to restart the service.



  • Nachtfalke thanks for the response and sorry it took me so long to look at this i was out of the country for all of december. I will try these change to line 571 to disable the update from the GUI.

    Do you know if there is a limit on the number of lines on the USERS file? Can i have 10 000 users?

    Also i manage to update the USERS file with WINSCP replacing the current one and then have a script that runs and restarts the server for it to apply the changes. Is this the best way to do this? I am just trying to see if there is another way or if this is alright how i'm doing it now.

    It will only run maybe once a month if that sometimes once every 2 months.



  • @fsantaana:

    Nachtfalke thanks for the response and sorry it took me so long to look at this i was out of the country for all of december. I will try these change to line 571 to disable the update from the GUI.

    Do you know if there is a limit on the number of lines on the USERS file? Can i have 10 000 users?

    Also i manage to update the USERS file with WINSCP replacing the current one and then have a script that runs and restarts the server for it to apply the changes. Is this the best way to do this? I am just trying to see if there is another way or if this is alright how i'm doing it now.

    It will only run maybe once a month if that sometimes once every 2 months.

    Managing the file this way should work. Of course the file formatting must be correct ;-) Make sure the freeradius server has the permission to read the file after you copied it. And just remember - changes on the users file will first take effect if you restart freeradius. When you do that and which way is up to you :-)

    10.000 users. This is big number of users.
    If you are just using "username" and "password" it could work if you have up to date hardware. But more it depends on how ofthen users authenticate and how many users are doing that at the same time. So you can have 1 million users in that file - if there is only one authenticating every 10s probably no problem. but if you have 100 users and they all try to authenticate in 1 or 2 seconds - this could be very much.

    So you should test this - do not set the timeouts to low. Increase the threads a little bit and try.
    But probably a database like mysql or postgreql would be better.

    If you are doing accounting there will be no way around a database as backend.

    HINT: You can do authentication using the users file and do accounting to a database. Thats no problem.

    If you have other attributes like VLAN-ID, Session-Timeout and so on - this will increase the usersfile fast and 10.000 with ~4 lines - thie file then counts 40.000 lines.

    Here are some references even if they are based on older freeradius versions. rlm_fastusers is todays just "users" by default.
    http://freeradius.org/testimonials.html
    Jeff Carneal - Apex Internet
    ~25k ppp users



  • @Nachtfalke:

    @fsantaana:

    Nachtfalke thanks for the response and sorry it took me so long to look at this i was out of the country for all of december. I will try these change to line 571 to disable the update from the GUI.

    Do you know if there is a limit on the number of lines on the USERS file? Can i have 10 000 users?

    Also i manage to update the USERS file with WINSCP replacing the current one and then have a script that runs and restarts the server for it to apply the changes. Is this the best way to do this? I am just trying to see if there is another way or if this is alright how i'm doing it now.

    It will only run maybe once a month if that sometimes once every 2 months.

    Managing the file this way should work. Of course the file formatting must be correct ;-) Make sure the freeradius server has the permission to read the file after you copied it. And just remember - changes on the users file will first take effect if you restart freeradius. When you do that and which way is up to you :-)

    10.000 users. This is big number of users.
    If you are just using "username" and "password" it could work if you have up to date hardware. But more it depends on how ofthen users authenticate and how many users are doing that at the same time. So you can have 1 million users in that file - if there is only one authenticating every 10s probably no problem. but if you have 100 users and they all try to authenticate in 1 or 2 seconds - this could be very much.

    So you should test this - do not set the timeouts to low. Increase the threads a little bit and try.
    But probably a database like mysql or postgreql would be better.

    If you are doing accounting there will be no way around a database as backend.

    HINT: You can do authentication using the users file and do accounting to a database. Thats no problem.

    If you have other attributes like VLAN-ID, Session-Timeout and so on - this will increase the usersfile fast and 10.000 with ~4 lines - thie file then counts 40.000 lines.

    Here are some references even if they are based on older freeradius versions. rlm_fastusers is todays just "users" by default.
    http://freeradius.org/testimonials.html
    Jeff Carneal - Apex Internet
    ~25k ppp users

    wow 25k that is a huge number. I will look into changing the configuration for a MySQL backend but i was trying to avoid this since i was looking for an embedded system to host the cp/authentication. Just for clarification the accounting is what keeps the time of the authenticated users in the CP correct?



  • Accounting is for example counting the users time "Amount of Time" and the kick the user after he used his amount of - let's say - 60 minutes.
    Or if you give a user 2000MB to use per day. Simultaneous-Use checks on freeradius will be done by accounting. But you could do that easier with the CP - if you need it.

    So what you say is correct.

    The huge number of users:
    Try it. Probably there will never be so many users which authenticate at the same time and - if I remember correct - on CP there is an option how many users can visit the CP page simultaneously.

    Here is a tutorial which could allow you to test the performance on your hardware:
    http://doc.pfsense.org/index.php/FreeRADIUS_2.x_package#Test_FreeRADIUS_performance_with_jRadius
    if you did some tests - please feel free to share your experience - I will add this to the pfsense doc so that other people know what can be done with freeradius on pfsense :-)



  • Thanks for the info. The accounting is what i would need the most because i need to be able to pause the 1 hour limit by logging out. I don't think the CP time limit allows this to happen which is why i'm relying to the freeradius and users for accounting to assist with this.

    is there no way to get accounting with the USERS file? i will try various work arounds to see i will keep you posted on any progress.



  • Accounting without any database is possible.
    The accounting data will be stored in

    /var/log/radacct/datacounter/
    

    or

    /var/log/radacct/timecounter/
    

    Of course you can test if the hardware can handle that, then you do not need any mysql database.



  • Thanks for the info Nachtfalke. I set it all up and have it working. next week thrusday i will be testing with a large amount of users on the USERS file and i will let you know the results.

    However i ran into some problems with the time expiring before the set time and i saw that you have a bug report for this on redmine and various posted about the issue in the forums. I also read your document on FR2 pfsense DOCS and on there you state that the problem was resolved on 2.0.2 I upgraded to this version and i can confirm that the error still continues. Please see my reply in the post from Periko with the logs to see if this helps. http://forum.pfsense.org/index.php/topic,57303.0.html

    Is there a work around i can do for the moment? I tried interim accounting but the problem with that is it doesn't report back the Session Timeout so it remains the same until the user logs out. Also in my logs i don't see the session timeout decreasing once the user is logged on.



  • hi,

    this is a really annoying problem but I depends on the data CP is sending.

    In the post you mentioned there are some links to github pull requests from another user who did some changes on CP code. Did you try that ?

    Further did you try with "accountion stop/start" on CP ?

    And please try with freeradius –> settings "acct_unique" di-/enabled.

    Enable "Session-Timeout" on CP.

    Do not set any "Session-Timeout" on a user which should use the time feature.

    If the daily usage time has expired then do not try again with the same username as longs as you:

    • did not delete the db.daily database
    • mindnight is not over
      create another username

    Try to disable the CP hardt imeout and idle timeout



  • Hi,

    I implemented the pfsense with fr2 and freeradius list of 30k codes yesterday in our test hotel. The machine handles the number of users and codes with no problems and the cp page runs fine. There are just 2 things that i'm stuck with maybe you can help.

    1. the "users" file gets reset every time i restart the box. I generated a custom users file not through the gui and uploaded it with all the correct information and it works but when i restart the box it gets erased. Will the method you posted earlier altering freeradius.inc line 571 prevent this from happening?

    2. i've tried all the methods below and still having the problems with the code timing when using start/stop acct from the cp. I tried to remove all the idle/session timeouts from fr2 and cp page. Also removed th eoption to use the radius session time out and there are no session timeouts on the user.

    However what i did notice is that the first session time that is sent is not correct. it should send 60 second increments but the first time packet send 40 seconds and from there it throws the entire time off and it keeps increasing. So let's say that every time it should be 60 the first is 40 then the second is 100, 160, 220, etc. do you know where this is coded? maybe if we change the variable to a static number of 60 it will correct the problem? just an idea and what i noticed.

    Also with that the only way i can keep the code timer correct is if i switch to interim accounting but then the fr2 session time never decreases and thus hinders the valid time useless because it would never expire.



  • 1.)
    Remove or comment the two lines I posted in the posts before if you want to prevent that any process overwrites your "users" file.

    2.)
    Try the patch/changes from ermal or find the corresponding file and lines:
    http://redmine.pfsense.org/issues/2164

    In this patch ermal is reducing the time every accounting stop by 60 seconds.



  • i've never done the patch before and so what is the command i'll need to run to add the patch from that cp.diff? Sorry i'm not the greatest with these BSD commands but inside the files i can find my way around.

    I did comment out the lines and now all is ok with the users file.

    Thanks!



  • I am no BSD guy, too ;-)

    Find the file "captiveportal.inc"

    Probably modify the file by hand would be the easiest way:
    The lines with an  "-" in front need to be removed.
    The lines with an  "+" in front need to be added

    What the patch - as far as I understand it does is:
    It just substracts 60 seconds from the stop-time.

    $stop_time - 60,
    

    You said that the first accounting stop packet sends 40 seconds instead of 60. The reason could be this comment:

    /* XXX: *start time* Timer is static since prunce records is every 60secs. Max it needs to be configurable with prune records interval */
    


  • Wouldn't it still be easier to install the MySQL server by hand inside pfSense so you can leave the current sources the way they are? Setting up users from outside the pfSense GUI is so much easier using a "realtime" DB setup.



  • nachtfalke - i tried both those changes and everything is ok for the moment!! thanks for your help with this. While updating the captiveportal.inc from the redmine link i saw that you posted a fix for the rladump error that comes up on the logs. Where do you add the line exec("sleep 1");? thanks!

    If found a fix for an other "bug":

    rlm_radutmp: Logout for NAS CP port 76, but no Login record
    rlm_radutmp: Login entry for NAS CP port 76 wrong order

    This is happening because the accounting stop/start packets are to short after another. I added this line:

    exec("sleep 1");

    fesoj- would but i'm running this on a CF card on embedded device. Don't think it's too smart to use mysql on a CF card because of the amount of writes it's required. So far with these latest patches the USERS file of 30k users is working great no problems and loads really quick. It's relativily small 1.3MB. However if you have any other ideas of how i can improve this system i would be glad to hear them. I am just testing all of the solution at the moment to see if it will hold up before we deploy to our different sites.



  • You may be right. My hotel solutions are hosted inside Atom boxes with real disks, so this is not an issue here.

    On the other hand, maybe not. If the CF card is large enough and there is enough RAM, it might still work. Only interim updates require a lot of writes.



  • @Fesoj:

    (…)
    Only interim updates require a lot of writes.

    For CP on pfsense this is not correct because the updates - no matter if interim or stop/start will be send every 60 seconds. So in this case there will be no difference.

    Unfortunately there are some thing which pfsense CP does not support or does not support correctly according RADIUS protocol. But as far as I can see that, there are some improvements made on CP on the new version 2.1.

    @fsantaana
    You are right - on redmine I posted this exec("sleep 1"); command … but I opend these tickets in the past when I developed the freeradius2 package and tried to test these features. I am not using this at the moment so I need some time to remember where I added this line ;-)



  • Ok, I think I found the part where I probably added this in the past  ::)

    This is the part in the captiveportal.inc which generates the accouting stop/start packets.
    First part is accouting stop and then there is immediatly an accounting start. In most cases this is to short after another so I added this one second of sleep time. Not sure if there is any other value shorter than one second which can be used or not. This is the actual code on github:

    
    	if ($config['captiveportal']['reauthenticateacct'] == "stopstart") {
    /* stop and restart accounting */
    RADIUS_ACCOUNTING_STOP($cpentry[1], // ruleno
    $cpentry[4], // username
    $cpentry[5], // sessionid
    $cpentry[0], // start time
    $radiusservers,
    $cpentry[2], // clientip
    $cpentry[3], // clientmac
    10); // NAS Request
    exec("/sbin/ipfw table 1 entryzerostats {$cpentry[2]}");
    exec("/sbin/ipfw table 2 entryzerostats {$cpentry[2]}");
    RADIUS_ACCOUNTING_START($cpentry[1], // ruleno
    $cpentry[4], // username
    $cpentry[5], // sessionid
    $radiusservers,
    $cpentry[2], // clientip
    $cpentry[3]); // clientmac
    
    

    So probably adding the sleep between this two lines would help:

    
    exec("/sbin/ipfw table 2 entryzerostats {$cpentry[2]}");
    exec("sleep 1");
    RADIUS_ACCOUNTING_START($cpentry[1], // ruleno
    
    

    Remember:
    This will make accouting a little bit inaccurate over a long distance because you lose one second every minute which will not be counted. In theory you could try to use:

    $stop_time - 59,
    

    instead of:

    $stop_time - 60,
    

    Not sure if there are side effects or not.
    If you calculate one day which has 24*60 accounting stop/start packets you lose 24min or your customer wins 24min per day ;-)

    PS: If this will help you and fix your problem(s) - It would be nice to comment this on redmine what you changed, on which version of pfsense and what it fixed so that there will be hopefully a commit on future pfsense 2.0.x and pfsense 2.1.x versions.



  • Nachtfalke:

    whether writing to MySQL results in any disk activity is largely a matter which storage engine is used. There's none if the MEMORY engine is used, though this is vulnerable to various hardware and power problems. Maybe a temporary copy to a disk based format make this usable.

    I'll try to measure the disk activity on my boxes with iostat or s.th. similar to see whether FreeRadius on a pfSense box really triggers a lot of disk activity. I could report later about this.

    Except for the transferred number of octets FR basically works for my setups.



  • UPDATE Everything between the FR2 and the CP are working great. I am able to manage the users and they are now only decreasing 60 seconds at a time. I am thinking of increasing the stop time to 61 and adding the exec sleep 1 so it removes that rwadump error and still gives the customer the full time. If i got it right from your explanation that leaving it at 60 and adding the sleep the customer would lose 1 second of time so in a full day it would be 24 min. But what if we compensate with the 1 second in the stop time to 61 and then they would actual start again from 60. This is just theory so i'm not sure.

    Also a big problem i'm seeing now is the logging of the radius server is eating up all of the /var filesystem and it's causing my DHCPD to stop giving ip's since it can't write to the database. Is this somethign that you have seen before? I'm looking into reducing the logs since I don't need too much logging.

    Maybe there is a way to decrease the logging of FR2 or increase the /var filesystem size. The ideal would be to have a reset of the logs every 8 hours or so.

     Jan 22 17:02:46	kernel: pid 58856 (dhcpd), uid 1002 inumber 22 on /var: filesystem full
    Jan 22 17:02:49	kernel: pid 58856 (dhcpd), uid 1002 inumber 23 on /var: filesystem full
    Jan 22 17:02:56	kernel: pid 58856 (dhcpd), uid 1002 inumber 22 on /var: filesystem full
    Jan 22 17:03:11	kernel: pid 58856 (dhcpd), uid 1002 inumber 23 on /var: filesystem full
    Jan 22 17:03:34	kernel: pid 58856 (dhcpd), uid 1002 inumber 22 on /var: filesystem full
    Jan 22 17:04:01	kernel: pid 58856 (dhcpd), uid 1002 inumber 23 on /var: filesystem full
    Jan 22 17:08:33	kernel: pid 58856 (dhcpd), uid 1002 inumber 22 on /var: filesystem full
    
    Jan 22 17:02:45	dhcpd: DHCPDISCOVER from 00:21:5a:f7:76:67 (CGASWS9189) via em2
    Jan 22 17:02:45	dhcpd: unexpected ICMP Echo Reply from 10.1.3.254
    Jan 22 17:02:46	dhcpd: DHCPOFFER on 172.14.255.249 to 00:21:5a:f7:76:67 (CGASWS9189) via em2
    Jan 22 17:02:46	dhcpd: write_lease: unable to write lease 172.16.0.135
    Jan 22 17:02:46	dhcpd: DHCPREQUEST for 172.14.255.249 (172.14.0.1) from 00:21:5a:f7:76:67 (CGASWS9189) via em2: database update failed
    Jan 22 17:02:49	dhcpd: write_lease: unable to write lease 172.16.0.135
    
    
    $ df -h
    Filesystem           Size    Used   Avail Capacity  Mounted on
    /dev/ufs/pfsense0    908M    337M    498M    40%    /
    devfs                1.0K    1.0K      0B   100%    /dev
    /dev/md0              38M    632K     35M     2%    /tmp
    /dev/md1              58M     58M   -4.6M   109%    /var
    /dev/ufs/cf           49M    1.1M     44M     2%    /cf
    devfs                1.0K    1.0K      0B   100%    /var/dhcpd/dev
    
    $ du -k -h -d 1 /var
    2.0K	/var/.snap
    4.0M	/var/db
    6.0K	/var/tmp
     50K	/var/etc
     38K	/var/run
     50M	/var/log
    4.0K	/var/at
    2.0K	/var/empty
    3.3M	/var/dhcpd
    4.0K	/var/cron
     58M	/var
    


  • @fsantaana
    I think you are right. using 61 instead of 59 will make it accurate for the user.

    Logrotation:
    Possibilities are:
    a) Disable logging
    b) logging to syslog - should be cleared/limited by pfsense 2000 lines max - I think but not sure
    c) logging to external syslog server
    d) logging to /var/log/radiusd.log and create a small script which does a log rotation of the last 5.000 lines every night and run this with cron. Example could be the one from squidguard:

    
    #!/bin/sh
    # Rotates radiusd.log file
    tail -5000 /var/log/radiusd.log > /var/log/radiusd.log.0
    tail -5000 /var/log/radiusd.log.0 > /var/log/radiusd.log
    rm -f /var/log/radiusd.log.0
    
    


  • @nachtfalke

    I tried with the time to 61 and it's reducing an extra second. When I was doing the count i realized it doesn't matter if the sleep is 1 because it's only using the session time. So in reality it just takes an extra second for the account start to load and doesn't affect the session time which is what it uses. The point is that the sleep 1 correct the logging issue and that is now ok!

    I come to find what was keep all the space in /var/log is the detail-YYYYMMDD file in the /var/log/radacct/IP Is there a way to remove these? It's the detail of each accounting start/stop packet throughout the day and i have about 200 simultaneous connections during those days which leads to a ~20MB file each day.

    I'm trying to create a cron job that will erase the detail-YYYYMMDD that is older that 1 week. Do you have any suggestions on purging these old detail files? can i even purge them?

    Also is there a way to see the Session Timeout (time remaining) of the users without them being logged in? I know i can add a line to the logging that will show me the Session Timeout remaining but what if they are not logged in? Is there a way i can pull that info from there? Would this be outside the scope of the USERS file and more into SQL for accounting?

    Thanks



  • Hi,

    perhaps radwho command can give you some information:
    http://freeradius.org/radiusd/man/radwho.html

    For accounting - go to freeradius.inc a search this part:

    
    #
    #  Accounting.  Log the accounting data.
    #
    accounting {
    	#
    	#  Create a 'detail'ed log of the packets.
    	#  Note that accounting requests which are proxied
    	#  are also logged in the detail file.
    	detail
    
    

    Then comment the line with "detail" - this will probably disable logging of accounting information.
    A better way could be to modify this file to make the file rotation every week or something like that:
    http://freeradius.org/radiusd/man/rlm_detail.txt

    I suppose somewhere in ../raddb/modules/ is this file and we should modify it a little bit.

    Instead of this:

    
    %{radacctdir}/%{Client-IP-Address}/detail-%Y%m
    
    

    try something like that:

    
    %{radacctdir}/%{Client-IP-Address}/detail-%Y%m%d
    
    

    http://wiki.freeradius.org/config/Run%20time%20variables



  • Nacht thanks for all your help you are awesome!!

    I tried those changes in the detail file and it doesn't help me too much as it just generates new files. What i need is to flush the files as to avoid having such a large log sizes.

    I managed this with a simple cron job every week to delete all the detail files in /var/log/radacct/IP-Address/ this works for what i need and it's great.

    I have another question- Is there a way to set timeouts on individual users? I know there is the session-timeout that will force them to re-login at the specific time set. I was looking for something like an idle timeout per user instead of a global one in the CP page.

    Also i have done the testing as you said earlier and all signs look great! I have a USERS file with 25K Lines and the users are authenticated very quickly! Alot faster than what i expected. I haven't seen any performace issues with this. I always have approx 30-40 simultaneous connections at this current site and everything works great. I am now deploying to 2 other sites and will post the results from those. They will have heavier usage than the current one.


Log in to reply