Clients can't reconnect after pfsense reboot



  • Hi everybody.
    I just made a fresh test installation as pre-production pilot - 2.4.4-RELEASE (amd64).
    Configured everything (I'll use pfsense as captive portal) and everything worked perfectly as captive portal without user authentication (a simple splash screen with login button and no auth is enough for me).

    However, i soon discovered that after reboot clients can't reconnect anymore.
    To be more accurate, devices can still connect to the network, get a valid ip address, and their mac addresses are still listed in Captive Portal Status. But when a browser is opened client side a blank page appears with a tiny message ("You are connected"), and browser is stuck there, with no chance of browsing.
    To get it working again i have to manually disconnect user entries in Captive Portal Status. After that, connecting with the same device splash screen is normally displayed and clients can browse again.

    Any ideas about that? I'm thinking about some magic ipfw flushing happening at reboot, but i may be barking at the wrong tree of course.
    Thank you



  • Hi,

    You discovered that the pfSense database with connected clients was not flushed, but "ipfw", the captive portal firewall was reset.

    A "disconnect all portal clients" (and stop the portal during upgrade) before upgrading would have masked this issue.

    Anyway, your up and running again ;)



  • Sorry i did not understand, what should i do in order to make client re-connect safely after each reboot?

    Thank you



  • Click here :

    0_1538478460452_fe71cd30-cd74-47fe-9ef7-8d39aca6b542-image.png



  • Ok, this is what i'm actually doing to restore the service.
    But of course i can't do this each time pfSense is rebooted and have 300 clients blocked waiting for my action, so i was looking for a long term solution.

    Thank you



  • I have the exact same issue. Can someone help?



  • As there is no solution yet, I will try to install 2.3.5 and see if its better with that. I will post again for updates



  • @kengo i still can't understand what's going on. I tried to manually re-insert client IPs from sqlite (as they still exist there) in ipfw relevant tables (auth_up, auth_down) but they still can't reconnect.

    I must be missing something running under the hood



  • @prophet I'm trying 2.3.5 at this moment and I still have the same problem. If you turn off captive portal, does the problem persist? Mine doesn't so I think its a captive portal issue



  • @kengo I confirm that the issue is related to captive portal



  • @kengo if you dive deeper you will find that Captive portal status is fine after reboot (you can see authenticated clients, stats etc), while firewall is flushed. But based on what i see, it is not enough to recreate firewall rules in auth_down and auth_up tables (the only ones that seem to change when a client connect to captive portal) to make it work again, so there must be something else going on...



  • @prophet thanks for all the info. i actually have an older box running pfsense 2.4.1 and there are no issues so far with the captive portal. i will try to update this version of pfsense to 2.4.1 and see how it goes.



  • @kengo so with 2.4.1 everything works after reboot? did i get it right?



  • @prophet no, i was trying to upgrade the 2.3.5 version of pfsense to 2.4.1 (according to the dashboard) but what happened was it upgraded directly to 2.4.4 and still the same issue persists. i cannot get it to work. as soon as i turn on captive portal, the internet connection is lost.



  • @prophet said in Clients can't reconnect after pfsense reboot:

    @kengo I confirm that the issue is related to captive portal

    2.3.5 == 2.4.4 main difference is the code-base. The first is 32 bits - the latter 64 bits.
    So, totally normal that you found the same issue.

    The issue has a name and a number : https://redmine.pfsense.org/issues/8783



  • @gertjan sorry but this isn't the same issue.
    When pfSense is up i can save/edit anything without problems.

    I only have problems after reboot, with clients stuck at "You are connected" message in their browser.



  • @prophet said in Clients can't reconnect after pfsense reboot:

    @gertjan sorry but this isn't the same issue.
    When pfSense is up i can save/edit anything without problems.

    Can't tell what happens with 2.4.1 - that's old code and ditched because of "security issues".
    There is no such thing as a bug list "2.4.1". You're free to use it as long as you accept that product is unsupportable.
    So, again, ok to me ☺

    I only have problems after reboot, with clients stuck at "You are connected" message in their browser.
    And that's the situation right now with 2.4.4 and 2.3.5 (can't test that - have no 32 bits devices).

    And "Save" on the captive portal's setting will "redo" the ipfw firewall rules and tables. The captive portal's "connected client database" will not get emptied. This is what this issue is all about.



  • @gertjan never tried 2.4.1 and not planning to use it, i just asked @kengo if that version was ok.

    by the way bug #8783 is marked as "resolved", so it can't be the same issue. if it was i wouldn't be here :)



  • @prophet said in Clients can't reconnect after pfsense reboot:

    by the way bug #8783 is marked as "resolved", so it can't be the same issue. if it was i wouldn't be here :)

    You're right.
    When 8783 repaired something, this arrived https://redmine.pfsense.org/issues/8616 (other might exist).



  • Will be testing older versions of pfsense 2.2 and 2.3 tonight. i will post an update again.



  • @prophet I'm using 2.3.2 and its working like a charm so far. I will post again after 24 hours as I continue to monitor this



  • @kengo excellent!

    does it also "remember" clients/users across reboot or do they have to sign in again?



  • I'm curious too.
    I've been using 2.3.2 for a while, and I don't remember if users are all logged out. Normally, yes, they would have been.

    Most of the captive portal settings don't have anything to do with created entries in the related ipfw tables. Better yet : back then, there were no "tables" to hold the authorized devices, there were just rules.
    Only this one : "Per-user bandwidth restriction" ( Default download (Kbit/s) and Default upload (Kbit/s)) are used when creating rules.

    I can imagine that, when FreeRadius is used to restrict "bandwidth restriction" or "Amount of Download and Upload Traffic" is counted, and the rule (and related limiter/pipe) vanishes a moment for a device, things really start to break.
    So, saving the config == everybody has to start over. This is far more saver.

    Btw : Why should one want the captive portal setting regularly ? I didin't touch mine for weeks, if not months. Ones set up as needed, no need to change something.



  • hi everyone, sorry for the delayed response.

    I can confirm that 2.3.2 has resolved the captive portal issue on one box. but what i am curious is why on another box, 2.4.4 still works fine.

    @prophet upon reboot of the 2.3.2 machine, users need to login again. I think this is the intended behavior. captive portal works flawlessly now and doesn't kick machines off the internet after a period of time.

    as for my setup, I added all the mac address of my routers/access points to the Mac filter in captive portal, not sure if that helped.



  • @kengo said in Clients can't reconnect after pfsense reboot:

    as for my setup, I added all the mac address of my routers/access points to the Mac filter in captive portal, not sure if that helped.

    I did the same thing.
    Even basic access points could have NTP services (time keeping) or could need updates, so these should be able to communicate with the net.

    @kengo said in Clients can't reconnect after pfsense reboot:

    upon reboot of the 2.3.2 machine, users need to login again. I think this is the intended behavior

    Exact.

    @kengo said in Clients can't reconnect after pfsense reboot:

    captive portal works flawlessly now and doesn't kick machines off the internet after a period of time.

    by default, a captive portal should kick of user after a certain time (hard or soft time out).
    Except the ones listed on the MACs / Allowed IP / Allowed host names tabs



  • @prophet I have same problem :(



  • I have the exact same problem of OP @kengo which he mentioned in the 1st message. I cannot downgrade now. Is there a fix or workaround? Does setting an hard and idle time out fix the issue? Is there a way we can automatically log all users out of the captive portal every time the firewall reboots? Please advice. Thanks.



  • @darkblack to solve problems after reboot i just customized pfSense-rc script to automatically truncate captive portal sqlite3 database.

    Still looking for a way to automatically login devices after reboot, but i'm working with Fauxapi and i could be discovering something useful soon.



  • @prophet Thanks a lot for your response. Can you please tell me what piece of code you added and at what position (between which lines), it will be helpful and others who stumble on this rather frustrating issue



  • @darkblack

    in /etc/pfSense-rc, at the very end just before "exit 0".

    sqlite3 /var/db/captiveportalyourname.db <<EOF
    DELETE FROM captiveportal;
    EOF
    

    Where "yourname" is obviously the name of your captive portal.



  • Instead of deleting files - and editing pfSense core fils, I prefer to use the API.

    Consider this :
    Install the Shellcmd package which permit us to execute 'commands' at startup.

    Place a file called "captiveportal_disconnect_all.php" in the directory /root

    #!/usr/local/bin/php -q
    <?php
    	/* Disconnect all clients on all captive portal instances */
    
    	require_once("/etc/inc/util.inc");
    	require_once("/etc/inc/functions.inc");
    	require_once("/etc/inc/captiveportal.inc");
    
    	global $g, $config, $cpzone, $cpzoneid;
    
    	/* Are there any portals  ? */
    	if (is_array($config['captiveportal'])) {
    		/* For every portal (cpzone), do */
    		foreach ($config['captiveportal'] as $cpkey => $cp)
    			/* Sanity check */
    			if (is_array($config['captiveportal'][$cpkey])) 
    				/* Is zone enabled ? */
    				if (array_key_exists('enable', $config['captiveportal'][$cpkey])) {
    					$cpzone = $cpkey;
    					$cpzoneid = $cp['zoneid'];
    					captiveportal_disconnect_all();
    				}
    		log_error("All users disconnected after system start-up");
    	}
    ?>
    

    Add a command in the Shellcmd package :

    0_1542297148892_9003a358-315a-4155-805f-139923138a46-image.png
    Done.



  • @gertjan much better, thanks.
    do you have something similar to programmatically login users? given that we already have mac address, last ip, username.



  • @prophet said in Clients can't reconnect after pfsense reboot:

    do you have something similar to programmatically login users? given that we already have mac address, last ip, username.

    You were reading my mind ☺

    What you're asking for is the other way around : if a database exists with logged in user(s), why not rebuilding the ipfw rules for them ?!
    Right ?

    I guess it's possible.
    But : it's time out for me right now. I wrote the lines in the post above on a live system, throwning out users and restarting pfSense while testing : people are yelling all around me.
    I'll have a shot at it tomorrow.

    Basically, it should be a loop that iterates the database, and for every user

    function portal_allow($clientip, $clientmac, $username, $password = null, $attributes = null, $pipeno = null, $authmethod = null, $context = 'first') {
    

    should be called (see /etc/inc/captiveportal.inc, around line 2212)

    Except that this function does all the work (a lot of work !), and adds the user to the database ☹

    Keep in mind : firewall states and stuff like that will be gone.
    It should be something that is "voucher compatible", "Radius compatible", etc.



  • Thanks @prophet @Gertjan , will try the shellcmd fix today. Wondering, why this buy isn't fixed yet (officially) if it affects wide scale of users..



  • @darkblack I'm using the captive portal for years, but yesterday, during testing, I saw for the first time that connected users during a Diagnostics => Reboot are still listed as connected, but the related ipfw firewall rules were gone.
    My pfSense never reboots by itself - it has an UPS. I never saw the issue before.
    So, for myself, I can't call it an urgent matter. This issue exists only in the latest version or two.

    I guess the pfSense authors should wipe the sqlite file during reboot, as @prophet proposed.



  • I understand. But our case is where, the entire office powers down at EOD (UPS, lighting, et al) and powers up the next morning. . So a major headache for us. Eager to try the solution today.



  • @gertjan said in Clients can't reconnect after pfsense reboot:

    What you're asking for is the other way around : if a database exists with logged in user(s), why not rebuilding the ipfw rules for them ?!
    Right ?

    Yes sir.
    But which firewall rules are created for a succesfully authenticated client?
    I notice ipfw creates these rules, output filtered with relevant tables:

    --- table(captive_auth_up), set(0) ---
    172.17.17.11/32 18:65:90:82:81:c3 2004 181565 12852760 1542305056
    --- table(captive_auth_down), set(0) ---
    172.17.17.11/32 2005 418527 595361621 1542306032
    
    1. I tried to manually recreate these but i couldn't figure out how to make mac address based rules in ipfw.
    2. Aren't these rules just doing some traffic shaping? How are they critical to the system?

    Thank you



  • @prophet said in Clients can't reconnect after pfsense reboot:

    I tried to manually recreate these but i couldn't figure out how to make mac address based rules in ipfw.

    ipfw : you have 2 possibilities : the easy one : the manual. https://www.freebsd.org/cgi/man.cgi?ipfw(8)
    How pfSense makes rules is not hard to find : it all in /etc/inc/captiveportal.inc ^^

    @prophet said in Clients can't reconnect after pfsense reboot:

    I notice ipfw creates these rules, output filtered with relevant tables:
    --- table(captive_auth_up), set(0) ---
    172.17.17.11/32 18:65:90:82:81:c3 2004 181565 12852760 1542305056
    --- table(captive_auth_down), set(0) ---
    172.17.17.11/32 2005 418527 595361621 1542306032

    Exact.
    Here are the details https://www.netgate.com/docs/pfsense/captiveportal/captive-portal-troubleshooting.html

    @prophet said in Clients can't reconnect after pfsense reboot:

    Aren't these rules just doing some traffic shaping? How are they critical to the system?

    Exact. Pipes, (queues ?) are build for every connection.
    Because this option exists :
    0_1542367388504_884d9813-3b41-4ebc-8e92-d358b5f74fcc-image.png

    Pipes exists, wether you use them, or not.
    These pipes have to be created when entering the IP's of the client into the two tables myzone_auth_down and myzone_auth_up.