Captive portal ignoring MACs in latest version and allowing all machines access
-
Hi,
Don't worry, this must be some minor issue : pfsense diagnose debug captive portal and take the first link.
You'll find some magic commands ^^
edit : great : free4 already said essentially the same thing.
Btw : I'm using the captive portal myself, and even some "MAC pass" addresses so my AP's can talk to the net through the portal - works great - always worked great.
There were no portal changes during the last p1-p2-p3 versions .... -
@free4 said in Captive portal ignoring MACs in latest version and allowing all machines access:
could you go to diagnosis -> command prompt, and execute the two following commands then paste here anonymized results?
ipfw table all list
ipfw pipe listresults for two requested commands, thank you for your help.
ipfw_table_all_list.txt
ipfw_pipe_list.txt -
@Gertjan said in Captive portal ignoring MACs in latest version and allowing all machines access:
Hi,
Don't worry, this must be some minor issue : pfsense diagnose debug captive portal and take the first link.
You'll find some magic commands ^^
That's a pretty condescending and useless response.
I've been using pfSense for many years, since version 1. The only thing that changed is an upgrade. Configuration did not change. This is very unhelpful. -
@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
That's a pretty condescending and useless response.
I did not meant to be like that.
I did not saw the message that @free4 free posted just before me, So I need info - and informed you a very help full help page that permit to troubleshoot issues ... ( and ok, how to fund it - bookmark it - it is usefull )
Your question didn't permit me to find any answers just be reading your the description of the problem.
ipfw rules are very useful to rule out many issues - but not all.
I threw myself of my captive portal I use myself (hotel - like you, since 1.1
) - and added the MAC of my Phone to the MAC list as "Blocked".
I tried to login again.
I received a red text (thus the portal error page) : "The MAC address has been blocked".Using a "Blocked MAC address redirect URL" worked also for me (the site was on Internet, so I whitelisted the URL/IP first).
Important to know is that blocked MAC's are not listed in some rule or table of ipfw. They are being put in an PHP array, and compared with the actual client who "knocks on the portal". When there is a match, the portal bails out with an error. Show the MAC blocked text.
Question : are you sure all devices use their MAC against the portal - check this with the DHCP server log.
If someone put a router between pfSense and the rest of the captive portal network, then that would explain a lot. pfSense would be seeing the same MAC for all devices.
On the other hand, the "xxxx_pipes_mac" table is loaded with 366 different MAC address. This table contains the list with "allowed" MAC addresses. Quiet a list ....When you check the config.xml file, in the <captiveportal> section, do you find something like this :
..... <passthrumac> <action>block</action> <mac>aa:aa:aa:aa:aa:aa</mac> <descr><![CDATA[test]]></descr> </passthrumac> <passthrumac> <action>block</action> <mac>b0:70:2d:44:fc:da</mac> <descr><![CDATA[test2]]></descr> </passthrumac> ..... -
I threw myself of my captive portal I use myself (hotel - like you, since 1.1
) - and added the MAC of my Phone to the MAC list as "Blocked".
I tried to login again.
I received a red text (thus the portal error page) : "The MAC address has been blocked".The problem started with the upgrade to 2.4.4.p3. On a separate machine, just now I installed the previous version of pfSense and restored the configuration files from the current setup, and then moved the connections over, and everything works perfectly. Unknown machines are blocked, and pfSense redirects "Blocked" machines to the website in the configuration. So the upgrade to 2.4.4.p3 is most certainly the trigger of this problem.
I know the network intimately and there are no routers in the network. Using the computer that I'm typing on, attached to a layer 2 switch connected to the LAN port on pfSense, from pfSense I can see my machine's IP address and MAC address in the ARP table, and they can PING each other, and a tracert shows there are no additional hops between. I can remove my machine from the Captive Portal MAC list with no effect. I can change it to "Block" with no effect. Without any login prompt or MAC address entry, any machine attaching to the network can access the Internet.
There are no entries in the "Allowed IP Addresses" list. There are no allowed hostnames, no vouchers, etc.
On the other hand, the "xxxx_pipes_mac" table is loaded with 366 different MAC address. This table contains the list with "allowed" MAC addresses. Quiet a list ....
It's a large layer 2 network.
When you check the config.xml file, in the <captiveportal> section, do you find something like this :
Lots of entries, yes, and they look like this:
<passthrumac> <action>pass</action> <mac>00:01:42:90:41:5c</mac> <descr><![CDATA[Z]]></descr> </passthrumac> <passthrumac> <action>pass</action> <mac>00:08:54:a0:39:9e</mac> <descr><![CDATA[K]]></descr> </passthrumac> <passthrumac> <action>pass</action> <mac>00:0a:cd:20:57:62</mac> <descr><![CDATA[S]]></descr> </passthrumac>thank you for looking at this
-
One quick note: as mentioned before, if I add a MAC and then set the "Bandwidth Up" and "Bandwidth Down" parameters, pfSense on 2.4.4.p3 does limit the bandwidth properly, which means it's seeing the machine and matching it with a MAC address just fine, just not blocking it. For now I'm blocking rogue machines by adding them and setting these parameters to "5" but having an expensive IT person on site just to block people isn't going to work for my client.
-
@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
these parameters to "5"
What parameter to 5 ?
Can you show the captive portal setup ?
If possible, an entire screen dump.@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
One quick note: as mentioned before, if I add a MAC and then set the "Bandwidth Up" and "Bandwidth Down" parameters, pfSense on 2.4.4.p3 does limit the bandwidth properly
Can you confirm that that MAC is be listed in the default_pipe_mac table ? (ipfw table default_pipe_mac list) if it's a "Pass" ?
The MAC should NOT be listed if it is setup as a "Block".Right now, I'm trying to replicate your issue.
@free4 :
See this function call :
https://github.com/pfsense/pfsense/blob/45f95753963e497b5ce14493f9cca05336d75c7b/src/etc/inc/captiveportal.inc#L736$cprules .= captiveportal_passthrumac_configure(true);and now see what goes on in that function captiveportal_passthrumac_configure(...);
https://github.com/pfsense/pfsense/blob/45f95753963e497b5ce14493f9cca05336d75c7b/src/etc/inc/captiveportal.inc#L1347function captiveportal_passthrumac_configure($filename = false, $startindex = 0, $stopindex = 0) { global $config, $g, $cpzone; $rules = ""; if (is_array($config['captiveportal'][$cpzone]['passthrumac'])) { if ($stopindex > 0) { $fd = fopen($filename, "w");A context might exist where the code would write to file file called "True" ??
I'm pretty sure that's not the problem here ....but this seems to 'dead code' to me. -
@Gertjan said in Captive portal ignoring MACs in latest version and allowing all machines access:
@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
these parameters to "5"
What parameter to 5 ?In each MAC listing, there is an "Edit MAC Address Rules" page where you can set "Bandwidth Up" and "Bandwidth Down" in Kbit/s. To disable a machine, I set these to "5" which essentially disables Internet access for that machine. (It's an arbitrary value.)
The biggest problem is the client must provide open Wifi access and allow only machines that can login or have their MAC address listed. Right now all machines are able to access the Internet with no login presented and no ability to disable any machine except by setting bandwidth restriction to "5"
Can you show the captive portal setup ?
If possible, an entire screen dump.I can upload the entire configuration if you want (with some things sanitized for security). Privately I could give you TeamViewer access to a machine attached to the router, if that would make this faster/easier.
Services/Captive Portal
Services/Captive Portal/Default/Configuration, part 1
Services/Captive Portal/Default/Configuration, part 2
Services/Captive Portal/Default/MACs (privacy items blocked out)
Can you confirm that that MAC is be listed in the default_pipe_mac table ? (ipfw table default_pipe_mac list) if it's a "Pass" ?
The MAC should NOT be listed if it is setup as a "Block".I will create a before.txt and after.txt using MAC address "28:c6:8e:0f:95:9b"
Before, with 28:c6:8e:0f:95:9b set to Pass
before.txtAfter, with 28:c6:8e:0f:95:9b set to Block
after.txtI note that the MAC is not found in the second output.
Right now, I'm trying to replicate your issue.
Thank you for your efforts!!
-
If a configuration file would help, here is the Captive Portal section, with most of the "passthrumac" section removed for security. You can probably remove the dots I added (to note the deleted section) and restore it to a test machine.
Do you want the entire configuration? It would take some effort to remove security items, but would be happy to provide if you think it would help.
-
No remote access needed.
IIt's already a daily job not to mess up my own system (in use all the time - it's live).
Don't want to mess with somebody data.Your captive portal settings look fine to me, although not having idle timeout out neither hard time out.
That's rare - it's always good to have something cleaning up things (connected devices being gone long time ago).@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
captiveportal-config.xml
Compare yours
<passthrumac><action>pass</action><mac>00:01:42:90:41:5c</mac><descr>Z</descr></passthrumac><passthrumac><action>pass</action><mac>00:08:54:a0:39:9e</mac><descr>K</descr></passthrumac><passthrumac><action>pass</action><mac>00:0a:cd:20:57:62</mac><descr>S</descr></passthrumac>with mine :
<passthrumac> <action>pass</action> <mac>4c:8d:79:91:ec:52</mac> <descr><![CDATA[Sophie]]></descr> </passthrumac> <passthrumac> <action>pass</action> <mac>64:80:99:9a:01:a0</mac> <descr><![CDATA[Serge Portable]]></descr> </passthrumac> <passthrumac> <action>pass</action> <mac>7c:76:35:f2:a9:0e</mac> <descr><![CDATA[Serge Nouveau]]></descr> </passthrumac>The <descr> element in a <passthrumac> should be 'escaped' using CDATA, because if there are any <>& (whatever) in the description, the parsing code could go wrong.
If you didn't have the issue, you should see this line in the portal log :
Jun 12 12:00:44 logportalauth 18891 Zone: cpzone1 - Blocked MAC address: b0:70:ff:44:fc:da, b0:70:ff:44:fc:da, 192.168.2.217The only place where MAC's are tested for 'Blocked', is here : The main index.php (line 142).
Check this : https://github.com/pfsense/pfsense/commits/master/src/usr/local/captiveportal/index.php
What happens when you undo that modification ? (not much I guess,, I'm using the most recent 2.4.4-p3 and it works for me).
Also :
ipfw showplease.
-
Your captive portal settings look fine to me, although not having idle timout out neiter hard time out.
That's rare - it's always good to have something cleaning up things (connected devices being gone long time ago).Nobody ever uses the login prompt... either they have a MAC address bypass or they cannot access the Internet at this location. So the timeout doesn't apply to anything. And it's the same configuration as the previous versions and it worked for months.
<passthrumac> <action>pass</action> <mac>7c:76:35:f2:a9:0e</mac> <descr><![CDATA[Serge Nouveau]]></descr> </passthrumac>The <descr> element in a <passthrumac> should be 'escaped' using CDATA, because if there are any <>& (whatever) in the description, the parsing code could wrong.
Again, same configuration as last week when it was working. We never use anything but alphanumerics in that field in any case.
If you diidn't have the issue, you shuld see this line in the portal log :
Jun 12 12:00:44 logportalauth 18891 Zone: cpzone1 - Blocked MAC address: b0:70:ff:44:fc:da, b0:70:ff:44:fc:da, 192.168.2.217Captive portal Auth has three entries this morning:
Jun 11 07:50:47 logportalauth 72563 Zone: default - Blocked MAC address: 0c:c4:7a:6c:24:5a, 0c:c4:7a:6c:24:5a, 10.55.1.79
Jun 11 07:49:29 logportalauth 72563 Zone: default - Blocked MAC address: 0c:c4:7a:6c:24:5a, 0c:c4:7a:6c:24:5a, 10.55.1.79
Jun 11 07:49:04 logportalauth 96863 Zone: default - Blocked MAC address: 0c:c4:7a:6c:24:5a, 0c:c4:7a:6c:24:5a, 10.55.1.79That happens to be the machine I'm typing on right now, and it's not being blocked, and it's not in the MAC list as "Deny" but it's in there set to "Pass". I also know there are about a dozen devices that don't have entries in the MAC list that are currently browsing because I see them in the Traffic Graph.
What happens when you undo that modification ? (not much I guess,, I'm using the most recent 2.4.4-p3 and it works for me).I'm afraid I don't know how to do this.
Also :
ipfw show
please.Right here:
01000 145774939 141217225879 skipto tablearg ip from any to any via table(cp_ifaces)
01100 145864684 141226838568 allow ip from any to any
02100 145763334 141217081935 pipe tablearg ip from any to any MAC table(default_pipe_mac)
02101 0 0 allow pfsync from any to any
02102 0 0 allow carp from any to any
02103 3364 0 allow ip from any to any layer2 mac-type 0x0806,0x8035
02104 0 0 allow ip from any to any layer2 mac-type 0x888e,0x88c7
02105 0 0 allow ip from any to any layer2 mac-type 0x8863,0x8864
02106 7333 0 deny ip from any to any layer2 not mac-type 0x0800,0x86dd
02107 0 0 allow ip from any to table(default_host_ips) in
02108 0 0 allow ip from table(default_host_ips) to any out
02109 386 75077 allow ip from any to 255.255.255.255 in
02110 0 0 allow ip from 255.255.255.255 to any out
02111 0 0 pipe tablearg ip from table(default_allowed_up) to any in
02112 0 0 pipe tablearg ip from any to table(default_allowed_down) in
02113 0 0 pipe tablearg ip from table(default_allowed_up) to any out
02114 0 0 pipe tablearg ip from any to table(default_allowed_down) out
02115 0 0 pipe tablearg ip from table(default_auth_up) to any layer2 in
02116 0 0 pipe tablearg ip from any to table(default_auth_down) layer2 out
02117 5 260 fwd 127.0.0.1,8002 tcp from any to any 80 in
02118 8 10558 allow tcp from any to any out
02119 495 55743 skipto 65534 ip from any to any
65534 509 58049 deny ip from any to any
65535 468 206880 allow ip from any to any -
@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
default_pipe_mac
Possible to test with an empty "default_pipe_mac" table == remove all "pass" MAC's on the MAC page.
With an empty page, are some devices still passing through ?
If traffic passes through, you can see this when you are using the
ipfw showThe second and third row :
[2.4.4-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: ipfw show 01000 112970410 80790521165 skipto tablearg ip from any to any via table(cp_ifaces) 01100 1168495079 809253512261 allow ip from any to any 02100 0 0 pipe tablearg ip from any to any MAC table(cpzone1_pipe_mac) 02101 0 0 allow pfsync from any to any 02102 0 0 allow carp from any to any 02103 8467 0 allow ip from any to any layer2 mac-type 0x0806,0x8035 02104 0 0 allow ip from any to any layer2 mac-type 0x888e,0x88c7 02105 0 0 allow ip from any to any layer2 mac-type 0x8863,0x8864 02106 522 0 deny ip from any to any layer2 not mac-type 0x0800,0x86dd 02107 16184 1452576 allow ip from any to table(cpzone1_host_ips) in 02108 17858 3990740 allow ip from table(cpzone1_host_ips) to any out 02109 441 147853 allow ip from any to 255.255.255.255 in 02110 0 0 allow ip from 255.255.255.255 to any out 02111 194 20444 pipe tablearg ip from table(cpzone1_allowed_up) to any in 02112 0 0 pipe tablearg ip from any to table(cpzone1_allowed_down) in 02113 0 0 pipe tablearg ip from table(cpzone1_allowed_up) to any out 02114 36 2736 pipe tablearg ip from any to table(cpzone1_allowed_down) out 02115 3090097 495709961 pipe tablearg ip from table(cpzone1_auth_up) to any layer2 in 02116 3776895 4800660171 pipe tablearg ip from any to table(cpzone1_auth_down) layer2 out 02117 22294 2428331 fwd 127.0.0.1,8003 tcp from any to any 443 in 02118 2705 250394 fwd 127.0.0.1,8002 tcp from any to any 80 in 02119 23349 5938873 allow tcp from any to any out 02120 5040 1004382 skipto 65534 ip from any to any 65534 278883 51331968 deny ip from any to any 65535 27 14905 allow ip from any to anyindicates byte throughput.
Just repeat the command.
With all the tables empty - most of yours are empty except a massive "allow ip from any to any" table (something like 332 entries) nothing can pass. Nothing should pass.
-
Possible to test with an empty "default_pipe_mac" table == remove all "pass" MAC's on the MAC page.
Perhaps just delete the entire Captive Portal profile, add a new one without an MAC bypass entries and test to see whether anyone can browse?
-
@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
Possible to test with an empty "default_pipe_mac" table == remove all "pass" MAC's on the MAC page.
Perhaps just delete the entire Captive Portal profile, add a new one without an MAC bypass entries and test to see whether anyone can browse?
That's even a better idea !
You can always 'mass import' the MAC's back in by editing a config backup file.
-
Perhaps just delete the entire Captive Portal profile, add a new one without an MAC bypass entries and test to see whether anyone can browse?
That's even a better idea !
You can always 'mass import' the MAC's back in by editing a config backup file.
Initial results:
-
Deleted the Captive Portal profile. Result: all machines can access the Internet.
-
Created a new Captive Portal profile. Result: All machines blocked by the "Captive Portal" login prompt
-
Added one machine's MAC address: that machine no longer gets login prompt, but cannot access the Internet. Strange.
-
Did a "Backup" of the Captive Portal, edited the file to insert all of the MAC address items, then "Restored" the Captive Portal backup file... now all machines can access the Internet... the MAC address list is now completely ignored again.
I need to let the client use their Internet connection without interruption during the day so I will try again at 4am tomorrow morning. I had to do all of this a little hastily so I might have missed something.
-
-
Something new after the above steps were taken:
If an entry in the MAC list has bandwidth limitations, but the Action is set to PASS, then only the download bandwidth is affected, and upload speed is still unrestricted.
If I change the MAC entry to "DENY" then both upload and download bandwidth is unrestricted.
So right now if I want to stop a machine from accessing the internet, I have to make sure it has a MAC address entry in the list, then I have to set it to PASS and then set the download speed to "5".... but they can still upload. This is different from yesterday when I could set any MAC entry's speed regardless of PASS or DENY and it would control both download and upload speed.
I'm honestly considering blowing the whole configuration away and starting the router configuration over from scratch. But as I mentioned before, if I restore this configuration to the last version of pfSense, it works just fine, so there's no guarantee that starting over will solve this problem.... just more time wasted.
-
@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
Deleted the Captive Portal profile. Result: all machines can access the Internet.
With default firewall rules on LAN, yes, that's normal.
@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
Created a new Captive Portal profile. Result: All machines blocked by the "Captive Portal" login prompt
All devices non authenticated should be blocked, that's normal.
Although, a second captive portal on the same LAN interface ? Even with the first interface disabled, I wouldn't test drive such a thing.@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
Added one machine's MAC address: that machine no longer gets login prompt, but cannot access the Internet. Strange.
As you showed above, MACs that are added as a pass, are added to the first table "default_pipe_mac"
--- table(default_pipe_mac), set(0) --- e0:ff:49:4a:91:f5 any 2689 0 0 0 any e0:ff:49:4a:91:f5 2688 0 0 0 ....The numbers 2689 and 2688 are the references to the pipes created for that MAC - on for each direction - your "ipfw_pipe_list.txt" above listed these two pipes as unlimited :
02689: unlimited 0 ms burst 0 q133761 100 sl. 0 flows (1 buckets) sched 68225 weight 0 lmax 0 pri 0 droptail sched 68225 type FIFO flags 0x0 16 buckets 0 activeand
02688: unlimited 0 ms burst 0 q133760 100 sl. 0 flows (1 buckets) sched 68224 weight 0 lmax 0 pri 0 droptail sched 68224 type FIFO flags 0x0 16 buckets 0 activeBut : the machine can not access the anything (Internet).
Be more precise please : can it access - or visit, at least the pfSense GUI (not the portal login page, but the GUI - check you GUI firewall if that's permitted) ? Or some other close by web server / other service I mean, a gateway problem make us looking at portal problems, but (example) Internet is simply down.@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
Did a "Backup" of the Captive Portal, edited the file to insert all of the MAC address items, then "Restored" the Captive Portal backup file... now all machines can access the Internet... the MAC address list is now completely ignored again.
This is what's bugging me.
No MAC's on the list : everybody is blocked => That normal.
Add one MAC : this one should pass : you can see it in the "default_pipe_mac" table twice. These two rules should match, traffic should flow, according to attached pipe constraints.
Two MAC's : identical, but for those 2.may MAC : everybody passes now .... ?? It's not even pfSense that misbehaves, it looks like you broke 'ipfw', for what I know, an industrial strength firewall .
I can't find an entry or rule or whatever that explains your issue.When no MACs or IPs are listed in any tables, a device will hit rule 02117 (and 02118 if https portal page) who interacts with the portal web server - rule 02119 permit the portal server to 'talk' to the portal interface - rule 02120 jumps to the final rule 65534 that represent the wall == block everything.
I understand that you can't tear the place down in pieces, the setup is in a live environment.
The captive portal is used by thousands - and it should have been know that adding some MAC's to the MAC "Pass or Block" page adds a free ride for everybody - I have some blocked MAC's and Passes on that MAC page : it just work for years now.
Also : I'm not a developer, but I can read Github : captive portal code didn't change between 2.4.4 p1 p2 and p3.
There is this "You are connected" 2.4.4-p3 bug which resets / flushes all tables in ipfw when an pfSense admin saves the portal settings. Authenticated users are still shown as "logged in" in the GUI, but related IP/MAC are flushed from the tables : these users can't login again (no login page is shown, just the text "You are already logged in") , because pfSense considers them as "logged in" and thus ipfw rules (tables) are NOT added for them to pass : users are locked out.
A pull request has been issued that resolves this issue.
IMHO : Your issue is not related to this one.@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
If an entry in the MAC list has bandwidth limitations, but the Action is set to PASS, then only the download bandwidth is affected, and upload speed is still unrestricted.
If I change the MAC entry to "DENY" then both upload and download bandwidth is unrestricted.It would be nice if @jimp could see this - if he could understand what situation would make this happen.
I imported into my config 200 records like (with incrementing MAC IDs):
<passthrumac> <action>pass</action> <mac>10:20:30:40:50:60</mac> <descr><![CDATA[test test]]></descr> </passthrumac>and 50 "block" records .
I also added the MAC of my Phone, as a "pass", and my pad as a blocked device and I tested : all well - phone could communicate, pad was blocked.What I can't test is the leaking through as you described : I don't have that many real devices.
I tested different Bandwidth up and Bandwidth down values like 4321 and 1234 (Kbits) : the speed was respected.
-
The captive portal is used by thousands - and it should have been know that adding some MAC's to the MAC "Pass or Block" page adds a free ride for everybody - I have some blocked MAC's and Passes on that MAC page : it just work for years now.
Like I said before, I've used it for years, and this client has used it for years.
We have had problems with Captive Portal. Two bugs were acknowledged and fixed recently.
I'm going to reset the settings on that router and enter everything again from scratch over the weekend.
There is this "You are connected" 2.4.4-p3 bug which resets / flushes all tables in ipfw when an pfSense admin saves the portal settings. Authenticated users are still shown as "logged in" in the GUI, but related IP/MAC are flushed from the tables :
This also happens here when people try to login, but we don't use the login feature anymore so it doesn't matter.
-
@h2professor said in Captive portal ignoring MACs in latest version and allowing all machines access:
Like I said before, I've used it for years, and this client has used it for years.
I understand - I'm just repeating myself a lot (close to rambling perhaps).
You and I use the same code. I really want see this bug. Seeing is is resolving it. -
I understand - I'm just repeating myself a lot (close to rambling perhaps).
You and I use the same code. I really want see this bug. Seeing is is resolving it.If I install the latest pfSense on a separate machine and import this configuration, the problem exists also on the new machine. Perhaps I could sanitize the configuration file and post it here. Perhaps you can see something I can't. But I know it can be reproduced. It's not a complicated router-- one WAN, one LAN. No hot backup.
