Unofficial E2guardian package for pfSense

pfsensation · Jun 27, 2017, 11:01 PM

@pfsensation:

Really don't wanna lose my configs :( – It took a while to configure everything to a usable state.

They will be there. :)
All config stays on pfSense config.xml. The etc files just reflect what were saved on GUI.

I've encountered a pfSense crash, again it seems to be related to E2Guardian. Wasn't able to collect any details on it. Then I realised there was an update for E2Guardian, I updated now and ended up with the same issue (couldn't start the service). Downloaded the blacklist again, then press play and it started.

So maybe force or keep old blacklist when upgrading / installing?

Also wanted to add that ShallaList categories still don't show D: – Just says "blocked site".

marcelloc · Jun 27, 2017, 11:09 PM

@pfsensation:

I've encountered a pfSense crash, again it seems to be related to E2Guardian. Wasn't able to collect any details on it. Then I realised there was an update for E2Guardian, I updated now and ended up with the same issue (couldn't start the service). Downloaded the blacklist again, then press play and it started.

Did you updated the inc file before the update? I did it on 3 different installs and upgrade was fine.

pfsensation · Jun 27, 2017, 11:11 PM

@marcelloc:

@pfsensation:

I've encountered a pfSense crash, again it seems to be related to E2Guardian. Wasn't able to collect any details on it. Then I realised there was an update for E2Guardian, I updated now and ended up with the same issue (couldn't start the service). Downloaded the blacklist again, then press play and it started.

Did you updated the inc file before the update? I did it on 3 different installs and upgrade was fine.

What inc file are you referring to? I haven't touched any specific inc files.

marcelloc · Jun 27, 2017, 11:21 PM

@marcelloc:

fetch this file and try resintalling, upgrading, removing version e2guardian pkg v 0.4.2.5
fetch -o /usr/local/pkg/e2guardian.inc https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-e2guardian4/files/usr/local/pkg/e2guardian.inc
This fetch 'fixes' uninstall process from previous versions ( < 0.4.2.4) by replacing with current 0.4.2.5 e2guardian.inc file.

After upgrading, apply settings under services -> e2guardian.

What's new on 0.4.2.5

Reduced uninstall remove file process to do not break reinstalls

Improved watchdog script and gui realtime view.

from this post. ::)

But if you are on 0.4.2.5, you don't need this fetch anymore.

pfsensation · Jun 28, 2017, 3:20 PM

@marcelloc:

@marcelloc:
fetch this file and try resintalling, upgrading, removing version e2guardian pkg v 0.4.2.5
fetch -o /usr/local/pkg/e2guardian.inc https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-e2guardian4/files/usr/local/pkg/e2guardian.inc
This fetch 'fixes' uninstall process from previous versions ( < 0.4.2.4) by replacing with current 0.4.2.5 e2guardian.inc file.

After upgrading, apply settings under services -> e2guardian.

What's new on 0.4.2.5

Reduced uninstall remove file process to do not break reinstalls

Improved watchdog script and gui realtime view.
from this post. ::)

But if you are on 0.4.2.5, you don't need this fetch anymore.

Let's see how this goes. Have you tried setting up HTTPS transparently? If I try forwarding port 443 to 8080 on the pfSense box, it breaks HTTPS. However it works fine on HTTP (port 80) this is how I have it setup already.

I know it can be done without breaking HTTPS because smoothwall has this capability. You can get onto HTTPS sites just by installing their CA certificate (without setting up proxy settings on Android). And I am mentioning Android in particular because it doesn't have WPAD or any auto detection, so all the magic must be happening on the actual router itself, it is making HTTPS traffic go through the filter.

Let me know if this can be done, it would be useful to have for guest devices. Not 100% sure but I think Squid has this, if we could set it up for certain IP's or ranges it would be fantastic. :)

It's annoying to go around and actually manually configure the proxy settings for Android in particular, and some apps.

EDIT: I think that Fortinet uses some kind of ARP poisoning as one way of working. Probably right up there with our brute force method of using NAT redirects. Even on IOS devices, they haven't always been the best when picking up the proxy settings from WPAD.

jetberrocal · Jun 28, 2017, 3:11 PM

e2guardian mitm does not work in transparent mode. If you you want transparent mode you have to turn off mitm and block only by blacklist.

I do not know the technicality why but I think is because to use SSL forging you need to authenticate the connection to a user/machine and that has to be in explicit mode.

marcelloc · Jun 28, 2017, 3:23 PM

@pfsensation:

Let's see how this goes. Have you tried setting up HTTPS transparently? If I try forwarding port 443 to 8080 on the pfSense box, it breaks HTTPS. However it works fine on HTTP (port 80) this is how I have it setup already.

Only when e2guardian code supports transparent ssl. Current version does not has it.
If you forward 443 to e2g, you may filter without mitm.
A working setup is e2g in sandwich mode (squid tranparent +splice all -> e2g without mitm -> automatic parent). You can deny access to sites but with no intercetion and no client config.

You can create groups acls that has proxy configured and interception and groups from squid splice all.

EDIT

I've included a request on e2guardian github project. hope they can do it soon
https://github.com/e2guardian/e2guardian/issues/254

pfsensation · Jun 28, 2017, 3:24 PM

@marcelloc:

@pfsensation:

Let's see how this goes. Have you tried setting up HTTPS transparently? If I try forwarding port 443 to 8080 on the pfSense box, it breaks HTTPS. However it works fine on HTTP (port 80) this is how I have it setup already.

Only when e2guardian code supports transparent ssl. Current version does not has it.
If you forward 443 to e2g, you may filter without mitm.
A working setup is e2g in sandwich mode (squid tranparent +splice all -> e2g without mitm -> automatic parent). You can deny access to sites but with no intercetion and no client config.

You can create groups acls that has proxy configured and interception and groups from squid splice all.

I'm trying to get this transparent setup only for the guest devices. Devices that I cannot get a CA on, however on all the rest I want normal MITM. Would that be possible?

When I get home I will try messing with it, because ideally for guests I want to just use splice all (via E2Guardian) which is what I'm already doing but Android clients don't want to happily work with this kind of setup. But for other groups it's MITM, I always try avoiding using a url based blocking because in today's day and age it's useless. And it's like going back to the old SquidGuard days for me, limited.

jetberrocal · Jun 28, 2017, 3:39 PM

Just thinking a vage idea. Maybe you can make all your Android devices to connect to a specific subnet. Then you can authenticate by that subnet in a Group. This way you can filter with block list and content.

Maybe you can have one Wireless Router with DHCP relay assigning IPs by the subnet segment with a password only fro Android users.

The hardway is to have a MAC roster file with each Android device, then assign IPs reservations from a subnet pool.

But for mitm to work you have to use the CA.

Someone may have a better defined idea to this.

pfsensation · Jun 28, 2017, 3:47 PM

@jetberrocal:

Just thinking a vage idea. Maybe you can make all your Android devices to connect to a specific subnet. Then you can authenticate by that subnet in a Group. This way you can filter with block list and content.

Maybe you can have one Wireless Router with DHCP relay assigning IPs by the subnet segment with a password only fro Android users.

The hardway is to have a MAC roster file with each Android device, then assign IPs reservations from a subnet pool.

But for mitm to work you have to use the CA.

Someone may have a better defined idea to this.

Android requires some extra setup for getting HTTPS filtering, that's kinda one of the big issues. And also, it would be hard to identify Android devices and slap them on a different subnet.

jetberrocal · Jun 28, 2017, 3:59 PM

@pfsensation:

@jetberrocal:

Just thinking a vage idea. Maybe you can make all your Android devices to connect to a specific subnet. Then you can authenticate by that subnet in a Group. This way you can filter with block list and content.

Maybe you can have one Wireless Router with DHCP relay assigning IPs by the subnet segment with a password only fro Android users.

The hardway is to have a MAC roster file with each Android device, then assign IPs reservations from a subnet pool.

But for mitm to work you have to use the CA.

Someone may have a better defined idea to this.

Android requires some extra setup for getting HTTPS filtering, that's kinda one of the big issues. And also, it would be hard to identify Android devices and slap them on a different subnet.

Yes. That is why I suggested the MAC address roster. I do not know how many Android devices are in your network but maybe doable.

Another idea is hard because requires coding. Maybe someone could create a Captive Portal page with java script that could identify the connecting device OS and MAC address. Then on a submit button execute other script that use the MAC to programmatically add it to the DHCP reservation table and invoke a command in the Android device to refresh the IP. Maybe on the same script invoke command to load the CA for the user to import it in the device.

pfsensation · Jun 28, 2017, 4:57 PM

@jetberrocal:

@pfsensation:

@jetberrocal:

Just thinking a vage idea. Maybe you can make all your Android devices to connect to a specific subnet. Then you can authenticate by that subnet in a Group. This way you can filter with block list and content.

Maybe you can have one Wireless Router with DHCP relay assigning IPs by the subnet segment with a password only fro Android users.

The hardway is to have a MAC roster file with each Android device, then assign IPs reservations from a subnet pool.

But for mitm to work you have to use the CA.

Someone may have a better defined idea to this.

Android requires some extra setup for getting HTTPS filtering, that's kinda one of the big issues. And also, it would be hard to identify Android devices and slap them on a different subnet.

Yes. That is why I suggested the MAC address roster. I do not know how many Android devices are in your network but maybe doable.

Another idea is hard because requires coding. Maybe someone could create a Captive Portal page with java script that could identify the connecting device OS and MAC address. Then on a submit button execute other script that use the MAC to programmatically add it to the DHCP reservation table and invoke a command in the Android device to refresh the IP. Maybe on the same script invoke command to load the CA for the user to import it in the device.

Most commercial products that do filtering seem to have a way to alert the user to install a CA. But the CA isn't the only issue here, android actually needs you to set pfsense as the proxy to allow https filtering. Because http traffic is easily redirected.

But I'm confused, if I try redirecting Port 443 to 8080 it breaks https, however smoothwall is able to do this without any client configuration. Just installing the CA. For guest devices though, ideally I don't want to deal with any CA's.

pfsensation · Jun 28, 2017, 5:13 PM

Came home to find my pfsense box crashed. :/

At this point I'm considering a reinstall of pfsense but not sure how much I see lose in terms of settings etc. And of course e2guardian settings. Also E2 Guardian requires the unofficial repository.

Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p19
FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:09:14 CDT 2017     root@ce23-amd64-builder:/builder/pfsense-234/tmp/obj/builder/pfsense-234/tmp/FreeBSD-src/sys/pfSense

Crash report details:

No PHP errors found.

Filename: /var/crash/bounds
2

Filename: /var/crash/info.0
Dump header from device /dev/label/swap0
  Architecture: amd64
  Architecture Version: 1
  Dump Length: 72704B (0 MB)
  Blocksize: 512
  Dumptime: Wed Jun 28 11:48:46 2017
  Hostname: pfSense.kortex
  Magic: FreeBSD Text Dump
  Version String: FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:09:14 CDT 2017
    root@ce23-amd64-builder:/builder/pfsense-234/tmp/obj/builder/pfsense-234/tmp/FreeBSD-src/sys/pfSense
  Panic String: 
  Dump Parity: 3988917314
  Bounds: 0
  Dump Status: good

Filename: /var/crash/info.1
Dump header from device /dev/label/swap0
  Architecture: amd64
  Architecture Version: 1
  Dump Length: 72192B (0 MB)
  Blocksize: 512
  Dumptime: Wed Jun 28 11:15:20 2017
  Hostname: pfSense.kortex
  Magic: FreeBSD Text Dump
  Version String: FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:09:14 CDT 2017
    root@ce23-amd64-builder:/builder/pfsense-234/tmp/obj/builder/pfsense-234/tmp/FreeBSD-src/sys/pfSense
  Panic String: vm_radix_remove: impossible to locate the key
  Dump Parity: 3671161642
  Bounds: 1
  Dump Status: good

Filename: /var/crash/info.last
Dump header from device /dev/label/swap0
  Architecture: amd64
  Architecture Version: 1
  Dump Length: 72192B (0 MB)
  Blocksize: 512
  Dumptime: Wed Jun 28 11:15:20 2017
  Hostname: pfSense.kortex
  Magic: FreeBSD Text Dump
  Version String: FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:09:14 CDT 2017
    root@ce23-amd64-builder:/builder/pfsense-234/tmp/obj/builder/pfsense-234/tmp/FreeBSD-src/sys/pfSense
  Panic String: vm_radix_remove: impossible to locate the key
  Dump Parity: 3671161642
  Bounds: 1
  Dump Status: good

Filename: /var/crash/textdump.tar.0
ddb.txt06000014000013124704616  7076 ustarrootwheeldb:0:kdb.enter.default>  run lockinfo
db:1:lockinfo> show locks
No such command
db:1:locks>  show alllocks
No such command
db:1:alllocks>  show lockedvnods
Locked vnodes
db:0:kdb.enter.default>  show pcpu
cpuid        = 1
dynamic pcpu = 0xfffffe010fd49100
curthread    = 0xfffff8002dad4960: pid 93640 "ntopng"
curpcb       = 0xfffffe009430dc80
fpcurthread  = 0xfffff8002dad4960: pid 93640 "ntopng"
idlethread   = 0xfffff80003521960: tid 100004 "idle: cpu1"
curpmap      = 0xfffff8002dd1a4b8
tssp         = 0xffffffff821138f8
commontssp   = 0xffffffff821138f8
rsp0         = 0xfffffe009430dc80
gs32p        = 0xffffffff82115350
ldt          = 0xffffffff82115390
tss          = 0xffffffff82115380
db:0:kdb.enter.default>  bt
Tracing pid 93640 tid 100220 td 0xfffff8002dad4960
pmap_remove_pte() at pmap_remove_pte+0x19a/frame 0xfffffe009430d8f0
pmap_remove() at pmap_remove+0x471/frame 0xfffffe009430d9a0
vm_map_delete() at vm_map_delete+0x1ed/frame 0xfffffe009430da30
sys_munmap() at sys_munmap+0xff/frame 0xfffffe009430da90
amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe009430dbb0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe009430dbb0
--- syscall (73, FreeBSD ELF64, sys_munmap), rip = 0x803cdcd7a, rsp = 0x7fffdf3f7d88, rbp = 0x7fffdf3f7db0 ---
db:0:kdb.enter.default>  ps
  pid  ppid  pgrp   uid   state   wmesg         wchan        cmd
28137 92144 92144     0  S       nanslp   0xffffffff82001570 sleep
26732 91933 91933     0  S       nanslp   0xffffffff82001570 sleep
24525 33868 26769     0  S       nanslp   0xffffffff82001571 sleep
92144 91589 92144     0  Ss      wait     0xfffff8002271a000 sh
91933 91277 91933     0  Ss      wait     0xfffff800220e54f0 sh
91589 25575 25575     0  S       piperd   0xfffff8001f686ba0 cron
91277 25575 25575     0  S       piperd   0xfffff8001f3d68b8 cron
90702 84685   286     0  S       nanslp   0xffffffff82001571 sleep
65342     1 65342     0  Ss      (threaded)                  e2guardian
101485                   S       accept   0xfffff8002db5d5d6 e2guardian
101484                   S       accept   0xfffff800606fd88e e2guardian
100497                   S       uwait    0xfffff80022475100 e2guardian
100489                   S       uwait    0xfffff80003b46f00 e2guardian
100485                   S       uwait    0xfffff8001f6c3a00 e2guardian
100448                   S       uwait    0xfffff8002d79fe80 e2guardian
100447                   S       uwait    0xfffff80022475a00 e2guardian
100446                   S       uwait    0xfffff80022170c00 e2guardian
100445                   S       uwait    0xfffff8002d79e200 e2guardian
100444                   S       uwait    0xfffff8002db6de00 e2guardian
100443                   S       uwait    0xfffff80022475700 e2guardian
100442                   S       uwait    0xfffff8002da57400 e2guardian
100441                   S       select   0xfffff8002278c9c0 e2guardian
100440                   S       uwait    0xfffff8002df18b00 e2guardian
100439                   S       uwait    0xfffff8002d79e300 e2guardian
100438                   S       uwait    0xfffff80022171e80 e2guardian
100437                   S       uwait    0xfffff8002db6e880 e2guardian
100436                   S       uwait    0xfffff8001f68a700 e2guardian
100435                   S       uwait    0xfffff8002df3a900 e2guardian
100434                   S       uwait    0xfffff8002db6da00 e2guardian
100433                   S       uwait    0xfffff8001f7acd80 e2guardian
100432                   S       uwait    0xfffff80022333f00 e2guardian
100431                   S       uwait    0xfffff80022474180 e2guardian
100430                   S       uwait    0xfffff80022474d80 e2guardian
100429                   S       uwait    0xfffff80022170200 e2guardian
100428                   S       uwait    0xfffff80022333c00 e2guardian
100427                   S       uwait    0xfffff80022475e00 e2guardian
100426                   S       uwait    0xfffff80022475600 e2guardian
100425                   S       uwait    0xfffff80022475900 e2guardian
100424                   S       uwait    0xfffff80022476080 e2guardian
100423                   S       uwait    0xfffff8002df3ed80 e2guardian
100422                   S       uwait    0xfffff80022475200 e2guardian
100421                   S       uwait    0xfffff8002df3e180 e2guardian
100420                   S       uwait    0xfffff8002db6d800 e2guardian
100419                   S       uwait    0xfffff8002d79ec00 e2guardian
100417                   S       uwait    0xfffff8002df3e480 e2guardian
100416                   S       uwait    0xfffff8002db6e700 e2guardian
100415                   S       uwait    0xfffff8001f68ac00 e2guardian
100414                   S       uwait    0xfffff8002d79fa80 e2guardian
100413                   S       uwait    0xfffff8002df3ab00 e2guardian
100411                   S       uwait    0xfffff80022474680 e2guardian
100410                   S       uwait    0xfffff8002d79f880 e2guardian
100409                   S       uwait    0xfffff8002d79f780 e2guardian
100408                   S       uwait    0xfffff8002d79f580 e2guardian
100407                   S       select   0xfffff80003d3d340 e2guardian
100406                   S       uwait    0xfffff80022170300 e2guardian
100405                   S       uwait    0xfffff8001f705480 e2guardian
100404                   S       uwait    0xfffff80022474280 e2guardian
100403                   S       uwait    0xfffff80022474780 e2guardian
100402                   S       uwait    0xfffff8002df3e880 e2guardian
100401                   S       uwait    0xfffff8002df18a00 e2guardian
100400                   S       uwait    0xfffff80022170400 e2guardian
100398                   S       uwait    0xfffff8002deb8280 e2guardian
100397                   S       uwait    0xfffff80022170e00 e2guardian
100396                   S       uwait    0xfffff8002dec5a00 e2guardian
100395                   S       uwait    0xfffff8002db6ed80 e2guardian
100394                   S       uwait    0xfffff8002dec5200 e2guardian
100393                   S       uwait    0xfffff80022170b00 e2guardian
100392                   S       uwait    0xfffff8002d79e800 e2guardian
100391                   S       uwait    0xfffff8001f6dc880 e2guardian
100390                   S       uwait    0xfffff80022170100 e2guardian
100389                   S       uwait    0xfffff80022171d80 e2guardian
100388                   S       uwait    0xfffff80022170500 e2guardian
100387                   S       uwait    0xfffff8002df3d700 e2guardian
100386                   S       uwait    0xfffff8002db6e780 e2guardian
100385                   S       uwait    0xfffff8002db6f100 e2guardian
100384                   S       uwait    0xfffff8002df19280 e2guardian
100383                   S       uwait    0xfffff80022333b00 e2guardian
100382                   S       uwait    0xfffff8002df3db00 e2guardian
100381                   S       uwait    0xfffff8002231cd80 e2guardian
100380                   S       uwait    0xfffff800606b8b00 e2guardian
100379                   S       uwait    0xfffff80022474c80 e2guardian
100378                   S       uwait    0xfffff80003c5cf00 e2guardian
100377                   S       uwait    0xfffff80022474a80 e2guardian
100376                   S       uwait    0xfffff8002de3fe80 e2guardian
100375                   S       uwait    0xfffff8002df3eb80 e2guardian
100374                   S       uwait    0xfffff8002db6f200 e2guardian
100373                   S       uwait    0xfffff8002d79ea00 e2guardian
100372                   S       uwait    0xfffff8002df3a680 e2guardian
100371                   S       uwait    0xfffff8002df18600 e2guardian
100370                   S       uwait    0xfffff80022333000 e2guardian
100369                   S       uwait    0xfffff8002db6e600 e2guardian
100368                   S       uwait    0xfffff80003b48480 e2guardian
100367                   S       uwait    0xfffff80022333a00 e2guardian
100366                   S       uwait    0xfffff80022475b00 e2guardian
100365                   S       select   0xfffff8002231dec0 e2guardian
100364                   S       uwait    0xfffff80022171980 e2guardian
100363                   S       uwait    0xfffff80003d3ec80 e2guardian
100362                   S       uwait    0xfffff80022333500 e2guardian
100361                   S       uwait    0xfffff8002db6db00 e2guardian
100360                   S       uwait    0xfffff8002deb9200 e2guardian
100359                   S       uwait    0xfffff8002deb9e00 e2guardian
100358                   S       uwait    0xfffff80022171c80 e2guardian
100357                   S       uwait    0xfffff80022474e80 e2guardian
100356                   S       uwait    0xfffff80022475500 e2guardian
100355                   S       uwait    0xfffff8002d79ee00 e2guardian
100354                   S       uwait    0xfffff8002df3e280 e2guardian
100353                   S       uwait    0xfffff8002de3f380 e2guardian
100352                   S       uwait    0xfffff80022475800 e2guardian
100351                   S       uwait    0xfffff8002d79f380 e2guardian
100350                   S       uwait    0xfffff80022171a80 e2guardian
100349                   S       uwait    0xfffff80003c75800 e2guardian
100348                   S       select   0xfffff800223b2140 e2guardian
100347                   S       uwait    0xfffff8002df3d000 e2guardian
100346                   S       uwait    0xfffff8002231d100 e2guardian
100345                   S       uwait    0xfffff80022333700 e2guardian
100344                   S       uwait    0xfffff80003d3fe80 e2guardian
100343                   S       uwait    0xfffff8002df3ee80 e2guardian
100342                   S       uwait    0xfffff8002db6f000 e2guardian
100341                   S       uwait    0xfffff8002df3e980 e2guardian
100340                   S       uwait    0xfffff8002dec4080 e2guardian
100339                   S       uwait    0xfffff8002d79f680 e2guardian
100338                   S       uwait    0xfffff8001f6d9e00 e2guardian
100337                   S       uwait    0xfffff8002d79fc80 e2guardian
100336                   S       uwait    0xfffff80003c74580 e2guardian
100335                   S       uwait    0xfffff8002df3ec80 e2guardian
100334                   S       uwait    0xfffff80003b46080 e2guardian
100333                   S       uwait    0xfffff8002dec5100 e2guardian
100332                   S       uwait    0xfffff80022474080 e2guardian
100331                   S       uwait    0xfffff8002db6f300 e2guardian
100330                   S       uwait    0xfffff8002df3af00 e2guardian
100329                   S       uwait    0xfffff8002dec4b80 e2guardian
100328                   S       uwait    0xfffff8002df3c980 e2guardian
100327                   S       uwait    0xfffff80022170f00 e2guardian
100326                   S       uwait    0xfffff8002dec5400 e2guardian
100325                   S       uwait    0xfffff80022170d00 e2guardian
100324                   S       uwait    0xfffff8002db6e280 e2guardian
100323                   S       uwait    0xfffff80022170800 e2guardian
100322                   S       uwait    0xfffff80022171180 e2guardian
100321                   S       uwait    0xfffff8002d79f280 e2guardian
100320                   S       uwait    0xfffff8002deb9300 e2guardian
100319                   S       uwait    0xfffff8002df3e680 e2guardian
100318                   S       uwait    0xfffff8002dec4880 e2guardian
100317                   S       uwait    0xfffff80022170900 e2guardian
100316                   S       uwait    0xfffff8002d79ed00 e2guardian
100315                   S       uwait    0xfffff800606b9d80 e2guardian
100314                   S       uwait    0xfffff80022170600 e2guardian
100313                   S       uwait    0xfffff800606bb600 e2guardian
100312                   S       uwait    0xfffff80022992a80 e2guardian
100311                   S       uwait    0xfffff80003d3f400 e2guardian
100310                   S       uwait    0xfffff8002df3e580 e2guardian
100309                   S       uwait    0xfffff80003d3f380 e2guardian
100308                   S       uwait    0xfffff80022474580 e2guardian
100307                   S       uwait    0xfffff800223b2380 e2guardian
100306                   S       uwait    0xfffff80022475d00 e2guardian
100305                   S       uwait    0xfffff80022d1c700 e2guardian
100304                   S       uwait    0xfffff80022475c00 e2guardian
100303                   S       uwait    0xfffff80022474380 e2guardian
100302                   S       uwait    0xfffff8002df3e380 e2guardian
100301                   S       uwait    0xfffff80003b0ae80 e2guardian
100300                   S       uwait    0xfffff8002df3e780 e2guardian
100299                   S       uwait    0xfffff8002d79f180 e2guardian
100298                   S       uwait    0xfffff8002d79e100 e2guardian
100297                   S       uwait    0xfffff80022474880 e2guardian
100296                   S       uwait    0xfffff8002d79e400 e2guardian
100295                   S       uwait    0xfffff80022475300 e2guardian
100294                   S       uwait    0xfffff80003b46000 e2guardian
100293                   S       uwait    0xfffff80022474480 e2guardian
100292                   S       uwait    0xfffff8002d79fd80 e2guardian
100291                   S       uwait    0xfffff80022475000 e2guardian
100290                   S       uwait    0xfffff8002df19f00 e2guardian
100289                   S       uwait    0xfffff8002df3e080 e2guardian
100288                   S       uwait    0xfffff80022171680 e2guardian
100287                   S       uwait    0xfffff8002d79ef00 e2guardian
100286                   S       uwait    0xfffff80022333e00 e2guardian
100285                   S       uwait    0xfffff8002d79f480 e2guardian
100284                   S       uwait    0xfffff8001f704200 e2guardian
100283                   S       uwait    0xfffff8002d79e500 e2guardian
100282                   S       uwait    0xfffff8001f707d00 e2guardian
100281                   S       uwait    0xfffff8002d79fb80 e2guardian
100280                   S       uwait    0xfffff80022333900 e2guardian
100279                   S       uwait    0xfffff8002dec4480 e2guardian
100278                   S       uwait    0xfffff80022333d00 e2guardian
100277                   S       uwait    0xfffff8002df3c380 e2guardian
100276                   S       uwait    0xfffff8002d79e900 e2guardian
100275                   S       uwait    0xfffff8001f6dfc00 e2guardian
100274                   S       uwait    0xfffff8002df19880 e2guardian
100273                   S       uwait    0xfffff800606ba280 e2guardian
100272                   S       uwait    0xfffff80022170000 e2guardian
100271                   S       uwait    0xfffff8002df19580 e2guardian
100270                   S       uwait    0xfffff8002d79e600 e2guardian
100269                   S       uwait    0xfffff8002d79e700 e2guardian
100268                   S       uwait    0xfffff8002df3ea80 e2guardian
100267                   S       uwait    0xfffff8002df3ce80 e2guardian
100266                   S       uwait    0xfffff8001f7acf00 e2guardian
100265                   S       uwait    0xfffff8002deb8480 e2guardian
100264                   S       uwait    0xfffff80022333800 e2guardian
100263                   S       uwait    0xfffff8002db6e080 e2guardian
100262                   S       uwait    0xfffff80022171080 e2guardian
100261                   S       uwait    0xfffff8002df18c00 e2guardian
100260                   S       uwait    0xfffff8002de3fd80 e2guardian
100259                   S       uwait    0xfffff8002df3a100 e2guardian
100258                   S       uwait    0xfffff80022474b80 e2guardian
100257                   S       uwait    0xfffff8001f68b780 e2guardian
100256                   S       uwait    0xfffff8002dec5e00 e2guardian
100255                   S       uwait    0xfffff8002dec5c00 e2guardian
100254                   S       uwait    0xfffff8001f704880 e2guardian
100253                   S       uwait    0xfffff8002d79f080 e2guardian
100252                   S       uwait    0xfffff8002db6e900 e2guardian
100251                   S       uwait    0xfffff80022170700 e2guardian
100246                   S       uwait    0xfffff80003d3c980 e2guardian
100131                   S       uwait    0xfffff8002de3f280 e2guardian
100103                   S       uwait    0xfffff80022333400 e2guardian
100472                   S       sigwait  0xfffff800223ea000 e2guardian
23318 81222 81222     0  S       accept   0xfffff8006064e88e php-fpm
74233 73167 72931   100  S       sbwait   0xfffff800606e03fc ssl_crtd
74211 73167 72931   100  S       sbwait   0xfffff8002dbda6b4 ssl_crtd
73964 73167 72931   100  S       sbwait   0xfffff80003e6b3fc ssl_crtd
73751 73167 72931   100  S       sbwait   0xfffff800227db3fc ssl_crtd
73505 73167 72931   100  S       sbwait   0xfffff8002dbc7144 ssl_crtd
73167 72931 72931   100  S       kqread   0xfffff8002de84100 squid
72931     1 72931   100  Ss      wait     0xfffff800220d2000 squid
81222     1 81222     0  Ss      kqread   0xfffff8002dc05a00 php-fpm
33868     1 26769     0  S+      wait     0xfffff800607979e0 sh
33408 32868 32868     0  S       kqread   0xfffff800221fa900 nginx
33110 32868 32868     0  S       kqread   0xfffff80022648700 nginx
32868     1 32868     0  Ss      pause    0xfffff800605c8a88 nginx
32187 31953 32187     0  S+      ttyin    0xfffff8000388f0a8 sh
31953 31589 31953     0  S+      wait     0xfffff800607994f0 sh
31914 62516 31914     0  Ss      (threaded)                  sshlockout_pf
100245                   S       nanslp   0xffffffff82001571 sshlockout_pf
100090                   S       piperd   0xfffff80003c132e8 sshlockout_pf
31589     1 31589     0  Ss+     wait     0xfffff80003b43000 login
17910 17755 17755     0  S       nanslp   0xffffffff82001570 minicron
17755     1 17755     0  Ss      wait     0xfffff80003c6e4f0 minicron
17486 16950 16950     0  S       nanslp   0xffffffff82001570 minicron
16950     1 16950     0  Ss      wait     0xfffff800220e69e0 minicron
16709 16073 16073     0  S       nanslp   0xffffffff82001571 minicron
16073     1 16073     0  Ss      wait     0xfffff8001f3f14f0 minicron
93640     1 93640     0  Rs      (threaded)                  ntopng
100227                   S       nanslp   0xffffffff82001570 ntopng
100226                   S       nanslp   0xffffffff82001570 ntopng
100224                   S       bpf      0xfffff8002dadf800 ntopng
100223                   S       nanslp   0xffffffff82001571 ntopng
100222                   S       nanslp   0xffffffff82001571 ntopng
100221                   S       nanslp   0xffffffff82001571 ntopng
100220                   Run     CPU 1                       ntopng
100219                   S       uwait    0xfffff800606b9f00 ntopng
100218                   S       uwait    0xfffff800606b9a80 ntopng
100217                   S       uwait    0xfffff80003c74d80 ntopng
100216                   S       uwait    0xfffff80003c75200 ntopng
100215                   S       uwait    0xfffff80003c75500 ntopng
100214                   S       select   0xfffff8001f689140 ntopng
100084                   S       nanslp   0xffffffff82001571 ntopng
93399     1   286     0  S       (threaded)                  redis-server
100213                   S       uwait    0xfffff8002db6eb80 redis-server
100212                   S       uwait    0xfffff8001f6c3080 redis-server
100088                   S       kqread   0xfffff8002da41e00 redis-server
85832 81513 81171   100  S       select   0xfffff80003c5d840 pinger
85623 81513 81171   100  S       piperd   0xfffff8001f3d5000 unlinkd
84685     1   286     0  S       wait     0xfffff80003ce89e0 sh
81513 81171 81171   100  S       kqread   0xfffff8002dbd5700 squid
81171     1 81171   100  Ss      wait     0xfffff80003c58000 squid
80210     1 80210     0  Ss      (threaded)                  filterdns
100209                   S       uwait    0xfffff8002db6ea80 signal-thread
100208                   S       uwait    0xfffff8002db6fc00 149.154.167.91
100207                   S       uwait    0xfffff8002db6fb00 telegram.org
100206                   S       uwait    0xfffff8002db6fa00 filterdns
100205                   S       uwait    0xfffff8002db6f900 filterdns
100204                   S       uwait    0xfffff8002db6f800 filterdns
100203                   S       uwait    0xfffff8002db6f700 filterdns
100202                   S       uwait    0xfffff8002db6f600 filterdns
100201                   S       uwait    0xfffff8002db6f500 adnxs.com
100200                   S       uwait    0xfffff8002db6f400 adnexus.net
100199                   S       uwait    0xfffff8002db70580 a.ads2.msn.com
100198                   S       uwait    0xfffff8002db70480 a.ads1.msn.com
100197                   S       uwait    0xfffff8002db70380 ads1.msn.com
100196                   S       uwait    0xfffff8002db70280 ads1.msads.net
100195                   S       uwait    0xfffff8002db70180 ads.msn.com
100194                   S       uwait    0xfffff8002db70080 ad.doubleclick.net
100193                   S       uwait    0xfffff8002db6ff00 preview.msn.com
100192                   S       uwait    0xfffff8002db6fe00 rad.msn.com
100191                   S       uwait    0xfffff8002db6fd00 filterdns
100190                   S       uwait    0xfffff800606bbd80 filterdns
100189                   S       uwait    0xfffff8002db70d80 filterdns
100188                   S       uwait    0xfffff8002db70c80 filterdns
100187                   S       uwait    0xfffff8002db70b80 filterdns
100186                   S       uwait    0xfffff8002db70a80 filterdns
100185                   S       uwait    0xfffff8002db70980 filterdns
100184                   S       uwait    0xfffff8002db70880 filterdns
100183                   S       uwait    0xfffff8002db70780 filterdns
100182                   S       uwait    0xfffff8002db70680 filterdns
100181                   S       uwait    0xfffff8002da55700 filterdns
100180                   S       uwait    0xfffff8002da55600 filterdns
100179                   S       uwait    0xfffff8002da55500 a-0001.a-msedge.net
100178                   S       uwait    0xfffff8002da55400 cs1.wpc.v0cdn.net
100177                   S       uwait    0xfffff8002da55300 filterdns
100176                   S       uwait    0xfffff8002da55200 filterdns
100175                   S       uwait    0xfffff8002da55100 filterdns
100174                   S       uwait    0xfffff8002da55000 filterdns
100173                   S       uwait    0xfffff800606bbe80 watson.live.com
100172                   S       uwait    0xfffff8002da56080 filterdns
100171                   S       uwait    0xfffff8002da55f00 filterdns
100170                   S       uwait    0xfffff8002da55e00 filterdns
100169                   S       uwait    0xfffff8002da55d00 filterdns
100168                   S       uwait    0xfffff8002da55c00 filterdns
100167                   S       uwait    0xfffff8002da55b00 filterdns
100166                   S       uwait    0xfffff8002da55a00 filterdns
100165                   S       uwait    0xfffff8002da55900 filterdns
100164                   S       uwait    0xfffff8002da55800 filterdns
100163                   S       uwait    0xfffff8002da56900 filterdns
100162                   S       uwait    0xfffff8002da56980 filterdns
100161                   S       uwait    0xfffff8002da56780 filterdns
100160                   S       uwait    0xfffff8002da56680 filterdns
100159                   S       uwait    0xfffff8002da56580 filterdns
100158                   S       uwait    0xfffff8002da56480 filterdns
100157                   S       uwait    0xfffff8002da56380 filterdns
100156                   S       uwait    0xfffff8002da56280 filterdns
100155                   S       uwait    0xfffff8002da56180 filterdns
100154                   S       uwait    0xfffff80003b5d800 filterdns
100153                   S       uwait    0xfffff80003b5d700 filterdns
100152                   S       uwait    0xfffff80003b5d600 filterdns
100151                   S       uwait    0xfffff80003d3f900 filterdns
100150                   S       uwait    0xfffff80003b5d300 filterdns
100149                   S       uwait    0xfffff80003b5d200 filterdns
100148                   S       uwait    0xfffff80003b5d400 filterdns
62516     1 62516     0  Ss      select   0xfffff8001f68c4c0 syslogd
31944     1 31944  1002  Ss      select   0xfffff8001f6d13c0 dhcpd
26322     1 26322     0  Ss      (threaded)                  ntpd
100119                   S       select   0xfffff800606bb0c0 ntpd
25575     1 25575     0  Ss      nanslp   0xffffffff82001571 cron
19818     1 19818     0  Ss      kqread   0xfffff8001f696a00 dhcpleases
18996     1 18996    59  Ss      (threaded)                  unbound
100488                   S       kqread   0xfffff80003b61700 unbound
100117                   S       kqread   0xfffff8002dbea700 unbound
17788     1 17788     0  Ss      (threaded)                  dpinger
100115                   S       accept   0xfffff8001f6e488e dpinger
100114                   S       nanslp   0xffffffff82001570 dpinger
100113                   S       nanslp   0xffffffff82001571 dpinger
100112                   S       sbwait   0xfffff8001f6e4c24 dpinger
100111                   S       uwait    0xfffff80003b5d500 dpinger
14228     1 14228     0  Ss      bpf      0xfffff80003cd6400 filterlog
11948     1 11948    65  Ss      select   0xfffff80003d3d040 dhclient
 7708     1  7708     0  Ss      select   0xfffff80003d3eb40 dhclient
 6600     1  6600     0  Ss      (threaded)                  sshlockout_pf
100106                   S       nanslp   0xffffffff82001571 sshlockout_pf
100096                   S       uwait    0xfffff80003b47680 sshlockout_pf
 6473     1  6473     0  Ss      select   0xfffff80003d3dd40 sshd
  337     1   337     0  Ss      select   0xfffff80003c5c940 devd
  326   324   324     0  S       kqread   0xfffff80003ce7200 check_reload_status
  324     1   324     0  Ss      kqread   0xfffff80003bae400 check_reload_status
   55     0     0     0  DL      mdwait   0xfffff80003b1a000 [md0]
   20     0     0     0  DL      syncer   0xffffffff82052508 [syncer]
   19     0     0     0  DL      vlruwt   0xfffff80003b439e0 [vnlru]
   18     0     0     0  DL      (threaded)                  [bufdaemon]
100086                   D       sdflush  0xfffff80003b7d8e8 [/ worker]
100075                   D       psleep   0xffffffff82051704 [bufdaemon]
   17     0     0     0  DL      pgzero   0xffffffff8206283c [pagezero]
    9     0     0     0  DL      pollid   0xffffffff81fffe90 [idlepoll]
    8     0     0     0  DL      psleep   0xffffffff82061bc0 [vmdaemon]
    7     0     0     0  DL      (threaded)                  [pagedaemon]
100079                   D       umarcl   0xffffffff82061540 [uma]
100071                   D       psleep   0xffffffff82112c04 [pagedaemon]
    6     0     0     0  DL      waiting_ 0xffffffff821036c0 [sctp_iterator]
    5     0     0     0  DL      pftm     0xffffffff80d5db10 [pf purge]
   16     0     0     0  DL      (threaded)                  [usb]
100061                   D       -        0xfffffe00009e4e70 [usbus4]
100060                   D       -        0xfffffe00009e4e18 [usbus4]
100059                   D       -        0xfffffe00009e4dc0 [usbus4]
100058                   D       -        0xfffffe00009e4d68 [usbus4]
100057                   D       -        0xfffffe00009e4d10 [usbus4]
100056                   D       -        0xfffffe00009d4f48 [usbus3]
100055                   D       -        0xfffffe00009d4ef0 [usbus3]
100054                   D       -        0xfffffe00009d4e98 [usbus3]
100053                   D       -        0xfffffe00009d4e40 [usbus3]
100052                   D       -        0xfffffe00009d4de8 [usbus3]
100050                   D       -        0xfffffe00009c4f48 [usbus2]
100049                   D       -        0xfffffe00009c4ef0 [usbus2]
100048                   D       -        0xfffffe00009c4e98 [usbus2]
100047                   D       -        0xfffffe00009c4e40 [usbus2]
100046                   D       -        0xfffffe00009c4de8 [usbus2]
100044                   D       -        0xfffffe00009b4f48 [usbus1]
100043                   D       -        0xfffffe00009b4ef0 [usbus1]
100042                   D       -        0xfffffe00009b4e98 [usbus1]
100041                   D       -        0xfffffe00009b4e40 [usbus1]
100040                   D       -        0xfffffe00009b4de8 [usbus1]
100038                   D       -        0xfffffe000099cf48 [usbus0]
100037                   D       -        0xfffffe000099cef0 [usbus0]
100036                   D       -        0xfffffe000099ce98 [usbus0]
100035                   D       -        0xfffffe000099ce40 [usbus0]
100034                   D       -        0xfffffe000099cde8 [usbus0]
    4     0     0     0  DL      (threaded)                  [cam]
100070                   D       -        0xffffffff81f360c8 [scanner]
100019                   D       -        0xffffffff81f36280 [doneq0]
   15     0     0     0  DL      -        0xffffffff81f579c0 [rand_harvestq]
    3     0     0     0  DL      crypto_r 0xffffffff82060098 [crypto returns]
    2     0     0     0  DL      crypto_w 0xffffffff8205ff40 [crypto]
   14     0     0     0  DL      (threaded)                  [geom]
100013                   D       -        0xffffffff820f7de8 [g_down]
100012                   D       -        0xffffffff820f7de0 [g_up]
100011                   D       -        0xffffffff820f7dd8 [g_event]
   13     0     0     0  DL      (threaded)                  [ng_queue]
100010                   D       sleep    0xffffffff81ef46f8 [ng_queue1]
100009                   D       sleep    0xffffffff81ef46f8 [ng_queue0]
   12     0     0     0  WL      (threaded)                  [intr]
100078                   I                                   [swi1: netisr 1]
100068                   I                                   [swi1: pfsync]
100066                   I                                   [swi1: pf send]
100063                   I                                   [irq1: atkbd0]
100062                   I                                   [irq14: ata0]
100051                   I                                   [irq16: uhci3]
100045                   I                                   [irq18: uhci2]
100039                   I                                   [irq19: uhci1+]
100033                   I                                   [irq23: uhci0 ehci0]
100032                   I                                   [irq258: re0]
100027                   I                                   [swi5: fast taskq]
100025

I've got it all setup like you recommend Marcello, squid is on splice all. I've disabled ntopng just in case that was interferin

jetberrocal · Jun 28, 2017, 6:01 PM

@jetberrocal:

Just thinking a vage idea. Maybe you can make all your Android devices to connect to a specific subnet. Then you can authenticate by that subnet in a Group. This way you can filter with block list and content.

Maybe you can have one Wireless Router with DHCP relay assigning IPs by the subnet segment with a password only fro Android users.

The hardway is to have a MAC roster file with each Android device, then assign IPs reservations from a subnet pool.

But for mitm to work you have to use the CA.

Someone may have a better defined idea to this.

Just had another idea that maybe doable.

Let say we enable a SSH Server in the network and create a VPN connection from the Android devices to the SSH Server. Then the SSH server is sent to the e2guardian/squid traffic. The Android devices will get IPs from the VPN subnet, The traffic then can be filtered through the VPN with blocklist and content. The authentication/group can be done by the subnet.

I dont know if pfsense can be the VPN Server in this scheme but might be also doable.

This is away to do SSH Tunneling but requires rooting:
https://www.howtogeek.com/121698/how-to-route-all-your-android-traffic-through-a-secure-tunnel/

Maybe there is away to do it without rooting. The link says that it needs rooting for Global Proxy, but we may not need this for our purpose.

Sorry I cant test this with my pfsense. What I have is a VM inside my PC with many Host only VMs that connect to outside world through the virtual pfsense. Maybe I can if I found an Android VM.

jetberrocal · Jun 28, 2017, 7:00 PM

pfsensation:

How do you load the CA certificate to the Android devices?

Just wandering how "easy" or complicated it is.

pfsensation · Jun 28, 2017, 8:40 PM

@jetberrocal:

pfsensation:

How do you load the CA certificate to the Android devices?

Just wandering how "easy" or complicated it is.

Well… I used to use a captive portal that I edited and made people install from. But then with WPAD and squid not having a patch for captive portal. It semi worked. So now for all the devices in the home I've installed the CA. Guest devices rely on splice all filtering (Basically only blacklist based filtering) and I use open dns. So DNS filtering too.

Ideally I'm hoping we can get the E2 Guardian devs to add a captive portal feature where clients are asked to install it before they are able to browse and use the Internet.

marcelloc · Jun 28, 2017, 8:56 PM

@pfsensation:

Ideally I'm hoping we can get the E2 Guardian devs to add a captive portal feature where clients are asked to install it before they are able to browse and use the Internet.

If you change to fully report, you can point it to captive porta login. Then you create an Default acl that accepts only captive portal page.

jetberrocal · Jun 28, 2017, 9:23 PM

I have a Captive Portal with no authentication that does not have a submit button. The CP page shows the instructions how to configure the proxy and where to get the certificate. Without the submit button the device cant be registered as authorized in CP.

Then I have a folder with the CA certificates served by the web server for download. I add the certificates extension to the exceptions so they can do the download without being block by e2g.

The problem that I see now is that for Android devices the certificates are a special kind used only in the Android. Instead of just one crt according to this link: http://wiki.cacert.org/FAQ/ImportRootCert#Android_Phones_.26_Tablets are two ('root.crt' and 'class3.crt').

Maybe someone with Android experience can shed some light.

Mr. Jingles · Jun 28, 2017, 10:17 PM

Hi Marcello,

I decided to try again.

Installed via package manager. Tinyproxy first, E2Guardian second. I pressed 'save' first in the blacklist-tab, then 'download', then 'apply'. GUI in the right top corner gve me 8 messages about blacklist being applied. However, in status/services, E2guardian was not running (red cross), and clicking start also doesn't make it run.

System log:

Jun 29 00:10:11 root /usr/local/etc/rc.d/e2guardian.sh: WARNING: failed to start e2guardian
Jun 29 00:10:11 e2guardian 99798 Error parsing the e2guardian.conf file or other e2guardian configuration files
Jun 29 00:10:11 e2guardian 99798 Error reading filter group conf file(s).
Jun 29 00:10:11 e2guardian 99798 Error opening filter group config: /usr/local/etc/e2guardian/e2guardianf1.conf
Jun 29 00:10:11 e2guardian 99798 Error opening bannedsitelist
Jun 29 00:10:11 e2guardian 99798 Error reading file: /usr/local/etc/e2guardian/lists/bannedsitelist.g_Default
Jun 29 00:10:11 e2guardian 99798 Error opening file: /usr/local/etc/e2guardian/lists/blacklists/adult/domains
Jun 29 00:10:11 e2guardian 99798 Error reading file /usr/local/etc/e2guardian/lists/blacklists/adult/domains: No such file or directory
Jun 29 00:10:11 e2guardian 99798 Error reading /usr/local/etc/e2guardian/lists/blacklists/adult/domains. Check directory and file permissions. They should be 640 and 750: No such file or directory

I do recall vaguely there should be a 'trick' to make it work, would you happen to know it?

Thank you & bye,

marcelloc · Jun 28, 2017, 11:32 PM

You don't need tinyproxy anymore and maybe you still have some files from old install.

All can be fixed via gui by reapplying blacklist under blacklist tab and then saving config then apply button.