Poor network performance



  • Cable modem is a Mototal SB6141, 8 channels bonded downstream.  Bridging.  A laptop with a GigE interface plugged directly in gets 177Mb/s down.

    Pfsense box is a dual Atom D2550, 1.86GHz hyperthreaded, 4GB RAM, running 2.1.2-release.

    Packages running are apinger, avahi, bandwidthd, darkstat, dhcpd, dnsmasq, ntpd, openvpn, and snort.

    CPU sits at 13% idle, moderate traffic takes it to 25-75%, its been as high as 100% for a few seconds.

    Bandwidth through pfsense runs between 50-70Mb/s.  Prior pfsense system was a T7200-based laptop, 2Ghz dual core, 4GB RAM, and had a 100mb interface on the WAN side – it got 50-70Mb/s.  The Atom has dual GigE interfaces, both running full duplex and full speed.

    Not sure why I'm seeing a half (or more) drop in network speeds.

    Any ideas?



  • If your CPU is at 100% when hitting 70mbps, you have a CPU bottleneck. See if any one process is largely responsible. Snort is a likely culprit. Try disabling it or play with the search method it uses.



  • Well disabling snort took me to 105Mbs.  And now, during speed tests and 70-80MB/s traffic going, I never get above 15% CPU.  So now I'm totally confused, and not running snort, which I really would like to run.


  • Netgate Administrator

    Snort is a resource hog but it can be tamed to some extent, mostly by choosing the correct pattern matcher. Check the various Snort threads and here if you haven't already:
    https://doc.pfsense.org/index.php/Setup_Snort_Package
    Just off hand that figure, 50-70Mbps, is suspiciously exactly what I expect the OpenVPN throughput to be.
    There's no reason, without Snort, that you shouldn't be seeing the full 170Mbps. That CPU should be good for >500Mbps of firewall/NAT.

    Steve



  • Well, turning off ALL services, I can get 117Mbs down.  Turning them on one by one slowly eats up the bandwidth.  Turing a couple of them on results in some others (dependencies?) starting as well.

    Not sure what to do here.  Even with snort off, other services (bandwidth monitor, darkstat, openvpn) seem to sap bandwidth.  Incidentally, when I turn openvpn off, I can no longer get speedtest.net to work (although there is no openvpn connection coming in).  With it on, it works again.

    I'm at a loss here.  Not sure what is going on.


  • Netgate Administrator

    Possibly there is a persistent state that is trying to use the VPN connection to get to speedtest.net, try clearing the state table. Other sites are accessible I assume?

    Since the box is capable of >500Mbps I would expect to be able to put a few services on there without reducing the throughput to <120Mbps. Are you saying as soon as you add any service the bandwidth drops?

    Steve



  • tucansam, how you measure performance? Iperf or similar tool?



  • @cutler:

    tucansam, how you measure performance? Iperf or similar tool?

    stephenw10: I've reset the states, and indeed the firewall, with the same effects.  I can get max bandwidth at about 70Mb/s, with no services running, and slowly creep it down to 50Mb/s once I have everything running.  Some services seem to take more bw away than others.

    Cutler: Well, initially I was using speedtest.net.  With my laptop connected directly to my cable modem (bridge), with a FDX GigE connection, 177Mb/s is the fastest I've seen (off-peak hours, averaged over a few days).  When I plug my network back in, the only easily accessible machine is an Atom-based XP system, into which I RDP and then run speedtest.net's app under a Chrome browser.  In the past this has given me 100Mbs+ speeds, as it is also FDX GigE to the switch.  Other devices on the network are linux or wireless, and I just found the speedtest tool to be the easiest.

    I did some googling and ran the "fetch" command with a variety of hosted files of 100MB in size, ran directly from pfsense, and was never able to top 25Mb/s, which I assume means the bottleneck is on each servers' end (speedtest under the XP system continued to show 50-55Mb/s down during these tests)

    I will have to google Iperf and see how to use it as I am unfamiliar.

    I am tempted to restore factory defaults (or re-load from a fresh ISO image completely) and reconfigure.  What would be the better option?



  • Well this is interesting:

    C:\Documents and Settings\Desktop>iperf -c 192.168.0.1
    –----------------------------------------------------------
    Client connecting to 192.168.0.1, TCP port 5001
    TCP window size: 64.0 KByte (default)

    [  3] local 192.168.0.7 port 2618 connected with 192.168.0.1 port 5001
    [ ID] Interval      Transfer    Bandwidth
    [  3]  0.0-10.1 sec  33.1 MBytes  27.6 Mbits/sec

    C:\Documents and Settings\Desktop>

    This is with iperf running (server) on pfsense, and client on the aforementioned XP machine.  All links are GigE FDX through a Netgear business class switch.  Speeds are atrocious!  Intel NICs on the pfsense machine; crappy Realtek on the XP box but speeds have never been this slow.  "Green ethernet" is enabled on the XP system, which I have only read a little about, but the cable length is 3' so it shouldn't matter.  With "Green ethernet" turned off I get 29Mbits/s

    More testing needed here.



  • Yeah replying to myself is poor form.

    OK, well, running the same iperf test back to back several times, I get 29Mb/s, 110, 84, 137, 64, 148, 148, 147…. and pfsense shows CPU never tops 29% use

    What could be causing such differences?  No other traffic is presently on the network.



  • @tucansam:

    Cable modem is a Mototal SB6141, 8 channels bonded downstream.  Bridging.  A laptop with a GigE interface plugged directly in gets 177Mb/s down.

    Pfsense box is a dual Atom D2550, 1.86GHz hyperthreaded, 4GB RAM, running 2.1.2-release.

    Packages running are apinger, avahi, bandwidthd, darkstat, dhcpd, dnsmasq, ntpd, openvpn, and snort.

    CPU sits at 13% idle, moderate traffic takes it to 25-75%, its been as high as 100% for a few seconds.

    Bandwidth through pfsense runs between 50-70Mb/s.  Prior pfsense system was a T7200-based laptop, 2Ghz dual core, 4GB RAM, and had a 100mb interface on the WAN side – it got 50-70Mb/s.  The Atom has dual GigE interfaces, both running full duplex and full speed.

    Not sure why I'm seeing a half (or more) drop in network speeds.

    Any ideas?

    For starters, I will reinforce that if you want to run intensive services such as snort and just everything really but snort is definitely causing issues with you as others have mentioned. I'm not even going to blame any of it really on pfsense simply because I could not imagine running my box at 1.86ghz. I do remember some time ago when I was going to upgrade my internet speed and my ISP actually recommended certain levels of performance for processors or it would just bog down the whole system.

    A good example would be my old Dell XPS that was 850mhz single core with 768MB of memory. Everything was fine when we just had 5MB down and 1MB up. For an 850mhz system it was very snappy with those internet speeds. As soon as we upgraded to 12MB down (16MB with boost, something to do with comcast) and 2 MB up because they were just cheap like that. My DELL XPS starting acting like a Pentium II even though it was a Pentium III. Everything was slower except for when I did bandwidth tests.

    Sure enough they would read true at anywhere from (remember comcast has boost) 16MB to 20MB down and still only 2MB up. So sometimes the speeds would actually increase and I think it was just something that they were doing to make you feel like you had something good.  The only time you would see the speed really is when downloading. Web page surfing forget about it. It felt like dial up sometimes but I believe that it had to do with my pc at the time being too slow for the internet.

    Not only that, but it will make your overall pc performance slower if it cannot keep up with the internet speed simply because your processor still has to process all of that data coming in  even if you have a separate nic.  One recommendation that I would give is to get yourself an AMD AM3+ motherboard. Doesn't have to be fancy. Just something around $80 and and an AMD FX 4130 3.8GHZ or Phenom 965 Black edition. I only say that because right now it is about the cheapest processors that you can get with the best performance. Keep in mind that I'm not one that's trying to save the planet through electricity. I don't buy into that hype. You can still run it efficiently because it will only go as fast as it needs to if you keep amd cool and quiet on. If you get that and say 8GB of DDR3 memory which is really cheap now.

    As of now I am on Verizon with 70MB down and 50MB up. I am running into the same problem sometimes with my 965 x4 black edition running at 3.8GHZ. I 'm just running into it. It's not full blown slowness yet but I can tell that upgrading to something that is 4ghz or more would help my pc performance. In hindsight though I realize that I really don't need the internet speed that they are providing. So it's also a good idea to choose the package that you want and make sure that your not going to have to upgrade a pc over it unless that 's what you want to do. The only reason I say this is because you will never see that true speed unless you're looking at an internet speed test or downloading a file. At times I'm willing to bet that even with no pfsense services it probably feels like dial up when your just surfing.


  • Netgate Administrator

    @Cmellons, I take it you're running a load of packages? your machine spec is way, way higher than anything I'm running.

    @tucansam I had an issue testing my connection speed some time ago that turned out to be a problem with my client machine, an older Windows XP box. When I booted the same machine from a live Linux CD I was suddenly able to max out the connection no problems. I did investigate the problem and I think it turned out to be the Windows default TCP window size but don't quote me on that.
    When you're looking at the pfSense CPU usage you cannot use the dashboard bar graph if your box has multiple CPU cores. That graph shows the average use across all cores. The D2550 is dual core with hyperthreading so it appears as 4 cores. If you have one core maxed out at 100% and only 10% use on the other cores the graph will show 32.5% but in fact the pf process has hit the cpu limit on one core. To get a much better idea run 'top -SH' at the console. That will show you the idle percentage for each core.
    Where were you fetching the file from directlt in pfSense? There's probably a better source nearer to you.

    Steve



  • @Cmellons:

    Sure enough they would read true at anywhere from (remember comcast has boost) 16MB to 20MB down and still only 2MB up. So sometimes the speeds would actually increase and I think it was just something that they were doing to make you feel like you had something good.  The only time you would see the speed really is when downloading. Web page surfing forget about it. It felt like dial up sometimes but I believe that it had to do with my pc at the time being too slow for the internet.

    This is essentially what I am seeing.  File downloads go OK, mostly.  Youtube stutters a bit, and only half loads otherwise.  Web surfing comes and goes.  Sometimes its fast, sometimes not so much.  This is on all systems on the network, wired or wireless, fast or slow.  At home I am running Core2Duos, Phenom II X4s, i5s… Clients are plenty fast for what we are doing.

    @Cmellons:

    Not only that, but it will make your overall pc performance slower if it cannot keep up with the internet speed simply because your processor still has to process all of that data coming in  even if you have a separate nic.  One recommendation that I would give is to get yourself an AMD AM3+ motherboard. Doesn't have to be fancy. Just something around $80 and and an AMD FX 4130 3.8GHZ or Phenom 965 Black edition. I only say that because right now it is about the cheapest processors that you can get with the best performance. Keep in mind that I'm not one that's trying to save the planet through electricity. I don't buy into that hype. You can still run it efficiently because it will only go as fast as it needs to if you keep amd cool and quiet on. If you get that and say 8GB of DDR3 memory which is really cheap now.

    My Atom draws less than 25W at the wall.  Its silent (in the entertainment center, the only place I can put it), and runs very cool.  I'm also running dual Intel NICs, on which I insist.  I also need the mini-ITX form factor, in a chassis no larger than the cardboard box the MB came on.  Power onboard the MB, 4GB RAM… No way I can justify a 95W CPU, and I'm ot aware of any mini-ITX AM3/+ boards with dual NICs.  Only option would be an Intel-based dual-NIC mini-ITX, and probably an i5 at the minimum given your recommendations.  That would make my firewall the fastest system in the house, which is contrary to everything I've ever read about pfsense  ;D

    @Cmellons:

    As of now I am on Verizon with 70MB down and 50MB up. I am running into the same problem sometimes with my 965 x4 black edition running at 3.8GHZ. I 'm just running into it. It's not full blown slowness yet but I can tell that upgrading to something that is 4ghz or more would help my pc performance. In hindsight though I realize that I really don't need the internet speed that they are providing. So it's also a good idea to choose the package that you want and make sure that your not going to have to upgrade a pc over it unless that 's what you want to do. The only reason I say this is because you will never see that true speed unless you're looking at an internet speed test or downloading a file. At times I'm willing to bet that even with no pfsense services it probably feels like dial up when your just surfing.

    Agreed.  I am buying the highest package my ISP offers for the monthly bandwidth allowance, not the speed.  Still, seeing blinding speeds at the modem, but 1/2-1/3 as fast behind pfsense, makes me wonder if I've got another problem.  Frankly, I don't know what i would ever do with 177Mb/s down, but if I'm not capable of getting it, it means something is amiss.

    I appreciate your recommendations, but building a firewall that is many orders of magnitude better config'd than my fastest workstation doesn't seem like the right approach.  Low power, low heat, "appliance" type devices is what turned me onto pfsense in the first place.



  • @stephenw10:

    @tucansam I had an issue testing my connection speed some time ago that turned out to be a problem with my client machine, an older Windows XP box. When I booted the same machine from a live Linux CD I was suddenly able to max out the connection no problems. I did investigate the problem and I think it turned out to be the Windows default TCP window size but don't quote me on that.

    That's a stellar idea.  I will have to give this a shot.  All of the PCs in the house exhibit the same symptoms, but I'll wire up a laptop and boot off a live CD to compare.

    @stephenw10:

    When you're looking at the pfSense CPU usage you cannot use the dashboard bar graph if your box has multiple CPU cores. That graph shows the average use across all cores. The D2550 is dual core with hyperthreading so it appears as 4 cores. If you have one core maxed out at 100% and only 10% use on the other cores the graph will show 32.5% but in fact the pf process has hit the cpu limit on one core. To get a much better idea run 'top -SH' at the console. That will show you the idle percentage for each core.

    Stellar, thank you.  I had no idea.  Yep, I've been using the dashboard, and occasionally top, but not with -SH  I will keep an eye on things with that from now on.

    @stephenw10:

    Where were you fetching the file from directlt in pfSense? There's probably a better source nearer to you.

    Yep, directly from pfsense.  I'll dig around for servers closer to me and test again.


  • Netgate Administrator

    If you used a fetch command example I posted anywhere I probably pointed to a thinkbroadband test file. They're great if you're in the UK but not so much from the US.  ;) Chris (cmb) once posted a similar site with test files he uses in the US but I can't find it now.

    Steve



  • @stephenw10:

    If you used a fetch command example I posted anywhere I probably pointed to a thinkbroadband test file. They're great if you're in the UK but not so much from the US.  ;) Chris (cmb) once posted a similar site with test files he uses in the US but I can't find it now.

    Steve

    Ha.  Yep, pretty sure it was your thread I read.

    An an aside, after uninstalling snort a few days ago, I just now reinstalled it, and its running.  My media downloader is showing 4.9-5.3MB/s download speeds, and 'top -SH' is showing 83-89% idle with that traffic passing.  I typically run it at 200KB/s, at which point 'top -SH' shows 94-99% idle.  3.5MB free memory during the duration.

    Do not believe this is a CPU issue…

    Just as a point of curiosity, has anyone ever ranked the most system-resource-hungry packages from top to bottom?  I know some of what I am running is probably unnecessary, and I'd like to leave enough headroom for other things.  For one thing, I am trying to get rules working to restrict the bandwidth of some devices, as well as schedules for those devices.  Not sure how much, if any, processing power that would take up.  I'll also be revisiting squid at some point (which I have never seemed to get installed correctly despite following numerous youtube tutorials) as well as squidguard (same traffic).


  • Netgate Administrator

    @tucansam:

    'top -SH' is showing 83-89% idle with that traffic passing.

    How is that divided between the cores? The central firewall/NAT process, pf, can curently only use one core so that's usually the limit. Snort will be able to use other cores though.

    Steve



  • cachefly is the one I tend to use that Steve referenced, they have links to a 10 MB and 100 MB test file on their site.
    http://cachefly.cachefly.net/10mb.test
    http://cachefly.cachefly.net/100mb.test

    As a CDN, they should be fast pretty much everywhere because you should end up at a server that's relatively close to you. Granted that depends on where you are, your ISP, and many other factors.



  • You should really take that XP box out back and shoot it. Or load some supported OS on it. Already a nice 0 day that isn't going to get patched on XP.
    http://arstechnica.com/security/2014/04/active-0day-attack-hijacking-ie-users-threatens-a-quarter-of-browser-market/

    Those are just going to keep coming and coming. XP is dead, it's been time to move on for years.



  • @stephenw10:

    @tucansam:

    'top -SH' is showing 83-89% idle with that traffic passing.

    How is that divided between the cores? The central firewall/NAT process, pf, can curently only use one core so that's usually the limit.
    Steve

    Only true before pfSense 2.2.


  • Netgate Administrator

    @cmb:

    You should really take that XP box out back and shoot it.

    Indeed, and that's coming from a die hard XP fan. I have seen little point in upgrading Windows versions until now. XP did everything I needed it to without too much system bloat. 2K was better!  ;) However I've now switched everything I had running XP to Xubuntu which runs great on older hardware. Also playing with GhostBSD which is nice with XFCE. Even so I still have one machine set to dual boot to XP which I had to use yesterday to open BIOS update distributed as a windows executable.  >:(

    Steve



  • @stephenw10:

    @tucansam:

    'top -SH' is showing 83-89% idle with that traffic passing.

    How is that divided between the cores? The central firewall/NAT process, pf, can curently only use one core so that's usually the limit. Snort will be able to use other cores though.

    Steve

    Just ran fetch on the 100mb cachefly file and got only 1944kBps, one cpu was 100% idle, other three were 90-94%, snort never went above 25%



  • @cmb:

    You should really take that XP box out back and shoot it. Or load some supported OS on it. Already a nice 0 day that isn't going to get patched on XP.
    http://arstechnica.com/security/2014/04/active-0day-attack-hijacking-ie-users-threatens-a-quarter-of-browser-market/

    Those are just going to keep coming and coming. XP is dead, it's been time to move on for years.

    Working on it, although I'll save my bullets for other purposes.  I need to migrate some things from that machine to another one, and I need to build the new one first.  So, yeah, working on it.



  • @stephenw10:

    @cmb:

    You should really take that XP box out back and shoot it.

    Indeed, and that's coming from a die hard XP fan. I have seen little point in upgrading Windows versions until now. XP did everything I needed it to without too much system bloat. 2K was better!  ;) However I've now switched everything I had running XP to Xubuntu which runs great on older hardware. Also playing with GhostBSD which is nice with XFCE. Even so I still have one machine set to dual boot to XP which I had to use yesterday to open BIOS update distributed as a windows executable.  >:(

    Steve

    Agreed, 2000 > XP > Win7 > DOS 1.0 > Win8.

    Actually scratch that, put "punch cards" ahead of Win8.

    My speeds are remaining consistent, just consistently slower than I had anticipated given my setup.


  • Netgate Administrator

    @tucansam:

    [Just ran fetch on the 100mb cachefly file and got only 1944kBps[/quote]

    Then I think you'll have to test your connection speed to cachefly without the pfSense box because that's slower that anything else.

    @tucansam:

    scratch that, put "punch cards" ahead of Win8.

    Technically I don't think punch cards count as an operation system. Then again you could say the same for Win8.  ;)

    Steve



  • @tucansam

    @Stephenw10

    I failed to mention that it was just lying around collecting dust. It was a previous gaming pc. However, I do realize the kind of damage that I was causing to the environment and now I am running pfsense in a virtual machine. Then again I just have that itch again to build it back up and run it. I need to get a dynamat for the box though. It was only $40 so as you could guess it is way too loud.  Something like that really does not draw that much power with amd cool and quiet on. Most of the time it ran at 800mhz  and .75v. The tdp was embarrassingly high at 1.47v and 140w at full load so yes I had a big thermaltake maxorb cooling it.

    edited in the interest of not messing up a topic. I must hold back on the caffeine intake late at night.



  • I just wanted to jump in on this as I am seeing the same type of issue.

    I bought an OPNsense appliance running Intel Atom 1.6Ghz, 2GB RAM, 2GB CF, and in a production environment on a 100Mbit fibre connection it gives me 15/87. Sent it back on warranty and got a reply that it was a config / software mismatch, but when it got back I rebuilt the entire config manually and the problem persists. I use some advanced NAT (reflection, Virtual IP's, 1:1 etc) but almost no packages except dhcpd, dns and whatever is default.

    I'm thinking hardware issue but I'm not sure. Seems strange since the hardware is brand new. Maybe you've got the same issue as I do?


  • Netgate Administrator

    Not quite sure what numbers you're giving us there. You're seeing 15Mbps down on a 100Mbps connection?
    If that's the case look for a duplex mismatch or possibly some flow control issue. Check the Status: Interfaces: page for errors/collisions.
    Test directly on the box to see which interface is throttling the connection. Look at 'top -SH' at the console to see if it's a CPU or interrupt problem.

    Steve


  • Netgate Administrator

    @Cmellons:

    I do realize the kind of damage that I was causing to the environment

    Don't underestimate what damage you are saving by not buying new hardware. Of course if you already have a VM host running then yes, no excuse!  ;)

    Steve



  • @stephenw10:

    Not quite sure what numbers you're giving us there. You're seeing 15Mbps down on a 100Mbps connection?
    If that's the case look for a duplex mismatch or possibly some flow control issue. Check the Status: Interfaces: page for errors/collisions.
    Test directly on the box to see which interface is throttling the connection. Look at 'top -SH' at the console to see if it's a CPU or interrupt problem.

    Steve

    Don't want to hijack the thread, but you're right. I'm getting 15Mbit down and 87Mbit up. Duplex settings look OK, flow control has been off throughout all of the testing but I actually put it on yesterday just to see if it makes a difference, but no it doesn't.
    The only other thing I've found of interest is that the backbone switch log contains rows where loop protection is saying "The Packet has failed crc check so discarding". But if I view loop protection on ports there's no report on any of the ports. And no other switch is saying the same thing, neither does the pfsense. And there are currently no other hardware connected to the backbone switch. Also when testing transmission speeds on the backbone switch it's very low, like it's getting spammed or something. Still the switch doesn't report any transmission errors, loops or anything of the sort.

    It's hard to know if it's the switch or the router that's at fault, but at the moment the AMD router is installed and at least everything is working at the moment. Not as fast as I'd like it to, but good enough for this network. I have another router on order that I'll install just to rule out certain things. We'll see after that I guess.



  • I am now having issues where things load randomly. 99% of stuff loads fine, some things (certain Youtube videos, sometimes pictures on shopping sites), simply don't load at all, ever.  Everything loads fine from my phone when on Verizon's network, so I know its a problem with my network.  Add to that random slowness, videos no longer downloading using Firefox plugins to save flv files, etc…  Sometimes a daily (!) pfsense reboot fixes it, most times not.

    In the past 18 months I've run pfsense, I have installed numerous packages, and then uninstalled them when things broke (read: often, never could get squid to work, never could get squidguard to work, etc).  I think snort is breaking things but I can't be certain, when I disable the service things are still broken, but they weren't broken before snort was installed.

    After installing/giving up/uninstalling/revisiting a dozen times, I think it has left pfsense in a state where there are artifacts remaining from various packages, and the system is simply not stable or performing.

    I am going to re-install pfsense from the ground up today and see what happens.

    Hopefully a vanilla install will work.  Although after seeing all the things squid is detecting, I really want to make sure that gets re-installed.... But it breaks things like Pandora (kids use it) and akamia stuff (youtube, amazon, etc) and I end up spending days resolving IP addresses to put them in the allow list, so we'll have to see.  And I really went with pfsense largely for squid, which has broke the internet every time I installed and configured it, despite much hacking, configuring, tutorial reading, and gnashing of teeth.

    I may just go back to running untangle exclusively, instead of behind pfsense.


  • Netgate Administrator

    Some websites not loading can be an MTU issue. Not seen that for while though.

    Steve


  • Netgate Administrator

    @Phatsta:

    the backbone switch log contains rows where loop protection is saying "The Packet has failed crc check so discarding".

    What NICs is the box running? Try disabling all the hardware offloading options if any are on especially checksum offloading.

    Steve



  • "I am now having issues where things load randomly. 99% of stuff loads fine, some things (certain Youtube videos, sometimes pictures on shopping sites), simply don't load at all, ever."

    I'm almost positive that it is Snort. The HTTP INSPECT goes wild often and for me anyways when pictures are loading on Amazon for instance this will happen:

    #(http_inspect) NO CONTENT-LENGTH OR TRANSFER-ENCODING IN HTTP RESPONSE
    suppress gen_id 120, sig_id 3

    #(http_inspect) INVALID CONTENT-LENGTH OR CHUNK SIZE
    suppress gen_id 120, sig_id 8

    #(http_inspect) HTTP RESPONSE GZIP DECOMPRESSION FAILED
    suppress gen_id 120, sig_id 6

    AND when downloading a file this happens sometimes:

    #ET POLICY PE EXE or DLL Windows file download
    suppress gen_id 1, sig_id 2000419

    I would just try suppressing those. Maybe even clear out your alert list afterward and try accessing the site again and then check your blocklist in Snort to make the necessary suppressions. That's the thing about Snort. It's a wonderful program but it also needs babysitting to make it work right. I have found that if my firewall rules are good then I don't even need it but then again I don't have anything facing the public.

    I would also just run one package at a time to see where the problem may be coming from as well.

    "After installing/giving up/uninstalling/revisiting a dozen times, I think it has left pfsense in a state where there are artifacts remaining from various packages, and the system is simply not stable or performing. "

    This can also be an issue so you're on the right track. I have noticed that for instance with HAVP, if I disable the proxy but I don't clear the checkboxes strange anomalies happen where things would just be very slow etc… Especially if you checked something that uses a RAM DISK. That needs to be unchecked along with any other customizations. I know that other packages have that option so you might want to check that out.

    Also, one way to fix some problems is to go to your console and run a shell and then type fsck.  I think that you only have to run a shell if your console is password protected. Normally I could just press CTRL C to get the # to popup and then you can type fsck. It will check your file system for integrity problems.

    "Hopefully a vanilla install will work.  Although after seeing all the things squid is detecting,"

    That may be your best bet. Until you get a handle on a package I just wouldn't use it and if you're really concerned about younger users and where they go, HAVP worked very well for me when I needed it for that purpose. Simply because, say they go to a site that you don't want such as something complicated where it's not just zzz.youtube.com or whatever it may be. Say it's zzz.cn.thissite.dontgothere.com Let's say the prefix changes from cn to zb. If you put an item on your blocklist like this the site and the whole domain would not be accessible.

    These are the formats that are available for HAVP.

    *Enter each destination URL on a new line that will be accessable to the users without scanning. Use '*' symbol for mask. Example: .github.com/, sourceforge.net/clamav-, /.xml, /.inc

    So you could type in the blacklist area something like this  .thissite.dontgothere.com/  so that even if the prefix changes it's blocked still. You could do it all the way up to just .dontgothere.com/  .  HAVP is very powerful in that effect. As you can see, by typing something like /.xml  you can block all of xml.  You can do the same thing to any extension. You could block anything like .org, .mil, .cn, .php or whatever your fancy is that day. You could essentially do the same thing with the allow list but I don't recommend that. Another thing to consider is to just make your own blacklists.

    I have found that downloading blacklists is not nearly accurate enough to provide a lot of use. Also, there is a great set of rules in snort that prevent going to sites that young people shouldn't be going to. Which is emerging-innapropriate.rules. Just enable them all and if there is a problem find which rule is doing it and suppress it. I had to remove that because it did not work for me. Perhaps Dans Guardian would do a better job.

    Back to HAVP though. Just like any other package of this sort there will be false positives such as when Adobe flash needs to be updated it will flag it as a virus so that's when you have to do your homework and find out exactly what addresses it needs to do the updating without problems and then use the allow list.  Like I said before though. If your Lan rules are golden then you really don't even need these packages. You could just make aliases and block the sites by way of ip address that you don't want people to go to. There's a lot of ways to use pfsense that are made redundant by some packages. Just something to keep in mind. Get used to using the ping tool in pfsense to help with sorting out IP addresses. Then go look it up at CIPB if you want to block an entire IP range via cidr.

    Have a good day.
    Cmellons


Log in to reply