Pfsense Struggling big time.



  • I was about to create a thread but luckily found this thread.
    I am facing this same issue. Someone help me please.



  • saturated  pipe?

    That's another way of saying you're using all of your bandwidth.

    The main leeching seeding is done on my other server (which is not on this network)

    Is it using pfSense as its gateway?

    State table? no clue ill see if I can find something on pfsense

    Look on the dashboard for the System Information widget which has a section titled State table size.

    I can;t seem to find where the logs are ill look

    Status - System logs.



  • I dunno why you are looking north when the problem is south.

    Bandwidth Saturation is common in every installation, not unique to pFsense, and on your first post u disclose starting this deluge thing on the media PC does it, so THAT is your problem.

    When an App takes up all available bandwidth, the network is not smart enough, by default, to say, hold it, I got other clients to service so u can't have the whole "pipe."  Fortunately you have the power to change this.  Pfsense solution is: TRAFFIC SHAPER.



  • Bandwidth Saturation is common in every installation, not unique to pFsense

    He said that the problem goes away if he swaps out pfSense for a consumer-grade router.



  • That's another way of saying you're using all of your bandwidth.

    Not really thought I would says its alot of connections if that means anything

    Is it using pfSense as its gateway?

    No its not even in my house.

    Look on the dashboard for the System Information widget which has a section titled State table size.

    1% (3423/401000) with deluge off
    5% (21722/401000) with it on

    Status - System logs.

    I see this every few lines

    Apr 5 10:22:09 check_reload_status updating dyndns WAN_DHCP
    Apr 5 10:22:09 check_reload_status Restarting ipsec tunnels
    Apr 5 10:22:09 check_reload_status Restarting OpenVPN tunnels/interfaces
    Apr 5 10:22:09 check_reload_status Reloading filter
    Apr 5 10:25:24 rc.gateway_alarm 20213 >>> Gateway alarm: WAN_DHCP (Addr:81.107.216.1 Alarm:1 RTT:87812ms RTTsd:49587ms Loss:21%)

    He said that the problem goes away if he swaps out pfSense for a consumer-grade router.

    Well its better other computers don't drop but do run slow



  • It's possible the consumer grade router is so limited that it is unable to saturate the internet connection or a limited number of states.

    I actually had the inverse issue. I went from a 60/3 cable connection to a 50/50 dedicated fiber connection and when downloading torrents, I found that my Netgear suddenly started to puke. I had to limit the number of connections my torrent client could make to keep it from dying. But prior to fiber, I could use torrent just fine.



  • @Harvy66:

    It's possible the consumer grade router is so limited that it is unable to saturate the internet connection or a limited number of states.

    I actually had the inverse issue. I went from a 60/3 cable connection to a 50/50 dedicated fiber connection and when downloading torrents, I found that my Netgear suddenly started to puke. I had to limit the number of connections my torrent client could make to keep it from dying. But prior to fiber, I could use torrent just fine.

    Seems to be the other way around for me pal.



  • Apr 5 10:25:24  rc.gateway_alarm  20213  >>> Gateway alarm: WAN_DHCP (Addr:81.107.216.1 Alarm:1 RTT:87812ms RTTsd:49587ms Loss:21%)

    This isn't good.  pfSense gateway quality detection thinks your WAN is really flaky.  What does it say under Status - Gateways?



  • @KOM:

    Apr 5 10:25:24  rc.gateway_alarm  20213  >>> Gateway alarm: WAN_DHCP (Addr:81.107.216.1 Alarm:1 RTT:87812ms RTTsd:49587ms Loss:21%)

    This isn't good.  pfSense gateway quality detection thinks your WAN is really flaky.  What does it say under Status - Gateways?

    WAN_DHCP 81.107* 81.107* 12.19ms 4.111ms 0.0% Online Interface WAN_DHCP Gateway
    WAN_DHCP6 fe80::201:5cff:fe80:1447 Pending Pending Pending Pending Interface WAN_DHCP6 Gateway
    with deluge off

    On

    WAN_DHCP 81.107.* 81.107* 598.681ms 216.247ms 38% Offline Interface WAN_DHCP Gateway
    WAN_DHCP6 fe80::201:5cff:fe80:1447 Pending Pending Pending Pending Interface WAN_DHCP6 Gateway



  • Try going to System - Routing - Gateways.  Edit your gateway and check the Disable Gateway Monitoring checkbox and try again.  It seems that your torrent app is filling your pipe to the point that the upstream monitor thinks your link is dying.



  • @KOM:

    Try going to System - Routing - Gateways.  Edit your gateway and check the Disable Gateway Monitoring checkbox and try again.  It seems that your torrent app is filling your pipe to the point that the upstream monitor thinks your link is dying.

    Seems to be the same Kom. Thanks again for the help.



  • Hmmm, crapping out under heavy traffic. What hardware is pfsense installed on?

    I'm betting there's a Realtek NIC in there.



  • @Jailer:

    Hmmm, crapping out under heavy traffic. What hardware is pfsense installed on?

    I'm betting there's a Realtek NIC in there.

    Smoothwall SWG700 I think its called



  • @KOM:

    This isn't good.  pfSense gateway quality detection thinks your WAN is really flaky.  What does it say under Status - Gateways?

    Flaky or stuffed to the gill.



  • While we may never get to the bottom of your actual issue, you would still be best served by some traffic shaping so that your torrent traffic doesn't hog all your bandwidth.



  • @KOM:

    While we may never get to the bottom of your actual issue, you would still be best served by some traffic shaping so that your torrent traffic doesn't hog all your bandwidth.

    Thanks is there some kind of guide on how you do this which you know of KOM



  • Traffic shaping is probably THE hardest element of pfSense to figure out.  Try:

    Youtube Video

    Youtube Video



  • @KOM:

    Traffic shaping is probably THE hardest element of pfSense to figure out.  Try:

    Youtube Video

    Youtube Video

    and this Darkvodka34 https://forum.pfsense.org/index.php?topic=126637.0 for general discussion, and this for message highlighting what to do.



  • @Darkvodka34:

    Thanks is there some kind of guide on how you do this which you know of KOM

    I suggest save your conf, then turn on Traffic Shaper and see what it does, always can revert back by restoring conf.

    If you use the Traffic Shaper's Wizard, it guides you through pages, and on the second page I believe, it presents you with common scenarios you want to deal with and one of them is ta-da, peer-to-peer protocol, u can simply ENABLE it and give it LOW Priority and see what happens.

    In Cisco-land, Traffic Shaper is called Priority Queuing, which is a term simpler to understand what it's doing underneath.

    In Traffic Shaper, a service sits on the WAN port controlling the uploads.  a second service sits on the LAN port controlling the downloads.

    The gists of it is, rather than letting traffic pass though these ports as they come, the TS services hold the packets on queues (or buffer if u prefer), each queue has a priority label, another service takes the packets from the queues and shoot them out the interface. The highest priority queue gets serviced more often than the lower queues.  Think of a traffic cop sitting at a intersection and letting go of the commuter lane for 1 minute, while let go of the next passenger lane only for 15 seconds.  This way the packets flow is controlled, giving a chance for everybody to go through, eventually.

    Well there is more to it, if you really get into it, but last paragraph is the gist and I hope easy to understand.

    Bottom line is, without flow control, an app, in this case peer-to-peer often takes over and everybody else get stuck/freeze.

    Don't know why your consumer grade box works, it could be just a coincidence. There is the possibility that it came with QOS (another Traffic Shaper term) turned on.  Lots of consumer boxes these days come with a variety versions of QOS, in part to deal with VOIP.



  • Your ISP obviously prefers ipv4. For now I would turn ipv6 off. That Gateway pending, pending would bug me. Waste of packets.
    With your ISP speeds your network seems to be slow or buggy from some of your replies here.
    "Well its better other computers don't drop but do run slow".
    Do you try to limit the seeding at all and to be clear your pfsense has direct line to ISP (no modem in front) correct. 21% loss not good.
    Your WAN has adjustable pre-set timings in DHCP Client Configuration under Interfaces/WAN.
    Maybe try spoofing your MAC on the PfSense to what the consumer MAC is. Heck could be many things.
    I would put your ISP unit back and log in and go over what settings it may have that you may have missed, and if you can look at any logs that unit has that could help you figure out things.

    Traffic shaping is great but do not forget the elephant in the room. Your Gateway link is crashing.
    It may have a problem PfSense is just making more obvious.
    https://www.dslreports.com/ Good info here also.



  • Update I turned off Ipv6 and Enabled UPnP & NAT-PMP allow 40000-41000 10.0.0.52/24 40000-41000

    Boom all working like it used to :) Odd right?

    Thanks again to all.



  • Well u must made those changes for a reason, hey as long as it works.



  • Odd it is then.  ;)



  • Nope seen to work fine for about 2 hours now back to how it was before. :(



  • @webtyro:

    Your ISP obviously prefers ipv4. For now I would turn ipv6 off. That Gateway pending, pending would bug me. Waste of packets.
    With your ISP speeds your network seems to be slow or buggy from some of your replies here.
    "Well its better other computers don't drop but do run slow".
    Do you try to limit the seeding at all and to be clear your pfsense has direct line to ISP (no modem in front) correct. 21% loss not good.
    Your WAN has adjustable pre-set timings in DHCP Client Configuration under Interfaces/WAN.
    Maybe try spoofing your MAC on the PfSense to what the consumer MAC is. Heck could be many things.
    I would put your ISP unit back and log in and go over what settings it may have that you may have missed, and if you can look at any logs that unit has that could help you figure out things.

    Traffic shaping is great but do not forget the elephant in the room. Your Gateway link is crashing.
    It may have a problem PfSense is just making more obvious.
    https://www.dslreports.com/ Good info here also.

    Ive turned of Ipv6

    There are no settings on my modem when in "modem mode" Virgin media for you.

    Clone the modem mac? sorry not sure what you mean

    looks like I may need traffic shaping :/



  • I checked Virgin Media and did not see 350/100 speeds. They did have 350/20 up at the website.
    So I assume you are running a Hub in modem mode(bridged) with PfSense behind it. OK this is making more sense.
    Well you did have it running better, progress. Did you try seeding during this time. If you did you still may have the wrong settings with torrent. Too high a rate of seeding (upload) leaves no room for downloads, yes.
    Know your speed too, very important with torrent. Test it.
    Your words:
    "Is bandwidth is unlimited I have no limits my speed is 350mb and 100mb up
    The traffic graph on single torrent is max when there is alot on its goes down.
    Sometimes it will work fine maybe 10% of the time"
    I am sure you  want it 100% of the time. ;)
    I think if you get the torrent tuned better then after that, if you want to tweak more bandwith away from other users then
    I would recommend traffic shaping.
    Why have the firewall work harder when proper torrent settings will work better.
    Check this site out. You may have a torrent problem not router problem.
    https://torrentfreak.com/calculate-your-optimal-bittorrent-settings/



  • im on 100mb up I pay extra for it, My seeding is nothing to do with this server also I do not even go close to max upload no where close.

    Also been using the same settings for years pal, Deluge is set up perfect.



  • uTorrent and Deluge both can auto manage too. That screws my last train of thought. You did mention you ran Ubuntu.



  • @webtyro:

    uTorrent and Deluge both can auto manage too. That screws my last train of thought. You did mention you ran Ubuntu.

    No it was the ubuntu torrent (its great for checking speed.)

    I okay im going to get rid of pfsense and just use pi-hole, I only really used pfsense for an ad-block anyway.

    Thanks again to all.



  • pfSense works fine out of the box. My guess is pfSense is faster, and thus allowing you to saturate your Internet connection, resulting in your performance issues. It could be 7am keeping me from noticing, but I do not see where you mentioned how fast torrent was running when the performance issues started and how fast torrent would run on your £15 one. Is your connection being saturated on pfSense? Is your connection being saturated on your £15 router?



  • @Darkvodka34:

    youtube changes from 1080 to 360 and buffers every second. Also everything is cat 6 leads and all nics are 1GB

    OP…cat6 has long, long time been outdated and that could be your bottle neck...switch to CAT 7.



  • CAT5e would even work just fine. CAT6 hasn't been replaced with CAT7, CAT7 just is a newer standard for newer issues like 10Gb.



  • @NollipfSense:

    @Darkvodka34:

    youtube changes from 1080 to 360 and buffers every second. Also everything is cat 6 leads and all nics are 1GB

    OP…cat6 has long, long time been outdated and that could be your bottle neck...switch to CAT 7.

    Spent £300 on cat7 all over the house everything is now on cat 7 same problem. In fact its prob worse.

    even did a 3GB trunk just to the server same story




  • just a little update if I have one single torrent on just 1 I get 25% packet loss. has anyone any ideas this is driving me nuts. I used to be able to have 150 going without issues



  • @Darkvodka34

    Did you ever get this sorted?

    Just swapped out from a Draytek router to a pfsense box (Netgate branded SG-3100)
    My torrent client kills my WAN - like you I am on Virginmedia but the problem did not happen with the Draytek so I assume the connection is not to blame.

    I have found a way to stop this issue but it is not great. Reduce TOTAL number of torrent connections to 10, and connections per second to '1'. This stops my WAN dying but also means I can't pull more than about 500kb/s over torrent.
    I'm on 350 down 20 up, upload speed pinned to 80kb/s.

    Also, setting "optimization" in Settings -> Advanced to "Aggressive" helped massively.

    for reference on the draytek I had it set to 200 connections, 50 active, 5 upload slots and upload pinned to 80kb/s. Speeds were phenomenal, now I can't have more than 10 connections...in total.

    I just thought I'd trawl the net for a fix before I give up and see if it is any better on a SG-2440 (can borrow one to test)



  • @daleus said in Pfsense Struggling big time.:

    @Darkvodka34

    Did you ever get this sorted?

    Just swapped out from a Draytek router to a pfsense box (Netgate branded SG-3100)
    My torrent client kills my WAN - like you I am on Virginmedia but the problem did not happen with the Draytek so I assume the connection is not to blame.

    I have found a way to stop this issue but it is not great. Reduce TOTAL number of torrent connections to 10, and connections per second to '1'. This stops my WAN dying but also means I can't pull more than about 500kb/s over torrent.
    I'm on 350 down 20 up, upload speed pinned to 80kb/s.

    Also, setting "optimization" in Settings -> Advanced to "Aggressive" helped massively.

    for reference on the draytek I had it set to 200 connections, 50 active, 5 upload slots and upload pinned to 80kb/s. Speeds were phenomenal, now I can't have more than 10 connections...in total.

    I just thought I'd trawl the net for a fix before I give up and see if it is any better on a SG-2440 (can borrow one to test)

    A few questions for you:

    1 - Have you logged into the pfSense console (not the webUI, but the serial console) to see if you are getting any error or strange output there?
    2 - Have you logged into pfSense (best to do this via the console, not webUI) and run top to see what the resource utilization is?
    3 - What other packages are you running, or is this a clean, default installation?
    4 - What do you see in your system logs?

    The root cause is most likely a resource issue, and determining where that's actually occurring requires some investigation. The original poster didn't leave much information about his configuration, but there is a chance he could have been saturating his ISPs gear (hence the WAN resetting all the time) and this never manifested because the consumer gear couldn't support the same amount of throughput as pfSense. There was no comparison of resource utilization between both routers, and more than likely the consumer router didn't have close to the amount of reporting you can get from pfSense.



  • Hi Tim,

    I'll get it hooked up via the USB Console port asap, however there's bugger all on any of the weblog interfaces, just notifications that my backup 3G connection resets every x hours, dyndns updated etc.

    I have been monitoring resources and - as you say, not at console level - but it looks spot on.
    I have been playing with it further since my last post and I have some insights that may prove useful for others in the mean time:

    I'm now up to about 25 connections in my torrent client, the trick appears to be keep the amount of 'connection attempts per second' low and now I can pull much better speed.

    I have also set the status -> gateways page to refresh every second, As I have a modem pf is automatically monitoring the first hop to the ISP. You can see when this issue is about to crop up as suddenly the RTT standard deviation will go from about 2ms and start rising, the RTT follows soon thenafter. The standard values are about 10ms and 2ms respectively. At 60ms and another 60ms standard deviation packetloss will occur and sits around 3% for a few minutes then begins jumping until the WAN is useless.

    Pausing the torrent client and going into states and filtering to the torrent box IP and killing all states recovers the connection; standard deviation drops within a few seconds and RTT eventually makes its way back to 10ms or so.

    Right now with it working I checked the diagnostics - > states summary page and noticed that my torrent server appeared to have 372 states, all of which are UDP. I am wondering if it might be worth while forcing my torrent client to use TCP, as if those connections do not work the firewall will automatically clear them out due to my "aggressive" setting mentioned previously. Interestingly during a "problem" time, the number of states overall was not much higher.

    edit
    Clean installation, I have installed speedtest-cli via 'pkg add', which I installed to help assist diagnose this issue, no additional packages installed or filtering/snort/anything like that, Only one non-standard firewall rule on LAN that sends any traffic destined for my work's VPN IP via a gateway group consisting of internet + 3G backup.


  • Netgate Administrator

    So what exactly happens when the WAN is "killed"?
    You can no longer send any traffic over it?
    Does the torrent client continue to send/recv?

    This seems a lot like it's just saturating the link. Those speeds should be well within the capability of the 3100.
    What download/upload throughput does it show when this happens?

    Steve



  • Solved: Bad sg-3100, borrowed a 2440 which is fine.

    Will contact local reseller and return/swap the 3100. I did connect to the console and reinstall via the adi image (usb) which did naff all, must be hardware related.

    I now have hundreds of connections set on the 2440 and I am a happy happy boy, the only thing I was a bit "eurgh" about is that the 2440 has only 1 LAN port opposed to the built in switch of the 3100 and I need 3 lan ports in the immediate vicinity to the firewall, so I had to bridge LAN,OPT1,OPT2 which works perfectly.

    Thanks all.


Log in to reply