50% performance hit on overall throughput.



  • Dell Optiplex 320
    Pentium Dual-core 1.6ghz
    2gb ddr2 800
    Western Digital 80gb Sata 3
    WAN nic = Broadcom BCM5709 pci-e
    LAN nic= Intel gigabit pci

    Ookla to a local node in my town through PFsense: 21mb/s down
    Ookla to same local node bypass PFsense: 59.26mb/s down

    I have squid3, squidguard, avahi, and that;'s about it as far as packages running..  I suspect my hardware not being able to keep up.. but during download test CPU and ram usage does not spike.  Maybe my old PCI nic?



  • Try turning all those packages off.



  • No help.. I uninstalled every package on my box.  rebooted and still the 20-21mb/s limit.  Cpu is not pegging out, plenty of free memory.  Whats the deal?



  • 119 views, minus my own of course.  This topic obviously interests people but dammit I find it funny how no one out there has any thoughts.  if I were tossing hundreds of bucks at a brand new build people will crawl out of the wood work to throw in their .02

    But some of you guys who've been using PF for years don't have any advice?

    It's highly frustrating how people who run into a tough issue that doesn't really make any sense have such a hard time finding help.  This is supposed to be a community.  Communities works because the 'elders' pass down their experience and knowledge to the less experienced.

    Surely to hell someone out there would have an idea as to my hardware level being sub-par, maybe there some OS level tweak I should be doing… something.

    Hell it's almost enough to make me jump ship and go to some bullshit like Untangle or Sophos.



  • Your hardware is powerful enough, certainly with no packages. You must have some physical problem. Swap cables , check for duplex issues and nic errors. If you have a switch, try connecting it between your modem and pfsense as a test.



  • Bad cables & duplex mismatches are where I'd go first.



  • @Jason:

    Bad cables & duplex mismatches are where I'd go first.

    Agree with this.  For some reason on all my pfsense boxes I have to force full duplex on the wan side.  Does not matter what nic I use.  BTW make sure you know the interface speed on your provider equipment.  You can't force gigabit if your provider is 100baseTX

    @roccor:

    Hell it's almost enough to make me jump ship and go to some bullshit like Untangle or Sophos.

    Don't give up.  pfsense is worth the effort IMO.  The community is good and the product works great once you learn its certain quirks.  What does the interface section on your dashboard look like?




  • Thanks guys.

    First off going by switch/nic LED's everything I have is 1gig full duplex.  I also have to force 1gig/FD on my WAN nic too.  My ISP is Charter, and my cable modem (Cisco DPC3208) has a 1gig ethernet jack.

    Here's some screenshots that might help.






  • Your screenshot shows 52mbps download happening. Is there something on your network downloading that is not connected when you bypass the router to run a test?



  • That's what's messed up.  A speedtest from every host on my network hits a 21mb limit.  But PF itself 'sees' way more than that in traffic.

    And yeah I realize that a web page based bandwidth test is not 100%.  However something has to be causing this hard limit?

    And if the only advice is to screw such pages as that.. then how does one accurately test bandwidth?  A few hundred for ixChariot?



  • stupid question maybe but is that cisco a router or is it in bridge mode?  also do you reboot it after you connect the pfsense box to it?  I only ask because I had a brighthouse cable modem at one time that needed a reboot to properly get the mac address of my pfsense box when I switched it over.



  • Nah beercan it's def a bridge, stupid thing doesn't even have a web interface or status page :(

    and yeah.. any time PF get's shut down for any reason it takes a lifetime of rebooting the modem and pf standing on leg, sticking our my tongue, crossing eyes and crap to get them two synced correctly so the WAN interface pulls an IP correctly.



  • heres a weird question.. the two bce interfaces I am using is a dual 1gigabit nic.. why would PF see no additional features on bce0 but flowcontrol/rxpause/txpause on the other?

    Any chance it could be the lack of/existence of flow control?



  • I am really grasping here but what are your settings in system>advanced>networking?  Try it with all the hardware stuff disabled if it is not already.

    Edited to add – do you have any other nics you can test with?  BTW none of my pfsense boxes show addition functions on the nic but most of mine are em or realtek.




  • did you see this already?  https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards
    it has some tweeks for bce cards



  • Yessir.. already created the loader.conf.local file with the bce entries plus teh one at the bottom with regards to killing flow control.  Admittedly I have no rebooted since adding the flow control line.



  • You're pulling in over 50 Mbps down it clearly shows. You're getting your full speed. What does LAN's traffic graph look like? Guessing it's pushing out over 50 Mbps as well. You're getting your speeds, just spread across multiple devices.

    Looks like you have other things on the network also using bandwidth, which leaves less for your speed tests to use. Many of the "performance hit" threads here are exactly that, wrong perception of what is actually happening. "I plug my laptop in behind the firewall and it's too slow, but unplug the firewall and plug my laptop in directly and it's full speed!" What they neglect to mention is they also plugged in an office of several dozen machines, or at home also plugged in their two kids' laptops that are simultaneously Bittorrenting every movie released in the last year in the entire world. And still expecting somehow speedtest.net is supposed to show their full connection speed.

    @roccor:

    119 views, minus my own of course.  This topic obviously interests people but dammit I find it funny how no one out there has any thoughts.  if I were tossing hundreds of bucks at a brand new build people will crawl out of the wood work to throw in their .02

    Which is a quick and easy thing to throw in an opinion on, and something a lot more people are experienced with than those who know enough to troubleshoot network performance problems. You've actually gotten very good help in this thread anyway.

    @roccor:

    But some of you guys who've been using PF for years don't have any advice?

    It's highly frustrating how people who run into a tough issue that doesn't really make any sense have such a hard time finding help.  This is supposed to be a community.  Communities works because the 'elders' pass down their experience and knowledge to the less experienced.

    Surely to hell someone out there would have an idea as to my hardware level being sub-par, maybe there some OS level tweak I should be doing… something.

    Hell it's almost enough to make me jump ship and go to some bullshit like Untangle or Sophos.

    Because they're just overrun with senior-level network professionals who spend significant amounts of time holding your hand troubleshooting performance issues for free? Which most of the time actually have 0 relation to the firewall itself. No, they don't. It'd probably be hard even as a paid customer of either of those two to get really top notch people to help. Here, if you're willing to put down the money for support, you're working with someone who'd be third level at places like that.

    Granted, this doesn't seem like a difficult one - there is no actual performance degradation. Look at things like traffic graphs on the firewall or switch ports to gauge performance, don't blindly rely on speed test sites.



  • You are wrong here CMB.  Yes normally idiots sitting at home don't realize their kids/wife/parents/whatever are streaming Netflix, torrenting Beiber BS or whatever.. while the same time armchair admin is trying to gauge his throughput.

    I guess I have to apologize for not stating the GD obvious which would be: I have had no other devices hitting the internet when I performed those tests. Period.

    Quite honestly I have no clue what two people you are talking about. I was not referring to any one person in particular, I mean damn there has to be at least a few hundred members of this board who are more experienced at me in freeBSD/PFsense tweaking and usage.

    And like I stated earlier on in my thread here.. I DO NOT expect any website to be 100% accurate.. But really.. a 20mb limit every single time whether I have a loaded LAN segment or not?  removing PFsense displays my results into the upper 50's but with it and zero other pc's/tablets/phones connected stops at 20.  Tell me that doesn't sound at least a tiny bit odd to you.  If it does not strike you as being weird, and you says that's just how it works then fine, I'll shut up.



  • roccor I won't speak for everyone but your tone is wrong. People are on this board helping people for free on their own time, so you can't come on here and make crazy comments because you can't figure your networking issues out. With that being said I don't want to get into a flame war with you, I will try to help.

    1. Have you looked at your Interface Status?
    2. Are you getting any error packets?
    3. A diagram might be helpful.
    4. Maybe a few pings from your host to the firewall might reveal something.
    5. How is your switched network performing?
    6. Can you try to make a transfer from one computer on your network to another?
    7. What is the link speed on your WAN? (Not your provisioned speed)
    8. What is PfSenese reporting your link speed at?
    9. 20Mbps sounds like CAT3 speeds, a poorly terminated cable can cause this.
    10. What type of cable modem do you have?
    11. Is your PfSense getting a private IP or a public IP?
    12. What does your rule set look like?
    13. Is this a clean install?
    14. What version of PfSense
    15. What the the client OS
    16. Are you running a personal firewall on your PC?

    These are just a few quick questions that come off the top of my head.

    There are a lot of questions that one could have, because of the lack of details most people reading your original post would probably would not respond. Now if it were me, I would backup my config file. Wipe my configuration back to factory defaults and then go from there. If performance is as expected then I would add packages one at a time, check performance and continue. I would keep repeating these steps until the problem has manifested itself or the setup you are looking for is complete.


  • Banned

    You are seeing some packetloss on WAN. What IP are you monitoring?



  • The only thing here that seems odd to me is the fact the only screenshot you posted of your traffic graph is actually higher than your connection's limit, so there is nothing here showing any kind of problem, yet you don't answer questions people have to try to help you narrow it down.

    What does the LAN graph look like at the time? WAN always up at ~45-60 Mbps?

    @roccor:

    You are wrong here CMB.  Yes normally idiots sitting at home don't realize their kids/wife/parents/whatever are streaming Netflix, torrenting Beiber BS or whatever.. while the same time armchair admin is trying to gauge his throughput.

    I guess I have to apologize for not stating the GD obvious which would be: I have had no other devices hitting the internet when I performed those tests. Period.

    Based on the information you provided thus far I'm right, you posted a screenshot that proves it - something is downloading at your connection's rated speed and actually in excess of it. The only question is what. Now if that particular traffic graph looks abnormal vs. every other test, post other graphs, that may not be true.

    @roccor:

    Quite honestly I have no clue what two people you are talking about.

    Not people, the two companies/products you offered as some savior.



  • im on Charter with the Cisco cable modem as well. im using a OLD Via c3 1.33Ghz based system with 6 intel Pro/1000's in it with 1G of ram and 80G hard drive.
    granted i only use 2 interfaces.

    i get Solidly more than my rated speed (40Mbit)… Now also realize that cable modem is a shared bandwidth technology. so you could be testing
    in a heavily congested time of the day and your not gonna your speed.

    somethings odd if you have to lock the duplex and speed.... id be looking at this to see whats causing it.



  • Guys, I apologize.  I am normally quick to anger but this past week/weekend was worse and coupled with these weird friggin issues made things worse for me.

    Since I'm an admin in IRL, I chose to work with computers 15 years ago because I lack the people skills to work with people.  That said I tend to try three handfuls of things in trying to resolve a problem but I don't always explain every one of them.  I play the assumption game.. like since I am posting here I assume you guys would know certain things like the not trying to test my throughput while my kids are streaming youtube and the like.

    I get irritated by questions like Mikeisfly posted because I find some of them beneath me.  However I've done my stint in technical support, I know you must treat every caller as an idiot.  That would work here too so if I had a perceived tone then I'm sorry.

    Suncatalyst: Another poster here mentioned he had to force/lock speeds and duplexes on his PF box so I don't feel that that is meaningful of a problem.

    cmb: Aside from these last couple posts from overnight I don't see where I have not answered someones question.  During the time I was running the OOkla tests I do not know what the WAN chart was showing.  I was running them at around 2:30am EST.. tv's, and other computers were all off.  Something would have had to be sucking down data at what.. 15-20mbps in order to cause Ookla to stop at 21mb itself.  Additionally I never used the word savior.. I was pissed and tossed them out there as alternatives to Pf the product.  I thought that was obvious.

    Supermule: I'm not sure I follow you here.. I'm not monitoring any IP.

    Mikeisfly: 1. Interface status are good, up, full duplex and 1 gigabit.
    2. To my knowledge no.
    3. Shortly.
    4. That night all were under 5ms with the occasional spike to 10ms
    5. Ok I guess, no observed weirdness or change from normal
    6. I can this evening
    7.  I have 60mb, but actual like to the modem is 1gb, link from modem to cloud.. no way of knowing.
    8. Link to.. what?
    9. All of my cables are pre-made save for the one feeding the WAP, I spliced it late one night because I did not have my crimpers at home.  While I've never had a problem with splices in the past I can terminate it correctly tonight and see if that was it.
    10. Cisco DPC3208
    11. Public IP
    12. Honestly I have no rules except the builtin couple.
    13. No it's an upgrade from 2.1.2
    14. 2.1.4
    15. Windows 8.1Pro, Windows 7 Pro
    16. Hell no!  Why would someone do that with a PF box?



  • At lunch today.. I have the kids pc's all set to shut down for a couple hours starting at noon.  The Roku was off, ipad was is sleep mode and charging..

    On a whim from Mike I cut and replaced the rj45's on both ends of the wap and my pc's uplink cables.  Visually they all looked pretty ok in condition, but they were both 5-6 years old.  WAN utilization on the Dashboard chart was showing under 1mbps in overall traffic.  Ookla speedtest to nearest node exceeded 45mbps.  That was with squid3 and squidguard all running.  Stopping those two services didn't really change the results.

    The only devices that could have been generating any traffic was my iphone and background traffic from my desktop.  But that's still a much improved result.  Laster I will re-test with everything else unplugged and compare results but it does seem that my once-thought sub-par hardware is good enough to handle the advertised 60mb download rate.

    Your home network is only as strong as it's weakest link.. it sucks that a cable with no visually apparent physical damage was indeed somehow going bad on me, but I guess the simplest causes should have been checked first.



  • @roccor:

    Your home network is only as strong as it's weakest link.. it sucks that a cable with no visually apparent physical damage was indeed somehow going bad on me, but I guess the simplest causes should have been checked first.

    This is the case everywhere, not just at home.  I run into bad cables all the time.  You can try buying better stuff but these days it's all made in China at cut-rate prices.  Buying a "Shielded CAT 7" cable doesn't mean it's any better than normal 5e or 6.



  • True Jason.. to a point.  I've been in IT professionally for over 15 years.  I can count the number of actual bad patch cables I've run into on less than two hands.



  • @roccor:

    True Jason.. to a point.  I've been in IT professionally for over 15 years.  I can count the number of actual bad patch cables I've run into on less than two hands.

    I think I'd run out of fingers just looking at 2013-2014.  All the flaky ones (some Twisted Pair, some Fiber, some TwinAx) ended up being attached to something really critical too.



  • Well, to be fair, there's a difference between being in IT and being a networking professional.

    I can easily see how you would almost never see any bad cables as an IT admin or similar.

    Being in networking.. whole different story though..

    Either that or Mr. Roccor is just that damn good at terminating cables and has top end tools.



  • Ok well out IT shop isn't large enough for a true segregation of duties.. so yes I've terminated all my cabling for at least the past decade.  Who doesn't use Black Box/Belkin rj45's and true 550mhz Cat5e? Ratcheting crimpers are a must.  I forget my kids birthdays.. but I'll remember the T568B color code forever.

    Seriously yeah… cables going bad just hasn't ever really been a problem since a job I had in 2001 where we'd have to re-punch wall jacks quite often.  But thinking back I have no clue what they used for infrastructure cabling so.. hell it coulda been Cat3! lol.



  • ha
    Bad cable issues have bitten me more than once.  It just seems like an item like that should not just go bad but I guess they do.
    I worked on an issue once that drove me crazy, spent 100's of dollars on new equipment and the issue turned out to be a $5 dollar cable.  I felt like ass not figuring it out sooner :)



  • Remember it's the basics. Splicing a cable in itself it not bad if done right (although I prefer a continuous cable). If you just twisted the wires together to make a electrical connection then you are going to get reflections causing retransmits. If you have a managed switch, depending on the kind you have you should have been getting errors.

    @roccor:

    Guys, I apologize.  I am normally quick to anger but this past week/weekend was worse and coupled with these weird friggin issues made things worse for me.

    Since I'm an admin in IRL, I chose to work with computers 15 years ago because I lack the people skills to work with people.  That said I tend to try three handfuls of things in trying to resolve a problem but I don't always explain every one of them.  I play the assumption game.. like since I am posting here I assume you guys would know certain things like the not trying to test my throughput while my kids are streaming youtube and the like.

    I get irritated by questions like Mikeisfly posted because I find some of them beneath me.  However I've done my stint in technical support, I know you must treat every caller as an idiot.  That would work here too so if I had a perceived tone then I'm sorry.

    Suncatalyst: Another poster here mentioned he had to force/lock speeds and duplexes on his PF box so I don't feel that that is meaningful of a problem.

    cmb: Aside from these last couple posts from overnight I don't see where I have not answered someones question.  During the time I was running the OOkla tests I do not know what the WAN chart was showing.  I was running them at around 2:30am EST.. tv's, and other computers were all off.  Something would have had to be sucking down data at what.. 15-20mbps in order to cause Ookla to stop at 21mb itself.  Additionally I never used the word savior.. I was pissed and tossed them out there as alternatives to Pf the product.  I thought that was obvious.

    Supermule: I'm not sure I follow you here.. I'm not monitoring any IP.

    Mikeisfly: 1. Interface status are good, up, full duplex and 1 gigabit.
    2. To my knowledge no.
    3. Shortly.
    4. That night all were under 5ms with the occasional spike to 10ms
    5. Ok I guess, no observed weirdness or change from normal
    6. I can this evening
    7.  I have 60mb, but actual like to the modem is 1gb, link from modem to cloud.. no way of knowing.
    8. Link to.. what?
    9. All of my cables are pre-made save for the one feeding the WAP, I spliced it late one night because I did not have my crimpers at home.  While I've never had a problem with splices in the past I can terminate it correctly tonight and see if that was it.
    10. Cisco DPC3208
    11. Public IP
    12. Honestly I have no rules except the builtin couple.
    13. No it's an upgrade from 2.1.2
    14. 2.1.4
    15. Windows 8.1Pro, Windows 7 Pro
    16. Hell no!  Why would someone do that with a PF box?

    Just to clear up some of the points that I was making:

    4. When pinging your gateway I would expect the ping time to be around 1ms or less consistently.
    8. Typically most people connect their PfSense box to a Switch which is the aggregation point for all the devices on their LAN.
    16. You have to think of your firewall like a draw bridge. You are safe from your enemies out side your kingdom but if one of your machines inside your LAN is infected, because you aren't running a personal firewall on your machine your vulnerable.

    I would like to see a diagram of your network. Remember double natting (this is unnecessary packet processing) is not good either.

    Just as a side note don't take it personal when people ask you for information, you have to remember that most people on these forums if not everyone; don't know who you are, your background … . So when people are trying to help you we need to gather as much information as possible without being able to gather the data ourselves. Especially CMB (He is a founder dude!) I like to solve problems on my own but sometimes that is not possible, so we are fortunate enough that we have a place to go, where us networking geeks can get our geek on. Sometimes I come on these forums and just read other peoples issues and fixes just to add to my virtual tool kit. I know IT people (me included) like to act like we know it all. No one can know everything so we are luck to have this resource.

    Thanks PfSense Team! I challenge everyone to donate some money to the team if you our enjoying this software. I have already made donations and I'm going to make another right now.



  • @roccor:

    True Jason.. to a point.  I've been in IT professionally for over 15 years.  I can count the number of actual bad patch cables I've run into on less than two hands.

    It is pretty unusual but not that unusual if you're in a scenario where you deal with a lot of networking. Via working with our support customers, I see roughly a handful a year, not that many considering the number of boxes. I've been drawing an IT paycheck for roughly 17 years and probably haven't hit triple digits on bad patch cables yet.

    This end result, with something you mentioned earlier, is making me wonder - you mentioned forcing it to gigabit, was it only negotiating to 100 Mb full duplex before you did that? That's precisely what a CAT5 cable would do. Probably half the confirmed patch cable issues I've seen in recent years were CAT5e or 6 cables that had an issue of some sort that prevented gigabit negotiations, they acted as a CAT5 (non-e) cable would in that scenario. Worked fine at 100 Mb though. Trashing and replacing the cable fixed.

    If you were at 100 Mb, and forced an inadequate cable to gigabit, that'd explain everything. If you're negotiating to 100 Mb with two gigabit devices, your cabling is almost certainly the issue. Don't force in that circumstance (or really most any circumstance, people break more than they fix there).

    Also I'd trash rather than replace the ends on any cable that's giving you issues. Yeah most likely the ends are the problem unless some part of the rest of the cable has sustained visible physical damage or excessive twisting, but IMO it's not worth taking the chance (well, maybe at home).



  • I've seen in recent years were CAT5e or 6 cables that had an issue of some sort that prevented gigabit negotiations

    We had to wire a whole section of a data room over because the installers used zip-ties and jacked up the cables clear to the wire tray entries.



  • Yeah no ya'll make perfect sense.  cabling is that one thing that is just there.. never really think about it unless I'm having to make a new patch cord.  I still have a few dozen feet of Cat6 on the spool.. I'll make new and replace my wan, lan and wap cables with cat6 and see what happens.


  • Netgate Administrator

    If the connection is forced to 1Gbps FD but the cable is not up to it, for whatever reason, would you not expect to see errors on the interface? Does the bge driver have sysctl stats like Intel does? (I don't have one here to check). Edit: Yes is does on dev.bge

    I would expect to see some evidence of a problem other than just a seemingly slow throughput from a bad cable. Interesting reading this thread though, a useful diagnostic exercise.  ;)

    Steve



  • Swapped out cables at lunch.. if I specify auto negotiate the port flaps.  If I reset to 1gig/FD I have to restart the modem and Pf before they link up.



  • Try a switch between the modem and pfsense.



  • i had the same issue with PF and the cisco cable modem. switch inbetween solved the problem for me.

    problem Seems to be that the port flaps too quickly on the Cisco and pfsense gets itself wedged somehow
    (DHCP wise) where it wont get a ip via DHCP after that and PF requires a reboot to solve it.
    for me this issue started in the 2.1-Release of PF and still exists if i remove the switch between the cable modem
    and PF.



  • i could add a switch.. and I might have to try it.  But I don't relish the idea of leaving yet another electricity consuming device in there for something as trivial as that.. when hard setting the speed/duplex works too.



  • Oh, I misunderstood. If it's all working to your satisfaction, then yeah don't try adding a switch. I thought you were still unhappy with the performance.