Performance with- and without pfsense



  • G'evening  ;D

    I've been calling my WAN1 VDSL ISP all kinds of bad names becauses the performance down/up went down from a meager 18 to a meager 10 (no, I wasn't polite on the phone :-[ ).

    However, today I tested my WAN2 Cable. I have a contract 200/20 on that.

    1. Via pfsense, to speedtest.telenet.be: 140 down.
    2. PC directly plugged into modem: 199 down (30 secs later from 1).

    This is a horrible difference (and so the VDSL ISP may not be to blame, nevertheless: no I am not going to send the VDSL ISP roses to apologize, since they are arrogant government *ssholes anyway, as opposed to the cable ISP who are polite and friendly people).

    What may be causing such a huge difference? The interface has Snort on it, is Snort such a horrible performance killer? In the box are Intel NICs only, but could it be my CPU (in my sig) is the bottleneck?

    Are there any advance NIC (or other) tweaks I can implement to change this performance?

    Thank you  :P


  • Banned

    Have you tried on a sane box without any packages?


  • Galactic Empire Netgate

    After reading this https://forum.pfsense.org/index.php?topic=96795.msg540158#msg540158 I think there's something fundamentally wrong with your pfSense install. Do a clean install without any config, preferably on a different machine and to test speeds.



  • @doktornotor:

    Have you tried on a sane box without any packages?

    No, I don't have a sane box: only pfsense ( ;D ;D ;D ;D ;D ).


  • Banned

    @ivor:

    After reading this https://forum.pfsense.org/index.php?topic=96795.msg540158#msg540158 I think there's something fundamentally wrong with your pfSense install.

    Well… afraid changing HW will not help if it ends up again like this:

    :o :o :o



  • @Mr.:

    1. Via pfsense, to speedtest.telenet.be: 140 down.
    2. PC directly plugged into modem: 199 down (30 secs later from 1).

    Your comparing apples and oranges.

    Pfsense manages the states, your modem is essentially stateless and thus no processing or other required overhead to ensure people dont backbone into your system is taking place.

    Try another stateful fw and see how it compares to pfsense, or give pfsense some faster processing capabilities and see how it compares.

    https://en.wikipedia.org/wiki/Stateful_firewall

    Also try a basic setup as Dok suggested as well in case you may have misconfigured anything.

    In pfsense, do backups of the config changes, theres also a facility which maintains the last 10 changes so you can download it as an XML file and compare in a XML editor if thats a way of working you prefer when comparing changes quickly and easily.

    fwiw.

    Edit. Its also worth pointing out, hard disks are the slowest part of the system so any top end Intel Xeon can be made to drag its arse so to speak with a super slow spin disk like a laptop spin disk, likewise a simple celeron with a SSD HD can match the mighty Xeon in some performance tests, as it depends on what instructions are used in the chip amongst other things. The instructions not in a chip have to be emulated in the OS hence a performance hit, so identify the right HW is also useful if thinking about getting some other equipment involved.



  • @ivor:

    After reading this https://forum.pfsense.org/index.php?topic=96795.msg540158#msg540158 I think there's something fundamentally wrong with your pfSense install. Do a clean install without any config, preferably on a different machine and to test speeds.

    Thanks, Igor  ;D

    I will not do that. Because: ever since 2.0 none of the upgrades worked. As such, every upgrade required me to do a fresh install and customize all my packages, and my firewall rules, and aliases, by hand. That takes you 2 days. Admins replied in the past "config restore works, must be something on your side". May be what it is, but I've wasted far too much time on the 'set it and forget it' firewall.

    My Zyxel bloatware didn't have all the features pfsense seems to have, but then again, it also didn't suck so much time (our most precious asset in life, together with health, says dr. economics…) out of me.


  • Galactic Empire Netgate

    @doktornotor:

    Well… afraid changing HW will not help if it ends up again like this:

    :o :o :o

    That goes without saying : ) In the other hand, I've seen some pretty "heavy" pfSense configs, and as long as everything was configured correctly… it worked without issues.


  • Galactic Empire Netgate

    @Mr.:

    @ivor:

    After reading this https://forum.pfsense.org/index.php?topic=96795.msg540158#msg540158 I think there's something fundamentally wrong with your pfSense install. Do a clean install without any config, preferably on a different machine and to test speeds.

    Thanks, Igor  ;D

    I will not do that. Because: ever since 2.0 none of the upgrades worked. As such, every upgrade required me to do a fresh install and customize all my packages, and my firewall rules, and aliases, by hand. That takes you 2 days. Admins replied in the past "config restore works, must be something on your side". May be what it is, but I've wasted far too much time on the 'set it and forget it' firewall.

    My Zyxel bloatware didn't have all the features pfsense seems to have, but then again, it also didn't suck so much time (our most precious asset in life, together with health, says dr. economics…) out of me.

    Then I will just link my reply to you from here https://forum.pfsense.org/index.php?topic=96795.msg540411#msg540411



  • @doktornotor:

    @ivor:

    After reading this https://forum.pfsense.org/index.php?topic=96795.msg540158#msg540158 I think there's something fundamentally wrong with your pfSense install.

    Well… afraid changing HW will not help if it ends up again like this:

    :o :o :o

    You're trolling me, Dok (you may do so by now, as I've discovered you're not the bad wulf  ;D ). That pic is old: squid and squidguard are gone.

    I previously also posted top, but will do it again:

    
    last pid: 76817;  load averages:  0.15,  0.20,  0.21                                                                                                                                                                 up 0+04:35:42  19:13:08
    63 processes:  1 running, 58 sleeping, 4 zombie
    CPU:  0.6% user,  0.0% nice,  0.6% system,  0.8% interrupt, 98.0% idle
    Mem: 360M Active, 2175M Inact, 1205M Wired, 528K Cache, 2009M Buf, 12G Free
    Swap: 32G Total, 32G Free
    
      PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
    14853 root          8  20    0  1984M  1881M uwait   1   6:11   0.88% suricata
    22287 root         15  20    0   219M 92964K nanslp  0   1:22   0.68% ntopng
    14138 root        150  20    0   193M 21948K uwait   0   0:25   0.00% filterdns
    23911 root          1  20    0 14656K  2436K select  1   0:20   0.00% syslogd
    96188 nobody        1  20    0 19060K  3516K select  0   0:11   0.00% darkstat
    63665 root          1  20    0 21720K  5852K select  1   0:07   0.00% openvpn
    30669 root          1  20    0 12456K  2180K select  0   0:06   0.00% apinger
    71884 unbound       2  20    0 88488K 32700K kqread  0   0:05   0.00% unbound
    17917 root          3  52    0 24572K  4716K uwait   0   0:03   0.00% redis-server
    49979 dhcpd         1  20    0 24812K 13732K select  1   0:02   0.00% dhcpd
    39033 root          1  20    0 50788K 10960K kqread  0   0:02   0.00% lighttpd
    66015 root          1  20    0 21720K  5832K select  0   0:02   0.00% openvpn
    65501 root          2  20    0   783M   386M nanslp  0   0:01   0.00% snort
    99052 root          1  20    0 14540K  2080K select  0   0:01   0.00% powerd
    79354 root          1  52   20 17136K  2708K wait    0   0:01   0.00% sh
      249 root          1  20    0   224M 23864K kqread  1   0:01   0.00% php-fpm
    27472 root          1  20    0 16804K  2340K bpf     1   0:01   0.00% filterlog
    89390 root          1  20    0 55720K  7336K bpf     0   0:00   0.00% bandwidthd
    91338 root          1  20    0 55720K  7252K bpf     0   0:00   0.00% bandwidthd
    90609 root          1  20    0 55720K  7236K bpf     0   0:00   0.00% bandwidthd
    89470 root          1  20    0 55720K  7312K bpf     0   0:00   0.00% bandwidthd
    90317 root          1  20    0 55720K  7276K bpf     0   0:00   0.00% bandwidthd
    91063 root          1  20    0 55720K  7248K bpf     0   0:00   0.00% bandwidthd
    90849 root          1  20    0 55720K  7292K bpf     0   0:00   0.00% bandwidthd
    89712 root          1  20    0 55720K  7288K bpf     0   0:00   0.00% bandwidthd
    26816 root          1  20    0 28164K 18052K select  1   0:00   0.00% ntpd
    14226 root          1  52    0 16664K  2524K nanslp  1   0:00   0.00% cron
     6133 root          1  20    0 43604K  6296K select  0   0:00   0.00% mpd5
    30999 root          1  20    0 28344K  3004K piperd  1   0:00   0.00% rrdtool
    99043 uucp          1  20    0 18832K  2580K nanslp  1   0:00   0.00% upsmon
    40664 root          1  20    0 55624K  6216K select  1   0:00   0.00% sshd
    40320 root          6  20    0   737M 16308K usem    0   0:00   0.00% radiusd
      264 root          1  40   20 19024K  2580K kqread  1   0:00   0.00% check_reload_status
    24280 root          1  20    0   224M 37024K accept  0   0:00   0.00% php-fpm
    28002 root          1  20    0 18780K  2344K select  0   0:00   0.00% inetd
      277 root          1  20    0 13164K  4464K select  1   0:00   0.00% devd
    41275 root          1  24    0 17136K  2756K wait    0   0:00   0.00% sh
    40969 root          2  20    0 14748K  2312K nanslp  1   0:00   0.00% sshlockout_pf
    54468 root          1  40    0 12404K  2008K nanslp  1   0:00   0.00% minicron
    43186 root          1  35    0 17476K  3856K pause   1   0:00   0.00% tcsh
    41378 root          1  52    0 17136K  2664K wait    1   0:00   0.00% sh
    76817 root          1  20    0 21988K  3152K CPU0    0   0:00   0.00% top
     7016 root          1  20    0 32420K  5228K select  0   0:00   0.00% sshd
    72822 root          1  20    0 12408K  2224K kqread  0   0:00   0.00% dhcpleases
    42562 root          1  20    0 43568K  2800K wait    0   0:00   0.00% login
    58733 root          2  20    0 14748K  2312K nanslp  0   0:00   0.00% sshlockout_pf
     7202 root          2  20    0 14748K  2220K nanslp  0   0:00   0.00% sshlockout_pf
    42883 root          1  21    0 17136K  2776K wait    1   0:00   0.00% sh
    42916 root          1  52    0 17136K  2660K ttyin   0   0:00   0.00% sh
    18833 nagios        1  52    0 23180K  4956K select  1   0:00   0.00% nrpe2
    98998 root          1  52    0 18832K  2552K piperd  0   0:00   0.00% upsmon
    54781 root          1  20    0 12404K  2008K nanslp  0   0:00   0.00% minicron
    96433 nobody        1  52    0 19060K  2396K sbwait  0   0:00   0.00% darkstat
    71115 root          1  52   20  8304K  1952K nanslp  1   0:00   0.00% sleep
    54289 root          1  20    0 12404K  1996K wait    1   0:00   0.00% minicron
    54475 root          1  21    0 12404K  1996K wait    1   0:00   0.00% minicron
    55145 root          1  21    0 12404K  1996K wait    1   0:00   0.00% minicron
      266 root          1  52   20 19024K  2404K kqread  1   0:00   0.00% check_reload_status
    55546 root          1  20    0 12404K  2008K nanslp  1   0:00   0.00% minicron
    
    

    I'm not saying my hardware could not be the cause, but from looking into these numbers I don't get that impression.



  • @Mr.:

    @ivor:

    After reading this https://forum.pfsense.org/index.php?topic=96795.msg540158#msg540158 I think there's something fundamentally wrong with your pfSense install. Do a clean install without any config, preferably on a different machine and to test speeds.

    Thanks, Igor  ;D

    I will not do that. Because: ever since 2.0 none of the upgrades worked.

    Just the other day I installed a 2.2.2 backup onto 2.2.0 and got the warning message on the console pointing out some things may not work as the backup is from a later version of pfsense. It still worked complete with rules & snort no problem, and the Firmware upgrade to bring it up to 2.2.2 worked fine.

    As such, every upgrade required me to do a fresh install and customize all my packages, and my firewall rules, and aliases, by hand. That takes you 2 days.

    10 mins max in my experience and thats even when reediting the XML backups to change IP addresses and names.

    Admins replied in the past "config restore works, must be something on your side". May be what it is, but I've wasted far too much time on the 'set it and forget it' firewall.

    Check out the backup and restore, others have and will draw their own conclusions about whether it works or not.

    For me it works even when using a backup from a later version of pfsense in an earlier installation of pfsense as mentioned above. Not many other systems have that backward compatibility even with mainstream server backup facilities.

    My Zyxel bloatware didn't have all the features pfsense seems to have, but then again, it also didn't suck so much time (our most precious asset in life, together with health, says dr. economics…) out of me.

    In a true DMZ using 2 firewalls, https://en.wikipedia.org/wiki/DMZ_(computing)#Dual_firewall

    Keep your Zyxel doing what you are happy with it doing and make the pfsense fill the gaps. How about that for a solution?


  • Banned

    Well seriously, if you want to test performance/throughput… You realize, that each packet on that box is copied at least 5 times? (snort, suricata, ntopng, bandwidthd, darkstat...)


  • Galactic Empire Netgate

    @doktornotor:

    Well seriously, if you want to test performance/throughput… You realize, that each packet on that box is copied at least 5 times? (snort, suricata, ntopng, bandwidthd, darkstat...)

    Look at the size of config backup. https://forum.pfsense.org/index.php?topic=96795.msg540460#msg540460 I think maybe he should send it to us (pfSense support) for dissection.



  • @firewalluser:

    Keep your Zyxel doing what you are happy with it doing and make the pfsense fill the gaps. How about that for a solution?

    Thank you for your reply, kind problem solving suggestion  ;D

    The problem is: I got so fed up with the Zyxel crap I threw it away and thought pfsense was my new great love (after WIFE and my Rottweilers, my dearest loves of all).

    The 'funny' thing is: I'm only a stupid economist, so you all guys can shoot me when it comes to IT knowledge. As a side effect, I work for one of the biggest Fortune-500 companies in the world, as a country CFO. As such country IT is on my desk too. My IT admins (they're not noobs, PhD's from serious tech universities) also tell me they have problems with pfsense in their test environments. That only helps me think maybe I'm not always the stupid noob  :-[


  • Galactic Empire Netgate

    @Mr.:

    The 'funny' thing is: I'm only a stupid economist, so you all guys can shoot me when it comes to IT knowledge. As a side effect, I work for one of the biggest Fortune-500 companies in the world, as a country CFO. As such country IT is on my desk too. My IT admins (they're not noobs, PhD's from serious tech universities) also tell me they have problems with pfsense in their test environments. That only helps me think maybe I'm not always the stupid noob  :-[
    [/quote]

    Working for Fortune-500 company doesn't make you somehow universally knowledgeable. Same goes for PhD's.

    pfSense isn't zyxel for a reason, it takes time and knowledge to configure pfSense correctly. That being said, I'm surprised you didn't reach out to pfSense support or got official pfSense appliance since you do work for Fortune-500 company.



  • pfSense isn't zyxel for a reason, it takes time and knowledge to configure pfSense correctly

    And less bugs, and better documentation. Which is not pointing at this thread, but at other topics.

    @ivor:

    Working for Fortune-500 company doesn't make you somehow universally knowledgeable.

    There is a reason why I am the self proclaimed eternal noob on this forum. I never said I am 'universally knowledgeable'. If I were I wouldn't be asking here for help.

    Same goes for PhD's.

    I have two of these titles. We like to think we know more about our fields than the one zillion 'For dummies' people who google their way to the next point-and-click. My field is economics, theirs is designing IT-infrastructures in the broadest sense. I seem to be an expert in economics yet a noob in networking (still no good book to be found, out of the gazillion books written), my admins are experts in their field yet noobs in economics. Life.

    That being said, I'm surprised you didn't reach out to pfSense support or got official pfSense appliance since you do work for Fortune-500 company.

    You may be surprised all you want, I will enlighten you: this is my home setup. pfsense support and pfsense appliances are too expensive for home users. And pfsense is not ready for a Fortune-500 company, so my admins only play with pfsense as they play with around 100000 projects. I even have budget for them to play with.


  • Galactic Empire Netgate

    That is simply not true. pfSense is being used in almost every possible industry available…  I don't want to start a argument, but what you're saying is wrong and you've pointed out multiple times that you're not exactly the most knowledgeable person regarding pfSense or IT. That's just a bad corporate-drone philosophy, which is completely false.

    Not to mention that you compared pfSense with SAP in a different thread, which is literally the most hated product by any knowledgeable admin of Fortune-500 companies.



  • @ivor:

    Not to mention that you compared pfSense with SAP in a different thread, which is literally the most hated product by any knowledgeable admin of Fortune-500 companies.

    I will leave it at this, Igor.


  • Galactic Empire Netgate

    @Mr.:

    I will leave it at this, Igor.

    It's Ivor actually. I'd suggest you have a really honest conversation with admins in your company regarding SAP.



  • @ivor:

    @Mr.:

    I will leave it at this, Igor.

    It's Ivor actually. I'd suggest you have a really honest conversation with admins in your company regarding SAP.

    I'll not leave it at this, Igor, I'll respond to this since you're seriously pissing me off. Yes, seriously.

    and you've pointed out multiple times that you're not exactly the most knowledgeable person regarding pfSense or IT

    I am the eternal noob on pfsense. But I also have two PhD's in economics from universities most people only dream of. Meaning: 'tmight be that the problem with pfsense is that it isn't perfect (if you catch my drift), nor is the documentation.

    You comfortably moved in 'or IT', by means of a fallacy. You should make that less obvious.

    You should not bully me about SAP, and me 'having to talk to my admins about SAP'. It so happens I have that very special badge SAP issues to very few people, very-few-people, they even have an official word for that badge.

    I was doing SAP in 1992, for Walldorf. And ever since. You are probably referencing a friend of yours who didn't pass the exams, and found SAP way too difficult. Which might very well be true: it is about 140 zillion times more complex than pfsense.

    You've managed to piss me off more than any other person on this forum in my years here, Igor, with your insults.

    Bless you.


  • Galactic Empire Netgate

    @Mr.:

    @ivor:

    @Mr.:

    I will leave it at this, Igor.

    It's Ivor actually. I'd suggest you have a really honest conversation with admins in your company regarding SAP.

    I'll not leave it at this, Igor, I'll respond to this since you're seriously pissing me off. Yes, seriously.

    and you've pointed out multiple times that you're not exactly the most knowledgeable person regarding pfSense or IT

    I am the eternal noob on pfsense. But I also have two PhD's in economics from universities most people only dream of. Meaning: 'tmight be that the problem with pfsense is that it isn't perfect (if you catch my drift), nor is the documentation.

    You comfortably moved in 'or IT', by means of a fallacy. You should make that less obvious.

    You should not bully me about SAP, and me 'having to talk to my admins about SAP'. It so happens I have that very special badge SAP issues to very few people, very-few-people, they even have an official word for that badge.

    I was doing SAP in 1992, for Walldorf. And ever since. You are probably referencing a friend of yours who didn't pass the exams, and found SAP way too difficult. Which might very well be true: it is about 140 zillion times more complex than pfsense.

    You've managed to piss me off more than any other person on this forum in my years here, Igor, with your insults.

    Bless you.

    What you do and for how long is really not my concern. However, how people behave on forum is something that concerns me. So please, change your attitude and behave politely. I too found your multiple threads annoying and full of false accusations yet I was nice and polite in attempt to reason with you.

    That being said, since you're obviously not paying attention, my name is Ivor, not Igor. Perhaps you should pay more attention with your pfSense config as well.


  • Banned



  • @Mr.:

    @firewalluser:

    Keep your Zyxel doing what you are happy with it doing and make the pfsense fill the gaps. How about that for a solution?

    also tell me they have problems with pfsense in their test environments.

    Can you spill the beans on this?

    So effectively you are still stuck at the first post, ie its not running fast enough?

    Can you say what HW you have?

    Some programming languages are not the quickest at processing, I havent looked at what code is used in pfsense, but I know php is not as fast as its interpreted which means its got to go through another program which then talks to the OS or baremetal. C/C++/Assembler can be baremetal languages talking straight to HW, cutting out the OS but can also talk the OS in most cases which could explain in part why you dont see the speed. The zxyel is likely to have the code on a chip which generally is faster than having code go through an OS, simple example Intel AES (encryption) on the chip will always be faster than Windows doing AES encryption as another example. So back in the earlier days you will remember the who-ha about pentium MX's having the MX instruction set on the cpu included to speed up windows, its the same sort of thing.

    Having the Milgram obedience to authority of certain uni attendance along with PHd's etc can be lucrative, this I dont deny, but does the Asch conformity of education make one a more clever person than another is often proved by criminals. Not knocking you, just giving you a different perspective.  ;)

    Edit. Its also worth pointing out that some of the firewalling is actually freebsd, some of it will be pfsense so your phd's might be seeing with problems with freebsd, until we know more its difficult to say where the problems exist WRT your phd's observations.


  • Netgate

    @doktornotor:

    I LOLed.


  • Netgate

    comparing SAP and pfSense is a major category mistake.

    SAP Business One costs $2,975 per-user up front, and then 18% of total software cost on an annual, go forward basis.

    This is a pfSense board.  We are not here to discuss SAP, nor your education, nor your CISSP/CCNA/CCNP/CCIE/PhD/…, nor the "dismal science".

    Keep it on-topic.