Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    WebGUI dying under heavy load (Internal Server Error 500 etc.) ?

    Scheduled Pinned Locked Moved webGUI
    39 Posts 8 Posters 16.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      sullrich
      last edited by

      Please run this from a shell (option 8) and retest:

      ps awux | grep lighttpd | awk '{ print $2 }' | xargs renice -20

      1 Reply Last reply Reply Quote 0
      • L
        lightning
        last edited by

        @sullrich:

        Please run this from a shell (option 8) and retest:

        ps awux | grep lighttpd | awk '{ print $2 }' | xargs renice -20

        Thanks for the command, cool pipe… :)

        Ok, here some more results:

        case "B" (polling, reproducing as before):

        ftp get: 2876, 2913, 2903, 2952 kbytes/s

        here the top (hitting S to display system processes and C for CPU %) copied just after the transfer (also freezes during the transfer):

        
        last pid:  6432;  load averages:  5.04,  2.88,  2.46              up 0+00:48:23  11:41:20
        76 processes:  9 running, 55 sleeping, 12 waiting
        CPU states:  0.1% user,  0.2% nice,  1.3% system, 98.3% interrupt,  0.0% idle
        Mem: 11M Active, 12M Inact, 16M Wired, 12M Buf, 78M Free
        Swap: 
        
          PID USERNAME  THR PRI NICE   SIZE    RES STATE    TIME    CPU COMMAND
           14 root        1 -44 -163     0K     8K RUN      5:46 85.11% swi1: net
           29 root        1 171   52     0K     8K RUN     37:22 15.38% idlepoll
           15 root        1 -16    0     0K     8K -        0:03  0.05% yarrow
           11 root        1 171   52     0K     8K RUN      0:21  0.00% idle
           28 root        1 171   52     0K     8K pgzero   0:13  0.00% pagezero
           12 root        1 -32 -151     0K     8K WAIT     0:12  0.00% swi4: clock sio
         4514 root        1  96    0  2332K  1560K RUN      0:06  0.00% top
           21 root        1 -64 -183     0K     8K WAIT     0:04  0.00% irq14: ata0
          908 root        1  -8   20  1648K  1156K piperd   0:02  0.00% sh
         2779 root        1  96    0  5564K  2588K select   0:02  0.00% sshd
            4 root        1  -8    0     0K     8K -        0:01  0.00% g_down
          723 root        1   4    0  3076K  2188K kqread   0:01  0.00% lighttpd
         1009 root        1   8   20   240K   144K wait     0:01  0.00% check_reload_status
            2 root        1  -8    0     0K     8K -        0:01  0.00% g_event
            3 root        1  -8    0     0K     8K -        0:01  0.00% g_up
          935 root        1   8  -88  1328K   796K nanslp   0:01  0.00% watchdogd
          729 root        1   8    0  7496K  3780K wait     0:01  0.00% php
           38 root        1 -16    0     0K     8K -        0:01  0.00% schedcpu
          262 root        1  96    0  1344K   976K select   0:01  0.00% syslogd
           58 root        1  -8    0     0K     8K mdwait   0:00  0.00% md1
           50 root        1  -8    0     0K     8K mdwait   0:00  0.00% md0
          794 root        1   8    0  1636K  1160K wait     0:00  0.00% sh
           31 root        1  20    0     0K     8K syncer   0:00  0.00% syncer
          888 root        1  -8    0     0K     8K mdwait   0:00  0.00% md2
          383 root        1 -58    0  3664K  1744K bpf      0:00  0.00% tcpdump
          728 root        1   8    0  7496K  3780K wait     0:00  0.00% php
         3030 root        1  20    0  3544K  2500K pause    0:00  0.00% tcsh
           33 root        1 -16    0     0K     8K sdflus   0:00  0.00% softdepflush
         1012 root        1   8    0  1568K  1240K wait     0:00  0.00% login
         2951 root        1   8    0  1648K  1200K wait     0:00  0.00% sh
           30 root        1 -16    0     0K     8K psleep   0:00  0.00% bufdaemon
           32 root        1  -4    0     0K     8K vlruwt   0:00  0.00% vnlru
            1 root        1   8    0   568K   364K wait     0:00  0.00% init
          867 proxy       1  96    0   656K   452K RUN      0:00  0.00% pftpx
          765 nobody      1  96    0  1328K   948K select   0:00  0.00% dnsmasq
          937 root        1   8    0  1304K   984K nanslp   0:00  0.00% cron
          853 proxy       1   4    0   656K   412K kqread   0:00  0.00% pftpx
          384 root        1  -8    0  1196K   684K piperd   0:00  0.00% logger
         1014 root        1   5    0  1644K  1196K ttyin    0:00  0.00% sh
         1013 root        1   8    0  1640K  1132K wait     0:00  0.00% sh
         6429 root        1 116   20  1632K  1168K RUN      0:00  0.00% sh
          930 dhcpd       1  96    0  2136K  1808K select   0:00  0.00% dhcpd
         6431 root        1 116   20  1492K   624K RUN      0:00  0.00% netstat
         6432 root        1  -8   20  1432K   936K piperd   0:00  0.00% awk
        
        

        Compared to idle:

        
        last pid:  6353;  load averages:  1.15,  2.35,  2.27              up 0+00:46:15  11:39:12
        72 processes:  4 running, 56 sleeping, 12 waiting
        CPU states:  0.4% user,  0.4% nice, 94.6% system,  4.7% interrupt,  0.0% idle
        Mem: 11M Active, 12M Inact, 15M Wired, 12M Buf, 78M Free
        Swap: 
        
          PID USERNAME  THR PRI NICE   SIZE    RES STATE    TIME    CPU COMMAND
           29 root        1 171   52     0K     8K RUN     35:59 89.50% idlepoll
           14 root        1 -44 -163     0K     8K RUN      5:05  3.12% swi1: net
           11 root        1 171   52     0K     8K RUN      0:21  0.00% idle
           28 root        1 171   52     0K     8K pgzero   0:13  0.00% pagezero
           12 root        1 -32 -151     0K     8K WAIT     0:12  0.00% swi4: clock sio
         4514 root        1  96    0  2264K  1492K RUN      0:05  0.00% top
           21 root        1 -64 -183     0K     8K WAIT     0:04  0.00% irq14: ata0
           15 root        1 -16    0     0K     8K -        0:03  0.00% yarrow
          908 root        1   8   20  1648K  1156K wait     0:02  0.00% sh
         2779 root        1  96    0  5564K  2588K select   0:01  0.00% sshd
            4 root        1  -8    0     0K     8K -        0:01  0.00% g_down
          723 root        1   4    0  3076K  2188K kqread   0:01  0.00% lighttpd
         1009 root        1   8   20   240K   144K nanslp   0:01  0.00% check_reload_status
            2 root        1  -8    0     0K     8K -        0:01  0.00% g_event
            3 root        1  -8    0     0K     8K -        0:01  0.00% g_up
          935 root        1   8  -88  1328K   796K nanslp   0:01  0.00% watchdogd
          729 root        1   8    0  7496K  3780K wait     0:01  0.00% php
           38 root        1 -16    0     0K     8K -        0:01  0.00% schedcpu
          262 root        1  96    0  1344K   972K select   0:00  0.00% syslogd
           58 root        1  -8    0     0K     8K mdwait   0:00  0.00% md1
           50 root        1  -8    0     0K     8K mdwait   0:00  0.00% md0
          794 root        1   8    0  1636K  1160K wait     0:00  0.00% sh
           31 root        1  20    0     0K     8K syncer   0:00  0.00% syncer
          888 root        1  -8    0     0K     8K mdwait   0:00  0.00% md2
          383 root        1 -58    0  3664K  1744K bpf      0:00  0.00% tcpdump
          728 root        1   8    0  7496K  3780K wait     0:00  0.00% php
         3030 root        1  20    0  3544K  2500K pause    0:00  0.00% tcsh
         1012 root        1   8    0  1568K  1240K wait     0:00  0.00% login
           33 root        1 -16    0     0K     8K sdflus   0:00  0.00% softdepflush
         2951 root        1   8    0  1648K  1200K wait     0:00  0.00% sh
            1 root        1   8    0   568K   364K wait     0:00  0.00% init
           30 root        1 -16    0     0K     8K psleep   0:00  0.00% bufdaemon
           32 root        1  -4    0     0K     8K vlruwt   0:00  0.00% vnlru
          867 proxy       1   4    0   656K   452K kqread   0:00  0.00% pftpx
          765 nobody      1  96    0  1328K   948K select   0:00  0.00% dnsmasq
          937 root        1   8    0  1304K   984K nanslp   0:00  0.00% cron
          853 proxy       1   4    0   656K   412K kqread   0:00  0.00% pftpx
          384 root        1  -8    0  1196K   684K piperd   0:00  0.00% logger
         1014 root        1   5    0  1644K  1196K ttyin    0:00  0.00% sh
         1013 root        1   8    0  1640K  1132K wait     0:00  0.00% sh
          930 dhcpd       1  96    0  2136K  1808K select   0:00  0.00% dhcpd
          319 root        1  96    0  2832K  2184K select   0:00  0.00% sshd
            9 root        1 -16    0     0K     8K psleep   0:00  0.00% pagedaemon
          155 root        1  96    0   488K   344K select   0:00  0.00% devd
        
        

        I then issued your command:
        ps awux | grep lighttpd | awk '{ print $2 }' | xargs renice -20

        ps awux | grep lighttpd | awk '{ print $2 }' | xargs renice -20

        723: old priority 0, new priority -20
        renice: 6545: getpriority: No such process

        (the process 6545 was the renice command itself ;) , so it worked).

        But no changes compared to previous top (except that lighttpd had nice of -20): still system freeze (no top refresh, no web GUI).

        Tried renicing the process 14, but didn't change its prio.

        I then switched off the poll mode in the advanced settings and got following get throughputs:

        ftp get 2904, 2895 kbytes/s.

        the top looked as follows:

        
        last pid:  8736;  load averages: 10.42,  4.50,  3.17              up 0+01:02:24  11:55:21
        80 processes:  10 running, 58 sleeping, 12 waiting
        CPU states: 19.6% user,  9.1% nice, 70.6% system,  0.8% interrupt,  0.0% idle
        Mem: 13M Active, 12M Inact, 16M Wired, 13M Buf, 76M Free
        Swap: 
        
          PID USERNAME  THR PRI NICE   SIZE    RES STATE    TIME    CPU COMMAND
           14 root        1 -44 -163     0K     8K WAIT     8:13 58.06% swi1: net
           20 root        1 -68 -187     0K     8K WAIT     0:30 37.94% irq10: sis0 sis1+
           11 root        1 171   52     0K     8K RUN      1:50  5.86% idle
         8720 root        1   8    0  7656K  5172K nanslp   0:00  0.34% php
         8651 root        1  -8    0  7668K  5184K piperd   0:00  0.24% php
         8597 root        1  -8    0  7668K  5184K piperd   0:00  0.20% php
          908 root        1  -8   20  1648K  1156K piperd   0:03  0.15% sh
         8628 root        1  -8    0  7668K  5180K piperd   0:00  0.10% php
         8698 root        1 139   20  1444K   972K RUN      0:00  0.05% awk
           29 root        1 171   52     0K     8K RUN     45:29  0.00% idlepoll
           28 root        1 171   52     0K     8K RUN      0:17  0.00% pagezero
           12 root        1 -32 -151     0K     8K RUN      0:15  0.00% swi4: clock sio
           21 root        1 -64 -183     0K     8K WAIT     0:05  0.00% irq14: ata0
           15 root        1 -16    0     0K     8K -        0:04  0.00% yarrow
         2779 root        1  96    0  5564K  2588K select   0:02  0.00% sshd
            4 root        1  -8    0     0K     8K -        0:02  0.00% g_down
            3 root        1  -8    0     0K     8K -        0:01  0.00% g_up
          723 root        1   4  -20  3100K  2212K kqread   0:01  0.00% lighttpd
         1009 root        1   8   20   240K   144K nanslp   0:01  0.00% check_reload_status
            2 root        1  -8    0     0K     8K -        0:01  0.00% g_event
         8259 root        1  96    0  2332K  1560K RUN      0:01  0.00% top
          935 root        1   8  -88  1328K   796K nanslp   0:01  0.00% watchdogd
           38 root        1 -16    0     0K     8K -        0:01  0.00% schedcpu
          729 root        1   8    0  7496K  3780K wait     0:01  0.00% php
           58 root        1  -8    0     0K     8K mdwait   0:01  0.00% md1
          262 root        1  96    0  1344K   980K select   0:01  0.00% syslogd
           50 root        1  -8    0     0K     8K mdwait   0:01  0.00% md0
          794 root        1   8    0  1636K  1160K wait     0:00  0.00% sh
           31 root        1  20    0     0K     8K syncer   0:00  0.00% syncer
          888 root        1  -8    0     0K     8K mdwait   0:00  0.00% md2
          383 root        1 -58    0  3664K  1752K bpf      0:00  0.00% tcpdump
         3030 root        1  20    0  3548K  2520K pause    0:00  0.00% tcsh
          728 root        1   8    0  7496K  3780K wait     0:00  0.00% php
           33 root        1 -16    0     0K     8K sdflus   0:00  0.00% softdepflush
         1012 root        1   8    0  1568K  1240K wait     0:00  0.00% login
           30 root        1 -16    0     0K     8K psleep   0:00  0.00% bufdaemon
           32 root        1  -4    0     0K     8K vlruwt   0:00  0.00% vnlru
          867 proxy       1   4    0   656K   452K kqread   0:00  0.00% pftpx
         2951 root        1   8    0  1648K  1200K wait     0:00  0.00% sh
          853 proxy       1   4    0   656K   412K kqread   0:00  0.00% pftpx
            1 root        1   8    0   568K   364K wait     0:00  0.00% init
          937 root        1   8    0  1304K   984K nanslp   0:00  0.00% cron
          384 root        1  -8    0  1196K   684K piperd   0:00  0.00% logger
         1014 root        1   5    0  1644K  1196K ttyin    0:00  0.00% sh
        
        

        So no difference in throughput, and the interrupt task not taking 100%…

        I then disabled the firewall in advanced settings to check:

        ftp get : 6637, 5548, 6720, 5711 kbytes/s...
        But webGUI and top still blocked !

        (without pfSense, direct connection, I get about 8800 kbytes/s)

        Here the top:

        
        last pid: 24362;  load averages:  1.87,  0.64,  0.34              up 0+02:13:39  13:06:36
        72 processes:  2 running, 57 sleeping, 13 waiting
        CPU states:  3.0% user,  0.2% nice, 14.8% system, 74.0% interrupt,  8.0% idle
        Mem: 11M Active, 11M Inact, 14M Wired, 13M Buf, 80M Free
        Swap: 
        
          PID USERNAME  THR PRI NICE   SIZE    RES STATE    TIME    CPU COMMAND
           20 root        1 -68 -187     0K     8K WAIT     1:16 54.25% irq10: sis0 sis1+
           11 root        1 171   52     0K     8K RUN     62:38 26.51% idle
           15 root        1 -16    0     0K     8K -        0:09  2.34% yarrow
           28 root        1 171   52     0K     8K pgzero   0:43  0.63% pagezero
           29 root        1 171   52     0K     8K pollid  45:29  0.00% idlepoll
           14 root        1 -44 -163     0K     8K WAIT     8:14  0.00% swi1: net
           12 root        1 -32 -151     0K     8K WAIT     0:34  0.00% swi4: clock sio
        16379 root        1  96    0  2264K  1504K RUN      0:18  0.00% top
           21 root        1 -64 -183     0K     8K WAIT     0:09  0.00% irq14: ata0
          908 root        1   8   20  1668K   628K wait     0:06  0.00% sh
          723 root        1   4  -20  3140K  1372K kqread   0:05  0.00% lighttpd
            4 root        1  -8    0     0K     8K -        0:04  0.00% g_down
         1009 root        1   8   20   240K   140K nanslp   0:03  0.00% check_reload_status
            3 root        1  -8    0     0K     8K -        0:03  0.00% g_up
            2 root        1  -8    0     0K     8K -        0:03  0.00% g_event
        16124 root        1 116   20  5564K  2660K select   0:02  0.00% sshd
          935 root        1   8  -88  1328K   356K nanslp   0:02  0.00% watchdogd
          729 root        1   8    0  7496K   848K wait     0:02  0.00% php
           38 root        1 -16    0     0K     8K -        0:02  0.00% schedcpu
           58 root        1  -8    0     0K     8K mdwait   0:01  0.00% md1
           50 root        1  -8    0     0K     8K mdwait   0:01  0.00% md0
           31 root        1  20    0     0K     8K syncer   0:01  0.00% syncer
          262 root        1  96    0  1344K   564K select   0:01  0.00% syslogd
          794 root        1   8    0  1636K   560K wait     0:01  0.00% sh
          888 root        1  -8    0     0K     8K mdwait   0:01  0.00% md2
          383 root        1 -58    0  3664K   472K bpf      0:01  0.00% tcpdump
          728 root        1   8    0  7496K   848K wait     0:00  0.00% php
           33 root        1 -16    0     0K     8K sdflus   0:00  0.00% softdepflush
           32 root        1  -4    0     0K     8K vlruwt   0:00  0.00% vnlru
           30 root        1 -16    0     0K     8K psleep   0:00  0.00% bufdaemon
          853 proxy       1   4    0   656K   272K kqread   0:00  0.00% pftpx
          867 proxy       1   4    0   656K   324K kqread   0:00  0.00% pftpx
        10132 nobody      1  96    0  1328K   984K select   0:00  0.00% dnsmasq
          937 root        1   8    0  1304K   520K nanslp   0:00  0.00% cron
         1012 root        1   8    0  1568K   328K wait     0:00  0.00% login
        16348 root        1  20    0  3452K  2424K pause    0:00  0.00% tcsh
            1 root        1   8    0   568K   224K wait     0:00  0.00% init
          384 root        1  -8    0  1196K   180K piperd   0:00  0.00% logger
        16315 root        1   8    0  1648K  1236K wait     0:00  0.00% sh
         1014 root        1   5    0  1644K   236K ttyin    0:00  0.00% sh
            9 root        1 -16    0     0K     8K psleep   0:00  0.00% pagedaemon
         1013 root        1   8    0  1640K   232K wait     0:00  0.00% sh
          155 root        1  96    0   488K   308K select   0:00  0.00% devd
        10153 dhcpd       1  96    0  2136K  1804K select   0:00  0.00% dhcpd
        
        

        Then I switched on the polling (while keeping the advanced->firewall off):

        ftp get: 6548, 6496, 5880 kbytes/s (without webGUI accesses)
        but "top" command refreshes work and webGUI also, very slowly.
        ftp get: 5244, 5118, 5063, 5177 kbytes/s with 1 webGUI system status refresh during transfer.
        ftp get (without top and webGUI page open): 6606, 6617, 6152 kbytes/s

        Here the top output in the case without webgui:

        
        last pid: 26417;  load averages:  1.80,  1.70,  0.97              up 0+02:20:46  13:13:43
        76 processes:  6 running, 58 sleeping, 12 waiting
        CPU states:  5.7% user,  3.4% nice, 85.7% system,  5.3% interrupt,  0.0% idle
        Mem: 12M Active, 11M Inact, 14M Wired, 13M Buf, 79M Free
        Swap: 
        
          PID USERNAME  THR PRI NICE   SIZE    RES STATE    TIME    CPU COMMAND
           29 root        1 171   52     0K     8K RUN     46:39 45.61% idlepoll
           14 root        1 -44 -163     0K     8K RUN      8:38 34.67% swi1: net
           15 root        1 -16    0     0K     8K -        0:12  2.29% yarrow
           28 root        1 171   52     0K     8K pgzero   0:46  0.54% pagezero
        16379 root        1  96    0  2332K  1572K RUN      0:22  0.34% top
           11 root        1 171   52     0K     8K RUN     64:56  0.00% idle
           20 root        1 -68 -187     0K     8K WAIT     1:46  0.00% irq10: sis0 sis1+
         1014 root        1   5    0  1648K   588K ttyin    1:09  0.00% sh
           12 root        1 -32 -151     0K     8K WAIT     0:37  0.00% swi4: clock sio
           21 root        1 -64 -183     0K     8K WAIT     0:10  0.00% irq14: ata0
          723 root        1   4  -20  3152K  1384K kqread   0:05  0.00% lighttpd
            4 root        1  -8    0     0K     8K -        0:04  0.00% g_down
         1009 root        1   8   20   240K   140K nanslp   0:03  0.00% check_reload_status
            3 root        1  -8    0     0K     8K -        0:03  0.00% g_up
            2 root        1  -8    0     0K     8K -        0:03  0.00% g_event
        16124 root        1 116   20  5564K  2660K select   0:03  0.00% sshd
          935 root        1   8  -88  1328K   356K nanslp   0:02  0.00% watchdogd
          729 root        1   8    0  7496K   848K wait     0:02  0.00% php
           38 root        1 -16    0     0K     8K -        0:02  0.00% schedcpu
           58 root        1  -8    0     0K     8K mdwait   0:02  0.00% md1
           50 root        1  -8    0     0K     8K mdwait   0:01  0.00% md0
          262 root        1  96    0  1344K   564K select   0:01  0.00% syslogd
           31 root        1  20    0     0K     8K syncer   0:01  0.00% syncer
          794 root        1   8    0  1636K   560K wait     0:01  0.00% sh
          888 root        1  -8    0     0K     8K mdwait   0:01  0.00% md2
          383 root        1 -58    0  3664K   472K bpf      0:01  0.00% tcpdump
          728 root        1   8    0  7496K   848K wait     0:00  0.00% php
           33 root        1 -16    0     0K     8K sdflus   0:00  0.00% softdepflush
           32 root        1  -4    0     0K     8K vlruwt   0:00  0.00% vnlru
           30 root        1 -16    0     0K     8K psleep   0:00  0.00% bufdaemon
          853 proxy       1   4    0   656K   272K kqread   0:00  0.00% pftpx
          867 proxy       1   4    0   656K   324K kqread   0:00  0.00% pftpx
        25829 root        1  -8   20  1648K  1192K piperd   0:00  0.00% sh
          937 root        1   8    0  1304K   520K nanslp   0:00  0.00% cron
         1012 root        1   8    0  1568K   328K wait     0:00  0.00% login
        16348 root        1  20    0  3452K  2424K pause    0:00  0.00% tcsh
            1 root        1   8    0   568K   224K wait     0:00  0.00% init
          384 root        1  -8    0  1196K   180K piperd   0:00  0.00% logger
        16315 root        1   8    0  1648K  1236K wait     0:00  0.00% sh
            9 root        1 -16    0     0K     8K psleep   0:00  0.00% pagedaemon
         1013 root        1   8    0  1640K   232K wait     0:00  0.00% sh
          155 root        1  96    0   488K   308K select   0:00  0.00% devd
        25139 root        1 139   20  1388K  1008K select   0:00  0.00% dhclient
        26298 root        1   8    0  1176K   448K nanslp   0:00  0.00% sleep
        
        

        finally, renicing to 0 lighttpd didn't change the accessibility of the webGUI itself (remains slowly but accessible with poll on).

        Conclusions:

        • polling does not change throughput under overload condition
        • still no solution to keep webGUI accessible when firewall is on.
        • the prioritiy of the firewalling process (swi1: net ?) seems very high, meaning that if it's busy defending an attack, the rest of the system freezes…
            (renice command doesn't work to change its priority to test).

        Thoughts ?, Hints ?  ???

        1 Reply Last reply Reply Quote 0
        • S
          sullrich
          last edited by

          Thanks for doing the comprehensive tests.  Let me run it by a couple of FreeBSD developers and get their input.

          1 Reply Last reply Reply Quote 0
          • L
            lightning
            last edited by

            @sullrich:

            Thanks for doing the comprehensive tests.  Let me run it by a couple of FreeBSD developers and get their input.

            Thanks. :) You're right. It looks like it could also affect FreeBSD generally…?! ??? ...so it must certainly already have been discussed and solved. ;)

            1 Reply Last reply Reply Quote 0
            • S
              sullrich
              last edited by

              Solved, I doubt.  What I bet money that people are going to say is that the box is under powered for the task.

              1 Reply Last reply Reply Quote 0
              • L
                lightning
                last edited by

                @sullrich:

                Solved, I doubt.  What I bet money that people are going to say is that the box is under powered for the task.

                As you already understood ;) : I'm not looking at the throughput performance, but to get a non-hanging system in overload conditions.

                They can try a 50 Mbits/s (*) 64-bytes packets load on any up-to-date multi-GHz machine and see if it stays alive  ;D

                Maybe then they will understand that it's not a question of horsepower, but of security design. A DDoS attack can be full line throughput of 64 bytes packets… A firewall should be able to handle that without freezing, whatever the line and size.

                (*) If my math is right:

                • 3000 kbytes/s of 1500 bytes packets is about 2000 packets/s handled on a 266 MHz machine.
                • Means 20'000 PPS on a 2.6 GHz machine, let's say 40'000 to take in account caching and other stuff. * 64 bytes, that's only 20 Mbits/s 64-bytes-packets throughput limit before freezing the system.

                [EDIT: Corrected my math, but conclusions remain same]

                1 Reply Last reply Reply Quote 0
                • S
                  sullrich
                  last edited by

                  Okay, please don't take this the wrong way but you are speaking to the chorus already.  I am just telling you that I know the FreeBSD community well and know how they will react to this.  The box is clearly underpowered.

                  1 Reply Last reply Reply Quote 0
                  • S
                    SatireWolf
                    last edited by

                    No offense, but a 266 mHz box with cheap cheasy realtek adapters being expected to handle a 64 byte packet storm at 30 mB/s is rediculous. The system is meant for 5 mBit and less WAN use. When in doubt, overbuild, don't underbuild your firewalls. You don't have the luxury of super optimized TCP offloading and handling by the NIC's unless you're using nice Intel Server grade PCI-X gigabit adapters (which I do when I can). The real issue of PPS is the fact that you are memory / copy throttled. You can only make a dozen or so memory copies so fast per packet before you run out of internal machine bandwidth. It's not so much GHz as much as memory latency, bus latency, and overhead on the packet processing. If you have specialized packet processors that do nothing but offload TCP headers and minimize copy processes you have something more in line with a Cisco / Juniper networks box.

                    Just speaking from personal experience here, if you want balls to the wall performance with 64 byte packets, overbuild the crap out of the box with an opteron processor or two and really fast low latency memory and a PCI-X TCP offloading nic. I've pushed over 100 mBit of 64 byte packets this way.

                    But yeah, the FreeBSD guys are going to laugh and say, get a bigger box.

                    The only way to fix this behavior at all is to study how many copy's happen between packet input and output, then reduce that number. Or otherwise implement some packet storm throttling / protection to damp down an attack.

                    But yeah, why would you expect a 266 mHz box to perform at 30 mBit? Cisco puts 166-266 mHz processors in their T1 grade routers with full TCP offload and only rate them for 1-2 mBit connections. Just FYI.

                    You won't ever reach overload conditions on a 2 mBit pipe. If you're trying to push a 10 mBit pipe you need a bigger box, period.

                    1 Reply Last reply Reply Quote 0
                    • J
                      JesperDJ
                      last edited by

                      Hi.. i´m dealing with the same "problem" and therefore i ask..:

                      How much hardware do you need to run +80MBit/s througput?

                      Im getting max. 10MBit/s with my pIII 933/512MB mem and a 3Com 3C982 dual NIC

                      If i try the m0n0wall with the same HW the result is 22MBit/s ??

                      1 Reply Last reply Reply Quote 0
                      • H
                        hoba
                        last edited by

                        @JesperDJ:

                        Hi.. i´m dealing with the same "problem" and therefore i ask..:

                        How much hardware do you need to run +80MBit/s througput?

                        Im getting max. 10MBit/s with my pIII 933/512MB mem and a 3Com 3C982 dual NIC

                        If i try the m0n0wall with the same HW the result is 22MBit/s ??

                        Something is wrong here. I'm able to get about 87 mbit/s throughput from one of my C3 1GHz LAN->WAN in factory default configs. If I remove the pfSense I only get about 2 mbit/s more and that is with crappy viarhine onboard nics. Your system should push much more.

                        1 Reply Last reply Reply Quote 0
                        • L
                          lightning
                          last edited by

                          Actually, the throughput should not be measured in Mbits/s but in packets/s (pps), like in routers.

                          This will be putting better light on the true performance, as the packet size usually matters less than the packet handling overhead.

                          Typically, with lots of 1500 bytes packets one way, and few 64 bytes ack packets the other way, if you are well below wirespeed, you can take the Mbits/s figure and divide it roughly by 1500*8 to get the pps throughput. e.g. 22 Mbits/s = 1'830 pps.

                          The true throughput performances are usually made with minimum size packets of 64 bytes, and I would be curious to see some of such performance figures :)

                          Please don't take this as critisism, pfSense is great work and runs great :)

                          1 Reply Last reply Reply Quote 0
                          • J
                            JesperDJ
                            last edited by

                            hmm wierd..

                            in bone stock config i get 45MBit/s with m0n0 and 22 with pfSense..

                            Any ideas?

                            My HW is P3 933CPU on MSI mainboard, 512MB PC133 SD, and a 3Com Dual NIC 3C982, well, it is the same with 2 Realtek 8139C…

                            1 Reply Last reply Reply Quote 0
                            • S
                              sullrich
                              last edited by

                              FreebSD 6.1 is much slower than 4.11.  This is the reason that m0n0wall was hesitant to switch initially.  Why they want to switch now is beyond me because its going to slow down every installation.

                              I would suggest going back to m0n0wall.

                              1 Reply Last reply Reply Quote 0
                              • J
                                JesperDJ
                                last edited by

                                yea but still the m0n0wall isnt doing the job either.. i should be able to get +80MBit througput on my existing hardware…

                                and the pfSense has some of the features i need

                                1 Reply Last reply Reply Quote 0
                                • S
                                  sullrich
                                  last edited by

                                  Well then it sounds like you need beefier hardware then if you want to stick with pfSense.

                                  1 Reply Last reply Reply Quote 0
                                  • J
                                    JesperDJ
                                    last edited by

                                    hmm..  ???

                                    hoba wrote:

                                    "Something is wrong here. I'm able to get about 87 mbit/s throughput from one of my C3 1GHz LAN->WAN in factory default configs. If I remove the pfSense I only get about 2 mbit/s more and that is with crappy viarhine onboard nics. Your system should push much more."

                                    1 Reply Last reply Reply Quote 0
                                    • S
                                      sullrich
                                      last edited by

                                      I can push 40 megabit with a 500 mhz box so I somewhat agree with Holger.

                                      What the exact problem is with your hardware is hard to predict, however.

                                      1 Reply Last reply Reply Quote 0
                                      • J
                                        JesperDJ
                                        last edited by

                                        hmm yes.. im am trying with the default config, så it must be the HW.. im gonna try another machine tommorow then..

                                        Thanks so far :)

                                        1 Reply Last reply Reply Quote 0
                                        • J
                                          JesperDJ
                                          last edited by

                                          Ok.. the results with the new HW

                                          HW: P4 2,5Ghz, 256MB DDR2, Onboard GB NICs

                                          Config: 100% default

                                          pfSense:
                                          Max speed = 60MBit/s
                                          Payload = 100%
                                          System unstabel!
                                          WebGUI = unreachable

                                          m0n0wall:
                                          Max speed = 85MBit/s (the switch couldt handle more, trying another GB Switch tommorrow)
                                          Payload = 30%
                                          System  very stabel!
                                          WebGUI = 100% functioning

                                          seems 2 me that pfSense has a problem?

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            sullrich
                                            last edited by

                                            You are not specifying which type of NICS.  Please specify your NIC types.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.