NAT Reflection / Massive inetd with UDP



  • Hi, i have checked out the forums for this and the only thing i can really see is something for v.1.2.3 and nothing with 2.0.

    I am running 2.0.1-RELEASE and my ram is getting eaten and i think it is due to mumble that i am hosting, it normally uses UDP but can fallover to TCP, so i might change that.

    This is my current inetd.conf

    
    [2.0.1-RELEASE][admin@firewall.home]/etc(61): cat /var/etc/inetd.conf
    tftp-proxy      dgram   udp     wait            root    /usr/libexec/tftp-proxy tftp-proxy -v
    19000   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.24 22
    19000   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.24 22
    19001   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.10 80
    19001   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.10 80
    19002   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.8 64738
    19002   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.8 64738
    19003   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.2 443
    19003   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.2 443
    19004   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.2 902
    19004   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.2 902
    19005   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.24 6667
    19005   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.24 6667
    19006   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.30 22
    19006   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.30 22
    19007   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.6 25565
    19007   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.6 25565
    19008   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.6 8140
    19008   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.6 8140
    19009   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.6 5839
    19009   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.6 5839
    19010   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.9 22
    19010   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.9 22
    19011   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.9 25566
    19011   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.9 25566
    19012   stream  tcp     nowait/0        nobody  /usr/bin/nc     nc -w 2000 10.0.0.14 22
    19012   dgram   udp     nowait/0        nobody  /usr/bin/nc     nc -u -w 2000 10.0.0.14 22
    
    

    And the current processes that tells me it is from the machine that is running mumble, the only thing that machine has is mumble so it was kinda easy to figure that out.

    
    [2.0.1-RELEASE][admin@firewall.home]/etc(62): ps aux | grep nc
    root      22  0.0  0.0     0     8  ??  DL    8:18AM   0:01.59 [syncer]
    nobody   439  0.0  0.1  3344   792  ??  Ss    8:27AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody   612  0.0  0.1  3344   792  ??  Ss    8:38AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody  1247  0.0  0.1  3344   796  ??  Is    2:18PM   0:00.01 nc -w 2000 10.0.0.2 443
    nobody  2041  0.0  0.1  3344   792  ??  Ss    8:46AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody  2564  0.0  0.1  3344   792  ??  Ss    8:29AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody  4594  0.0  0.0  3344    92  ??  Ss    8:19AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody  4847  0.0  0.0  3344    92  ??  Ss    8:19AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody  4848  0.0  0.0  3344    92  ??  Ss    8:19AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody  4887  0.0  0.0  3344    92  ??  Ss    8:19AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody  5139  0.0  0.0  3344    92  ??  Ss    8:21AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody  5146  0.0  0.0  3344    92  ??  Ss    8:19AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody  5183  0.0  0.0  3344    92  ??  Ss    8:19AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody  5296  0.0  0.0  3344    92  ??  Ss    8:19AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody  5906  0.0  0.0  3344    92  ??  Ss    8:19AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody  6005  0.0  0.0  3344    68  ??  Ss    8:19AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody  6280  0.0  0.0  3344    92  ??  Ss    8:19AM   0:00.08 nc -u -w 2000 10.0.0.8 64738
    nobody  7322  0.0  0.1  3344   792  ??  Ss    8:27AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody  9488  0.0  0.1  3344   796  ??  Is    2:25PM   0:00.01 nc -w 2000 10.0.0.2 443
    nobody 11284  0.0  0.1  3344   792  ??  Ss    8:34AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody 12114  0.0  0.1  3344   792  ??  Ss    8:47AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 14147  0.0  0.0  3344    92  ??  Ss    8:21AM   0:00.12 nc -u -w 2000 10.0.0.8 64738
    nobody 16836  0.0  0.0  3344    92  ??  Ss    8:20AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody 17758  0.0  0.1  3344   792  ??  Ss    8:26AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 19185  0.0  0.1  3344   792  ??  Ss    8:40AM   0:00.02 nc -u -w 2000 10.0.0.8 64738
    nobody 19940  0.0  0.1  3344   792  ??  Ss    8:58AM   0:00.03 nc -u -w 2000 10.0.0.8 64738
    nobody 21037  0.0  0.0  3344    92  ??  Ss    8:23AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody 24271  0.0  0.0  3344    68  ??  Ss    8:20AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 25239  0.0  0.1  3344   792  ??  Ss    8:25AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody 27651  0.0  0.0  3344    92  ??  Ss    8:23AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody 28439  0.0  0.1  3344   792  ??  Ss    8:32AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody 28511  0.0  0.0  3344    68  ??  Ss    8:19AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody 29738  0.0  0.0  3344    92  ??  Ss    8:20AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody 31308  0.0  0.1  3344   792  ??  Ss    8:57AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody 31969  0.0  0.1  3344   792  ??  Ss    8:35AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 32583  0.0  0.1  3344   792  ??  Ss    8:29AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 32785  0.0  0.1  3344   792  ??  Ss    9:36AM   0:00.02 nc -u -w 2000 10.0.0.8 64738
    nobody 35357  0.0  0.1  3344   792  ??  Ss    8:28AM   0:00.06 nc -u -w 2000 10.0.0.8 64738
    nobody 37386  0.0  0.1  3344   792  ??  Ss    8:49AM   0:00.03 nc -u -w 2000 10.0.0.8 64738
    nobody 38815  0.0  0.0  3344    92  ??  Ss    8:21AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody 38871  0.0  0.1  3344   792  ??  Ss    8:36AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 39291  0.0  0.0  3344    92  ??  Ss    8:27AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody 43660  0.0  0.1  3344   792  ??  Ss    9:12AM   0:00.04 nc -u -w 2000 10.0.0.8 64738
    nobody 44298  0.0  0.1  3344   792  ??  Ss    9:00AM   0:00.02 nc -u -w 2000 10.0.0.8 64738
    nobody 48674  0.0  0.1  3344   792  ??  Is    2:00PM   0:00.01 nc -w 2000 10.0.0.2 443
    nobody 50359  0.0  0.1  3344   792  ??  Ss    9:28AM   0:00.03 nc -u -w 2000 10.0.0.8 64738
    nobody 50546  0.0  0.1  3344   792  ??  Ss    9:45AM   0:00.03 nc -u -w 2000 10.0.0.8 64738
    nobody 50635  0.0  0.0  3344   352  ??  Ss    8:19AM   0:02.34 nc -w 2000 10.0.0.8 64738
    nobody 51204  0.0  0.1  3344   792  ??  Ss    8:56AM   0:00.02 nc -u -w 2000 10.0.0.8 64738
    nobody 52126  0.0  0.1  3344   792  ??  Ss    1:46PM   0:01.11 nc -w 2000 10.0.0.24 22
    nobody 55239  0.0  0.0  3344    92  ??  Ss    8:20AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 55350  0.0  0.1  3344   792  ??  Ss    8:28AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 56758  0.0  0.0  3344    92  ??  Ss    8:29AM   0:00.16 nc -u -w 2000 10.0.0.8 64738
    nobody 57279  0.0  0.1  3344   792  ??  Ss    8:50AM   0:00.07 nc -u -w 2000 10.0.0.8 64738
    nobody 57595  0.0  0.1  3344   792  ??  Ss    8:29AM   0:00.02 nc -u -w 2000 10.0.0.8 64738
    nobody 60742  0.0  0.1  3344   792  ??  Ss    8:25AM   0:00.04 nc -u -w 2000 10.0.0.8 64738
    nobody 61610  0.0  0.1  3344   792  ??  Ss    8:23AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 62499  0.0  0.1  3344   792  ??  Ss    9:04AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    nobody 63493  0.0  0.1  3344   792  ??  Ss    8:38AM   0:00.05 nc -u -w 2000 10.0.0.8 64738
    
    

    And this is some of the inetd's

    
    root   62367  0.0  0.1  3436   832  ??  I    11:27AM   0:00.00 inetd: wrapping (inetd)
    root   62415  0.0  0.1  3436   832  ??  I     9:30AM   0:00.00 inetd: wrapping (inetd)
    root   62432  0.0  0.1  3436   832  ??  I     1:20PM   0:00.00 inetd: wrapping (inetd)
    root   62474  0.0  0.1  3436   832  ??  I    10:27AM   0:00.00 inetd: wrapping (inetd)
    root   62475  0.0  0.1  3436   832  ??  I    11:38AM   0:00.00 inetd: wrapping (inetd)
    root   62577  0.0  0.1  3436   832  ??  I     1:00PM   0:00.00 inetd: wrapping (inetd)
    root   62596  0.0  0.0  3436   472  ??  I     8:30AM   0:00.00 inetd: wrapping (inetd)
    root   62606  0.0  0.0  3436   472  ??  I     8:22AM   0:00.00 inetd: wrapping (inetd)
    root   62616  0.0  0.1  3436   832  ??  I     9:54AM   0:00.00 inetd: wrapping (inetd)
    root   62774  0.0  0.1  3436   832  ??  I     1:00PM   0:00.00 inetd: wrapping (inetd)
    root   62801  0.0  0.1  3436   832  ??  I    12:23PM   0:00.00 inetd: wrapping (inetd)
    root   62811  0.0  0.1  3436   832  ??  I    11:10AM   0:00.00 inetd: wrapping (inetd)
    root   62907  0.0  0.1  3436   832  ??  I     9:19AM   0:00.00 inetd: wrapping (inetd)
    root   62911  0.0  0.1  3436   832  ??  I     1:19PM   0:00.00 inetd: wrapping (inetd)
    root   62944  0.0  0.1  3436   832  ??  I    12:58PM   0:00.00 inetd: wrapping (inetd)
    root   62950  0.0  0.1  3436   832  ??  I    12:45PM   0:00.00 inetd: wrapping (inetd)
    root   62970  0.0  0.1  3436   832  ??  I    11:57AM   0:00.00 inetd: wrapping (inetd)
    root   63077  0.0  0.1  3436   832  ??  I     9:29AM   0:00.00 inetd: wrapping (inetd)
    root   63112  0.0  0.1  3436   832  ??  I    12:54PM   0:00.00 inetd: wrapping (inetd)
    root   63148  0.0  0.1  3436   832  ??  I     9:29AM   0:00.00 inetd: wrapping (inetd)
    root   63165  0.0  0.1  3436   832  ??  I    12:58PM   0:00.00 inetd: wrapping (inetd)
    root   63180  0.0  0.1  3436   832  ??  I     2:11PM   0:00.00 inetd: wrapping (inetd)
    root   63183  0.0  0.1  3436   832  ??  I    12:09PM   0:00.00 inetd: wrapping (inetd)
    root   63208  0.0  0.1  3436   832  ??  I     9:43AM   0:00.00 inetd: wrapping (inetd)
    root   63214  0.0  0.1  3436   832  ??  I     9:19AM   0:00.00 inetd: wrapping (inetd)
    root   63222  0.0  0.1  3436   832  ??  I     1:19PM   0:00.00 inetd: wrapping (inetd)
    root   63228  0.0  0.1  3436   832  ??  I    12:26PM   0:00.00 inetd: wrapping (inetd)
    root   63232  0.0  0.1  3436   832  ??  I     9:20AM   0:00.00 inetd: wrapping (inetd)
    root   63267  0.0  0.1  3436   832  ??  I    12:03PM   0:00.00 inetd: wrapping (inetd)
    root   63290  0.0  0.1  3436   832  ??  I    11:03AM   0:00.00 inetd: wrapping (inetd)
    root   63344  0.0  0.1  3436   832  ??  I    12:59PM   0:00.00 inetd: wrapping (inetd)
    root   63357  0.0  0.1  3436   832  ??  I    12:21PM   0:00.00 inetd: wrapping (inetd)
    root   63444  0.0  0.1  3436   832  ??  I     2:20PM   0:00.00 inetd: wrapping (inetd)
    root   63459  0.0  0.1  3436   832  ??  I     1:07PM   0:00.00 inetd: wrapping (inetd)
    root   63544  0.0  0.1  3436   832  ??  I    12:13PM   0:00.00 inetd: wrapping (inetd)
    root   63574  0.0  0.1  3436   832  ??  I    11:22AM   0:00.00 inetd: wrapping (inetd)
    root   63579  0.0  0.1  3436   832  ??  I     9:56AM   0:00.00 inetd: wrapping (inetd)
    root   63587  0.0  0.1  3436   832  ??  I    11:12AM   0:00.00 inetd: wrapping (inetd)
    root   63635  0.0  0.1  3436   832  ??  I    12:02PM   0:00.00 inetd: wrapping (inetd)
    root   63658  0.0  0.1  3436   832  ??  I    12:13PM   0:00.00 inetd: wrapping (inetd)
    root   63734  0.0  0.1  3436   832  ??  I     1:45PM   0:00.00 inetd: wrapping (inetd)
    root   63744  0.0  0.1  3436   832  ??  I    11:01AM   0:00.00 inetd: wrapping (inetd)
    root   63746  0.0  0.0  3436   472  ??  I     8:35AM   0:00.00 inetd: wrapping (inetd)
    root   63760  0.0  0.1  3436   832  ??  I    11:14AM   0:00.00 inetd: wrapping (inetd)
    root   63787  0.0  0.1  3436   832  ??  I     9:31AM   0:00.00 inetd: wrapping (inetd)
    root   63822  0.0  0.1  3436   832  ??  I     2:28PM   0:00.00 inetd: wrapping (inetd)
    root   63849  0.0  0.1  3436   832  ??  I    12:11PM   0:00.00 inetd: wrapping (inetd)
    root   63930  0.0  0.1  3436   832  ??  I     1:45PM   0:00.00 inetd: wrapping (inetd)
    root   63952  0.0  0.1  3436   832  ??  I     1:22PM   0:00.00 inetd: wrapping (inetd)
    
    

    This is an wc of those processes, it is increasing with about one process every second/third second. And they never close, so it is eating all my ram somehow.

    
    [2.0.1-RELEASE][admin@firewall.home]/etc(64): ps aux | grep inetd |wc
        2017   26224  175500
    
    

    Anything i can do to prevent this, or should i just try to insert some kind of cron that restarts inetd once a day?
    I will also try to enforce TCP mode for mumble and see if that solves the problem.

    Mumble is a VOIP client like ventrilo btw, and during this increases there is 4 users online on the server and nothing more, no one disconnects, no one connects. It just increases all the time.

    Glad if someone could find some kind of work around or solution for this.



  • that's from reflection, disabling it will get rid of them.



  • @cmb:

    that's from reflection, disabling it will get rid of them.

    Yes i know it is from reflection, my questions was on how i could get them to stop just increasing all the time. It's not like the processess dissapeared after a while, they just stayed there for ever until the machine crashes. Shouldn't there be some kind of timeout, even tho there is no new connections it just increased anyhow.



  • I had the same problem with NAT reflection and UDP ports. After I disabled NAT reflection for those specific ports everything went back to normal.
    Wouldn't it be better if pfSense just never created reflection rules for UDP ports?


  • Banned

    How come this works fine in 1.2.3??



  • NAT reflection for UDP never worked in pfsense afaik (note: some time ago I offered some suggestion about replacing netcat with socat to solve this issue)



  • @Supermule:

    How come this works fine in 1.2.3??

    If reflection is working fine for you for UDP on that version, maybe there was some FreeBSD change that caused this.  I don't recall there being any changes to what gets written to inetd.conf.


  • Rebel Alliance Developer Netgate

    UDP NAT reflection didn't work in 1.2.x either.

    I did check in some changes recently to try to make it behave better but didn't get any more progress.

    Looked at socat the other day and need to look again, didn't look to be a drop-in replacement using our current methods.

    In the mean time you can edit your port forwards for UDP and manually choose to disable reflection for those rules. And if you use TCP/UDP port forwards, split them into a TCP rule and UDP rule and disable reflection just for the UDP port (or just use TCP if you really don't need UDP…)



  • @jimp:

    UDP NAT reflection didn't work in 1.2.x either.

    I did check in some changes recently to try to make it behave better but didn't get any more progress.

    Looked at socat the other day and need to look again, didn't look to be a drop-in replacement using our current methods.

    In the mean time you can edit your port forwards for UDP and manually choose to disable reflection for those rules. And if you use TCP/UDP port forwards, split them into a TCP rule and UDP rule and disable reflection just for the UDP port (or just use TCP if you really don't need UDP…)

    Jimp, don't you think it would be better to just disable nat reflection for UDP ports automatically at code level? At least for now until a solution is found…


  • Rebel Alliance Developer Netgate

    Well that ship has sailed for 2.0.x, which is why you have to do it manually in the rules.

    For 2.1 it's debatable. If someone can sort out the syntax for calling socat via inetd equivalent to what netcat is now, then it can be fixed up without too much trouble.


Log in to reply