Generating NAT rules from the command line

  • I have 500+ servers that will be dedicated HTCondor compute hosts.

    We can't put them all directly on the campus network, so I've placed them on a private network behind a pfSense firewall for now.

    The problem is, HTCondor requires that every compute host accept incoming connections from the scheduler.  It's not typical to have a larger number of compute hosts behind a single public IP like this.  Compute hosts are typically PC workstations on the public network, so the HTCondor project hasn't really addressed this situation.  There would also typically be concerns about the firewall becoming a bottleneck for file transfers, but we're not concerned about it in this case.

    The standard method for reaching a compute host behind a firewall is configuring each host to use a different static port number and creating a matching NAT rule for each host.

    So, my questions is, is there a good way to generate and install all these NAT rules from the command line?  I was initially excited to discover "easyrule", but it apparently only generates basic "allow" and "block" rules, not NAT.

    Entering 500+ rules via the web interface would be absurd, so I'd be happy even with a not-so-easy command-line alternative.  I'm considering generating XML rules from a template using here docs in a shell script and manually pasting them into config.xml, but I don't know what side effects that might have.



  • LAYER 8 Netgate

    As long as they go into config.xml you'll be ok.  That's what I would do.  Create one, then replicate it using perl or something.  Paste it into config.xml, reboot, and you're done.

    But I would take a copy of config.xml, make the change you want to make.  Get it working, then take a diff to be sure I was covering everything.  (Like NAT port forwards + firewall pass rules.)

  • I'd add them by hand - All 500…
    It will go faster than trying to figure out how to automate it.

  • LAYER 8 Netgate


  • Usually when people start screwing around with command line and they are not pfsense devs, bad things happen.
    I'm pretty sure in the time it took to ask the question and get a reply he could be half done or more.
    Actually….  He asked yesterday?  He would be done already.

  • LAYER 8 Netgate

    OP sounds like he has a clue.  At least enough to know the proper method is to modify config.xml.  A perl loop or two and some careful pasting is probably all he needs.

  • Yeah - I scipt abit and used to program abit.  I usually only bother scripting for tasks that would be repeated many times.
    However it would be nice to have something in pfsense that would allow you to port forward starting at IP192.168.1.x and port x and end on 192.168.1.y port y.

    That would probably get often used.

  • Guys,

    Thanks for the replies.  I do a lot of scripting and systems programming for a living, so scripting this would be far easier than doing it manually through the GUI.  The tedium of doing 500+ entries through the GUI would likely lead to mistakes as well.  I'd estimate < 10 minutes to write and test the script.  Just want to be sure that manually modifying config.xml is a reasonable approach before I do that.

    Thanks again,


  • Any time I'm setting up something like that, I write a script to populate and spit out the XML and just paste it into the config. You just have to be careful as that's bypassing the vast majority of the input validation, so if you get the XML wrong, it may break in ways that things configured in the GUI never can.

    One really simple example, a quick and dirty bit of PHP to spit out XML for 100 limiters.

    for ($i = 1; $i <= 100; $i++) {
    	echo "\t\t<queue>\n";
    	echo "\t\t\t<name>Limiter$i</name>\n";
    	echo "\t\t\t<number>$i</number>\n";
    	echo "\t\t\t<qlimit>\n";
    	echo "\t\t\t<plr>\n";
    	echo "\t\t\t<description>\n";
    	echo "\t\t\t<bandwidth>\n";
    	echo "\t\t\t\t\n";
    	echo "\t\t\t\t\t<bw>5</bw>\n";
    	echo "\t\t\t\t\t<burst>\n";
    	echo "\t\t\t\t\t<bwscale>Mb</bwscale>\n";
    	echo "\t\t\t\t\t<bwsched>none</bwsched>\n";
    	echo "\t\t\t\t</burst>\n";
    	echo "\t\t\t</bandwidth>\n";
    	echo "\t\t\t<enabled>on</enabled>\n";
    	echo "\t\t\t<buckets>\n";
    	echo "\t\t\t<mask>none</mask>\n";
    	echo "\t\t\t<maskbits>\n";
    	echo "\t\t\t<maskbitsv6>\n";
    	echo "\t\t\t<delay>0</delay>\n";
    	echo "\t\t</maskbitsv6></maskbits></buckets></description></plr></qlimit></queue>\n";

  • Sorry for the slow response.  I've been meaning to post a followup for a while…

    Because of the way pfSense stores rules in XML, scripting this would involve parsing out tags and generating matching NAT/PASS rules.  I concluded that the right way to to this is by enhancing easyrule to support generating NAT rules, which I don't think would be too difficult.

    It turned out to be unnecessary in my case, though.  I found out that HTcondor has a feature called "connection broker" that allows nodes to communicate with the scheduler from behind NAT just by switching it on and specifying the address of the scheduler.  It eliminates the need for the scheduler to connect to other nodes and instead routes all traffic through a connection initiated from the node to the scheduler (which I think is how it should have been done in the first place, but better late than never).  Hence, as long as the scheduler isn't behind NAT, there's no need for port forwarding.



Log in to reply