Any successful (production quality) deployments of pfSense with 3G networks?



  • We are considering using pfSense in a large-scale production roll-out whereby the remote sites are only accessible via an attached 3G cell phone.  Currently, a PC is installed at the remote site (isolate/desolate  - no humans within 2hrs in any direction) that performs both the network function and other stuff.  The goal is to move the network related stuff (VPN, etc) off the PC and onto a secured security device.  the client is currently considering the CradlePoint MBR1200 http://cradlepoint.com/products/mbr1200-failsafe-gigabit-n-router-mobile-broadband#

    Before I pitch pfSense as an alternative to the CradlePoint box ($299), I wanted to get a feeling from others who have actually deployed pfSense with 3G networks.  Specifically, can you tell me which 3G cards work the best?  We are looking at the small/embedded version of pfSense to run on something like an ATOM CPU.

    Thanks…



  • …why was this moved to the 2.0-BETA sub-forum?  Does this not work in 1.2.3-RELEASE?


  • Rebel Alliance Developer Netgate

    See the note I stuck on the 'moved' post:

    http://forum.pfsense.org/index.php/topic,22432.msg115267.html#msg115267

    (3G is only supported on 2.0, so this thread is more appropriate there)



  • AH - ok, sorry!  I did not see the "fine print" at the bottom   ;)

    Given this is only a 2.0 feature, is it reasonable to assume no one has 3G in production?


  • Rebel Alliance Developer Netgate

    Well, 2.0 isn't really fit for production, but that doesn't stop some people :)

    There might be someone out there who has it running "good enough" that they use it.

    Since 3G isn't on 1.2.x there could be an argument made for running it that way in production as long as you don't need any of the extra bits that don't work quite right



  • Hi,
    Judging from forum posts I've probably done as much 3G/EDGE/GPRS testing as anyone out there. Search this forum for "GPRS" and you'll see a bunch of posts from me.

    I'm running two networks running off 3G modems. Both are home/office size networks with 5 to 10 local hosts. One gets a 3G signal and the other a 2.5G signal. Both are running on ALIX 2D3 embedded boards with the 2Gb Nano flash image. I'm on the 2.5G modem right now. The 2.5G modem/router is powered from a single 40W solar panel with battery backup in a location where grid power is not available.

    I've used USB 3G modems from Sierra Wireless (Compass 885) and Novatel(MC950D).

    Everything in pfsense "works" with respect to 3G, but here are the cons.
    1. If your signal strength is shaky, the modem may not connect to the tower on the first, second, third, or Nth attempt to connect. This may not be a problem.
    2. Occasionally, the ppp program that FreeBSD/pfSense uses just hangs. Because of the way it's written, it doesn't fork into a daemon when you run it with the "-background" command line switch (default in pfSense) until AFTER the connection comes up or fails or times out. But, if the connection doesn't come up AND the program doesn't time out properly, the ppp program will just hang, and it will hang the web interface too. Then, you must reboot, or connect via the serial line and manually kill the ppp processes.
    3. Occasionally, my 3G provider will drop the connection silently on their end. The pfSense box believes the connection is still up. Currently, there is no code to my knowledge, that will stop and restart the connection if this happens. I'm thinking about working on this problem.
    4. pfSense is using (I think) mpd4 for its PPPoE connections. The mpd program can also do PPP and has this cool feature that if you dial the phone number of the attached modem from another phone, the modem won't answer, but it will immediately dial the ISP once the line is free. Cool huh? Call your modem and make it connect to the ISP. I've tried to float the idea of using mpd5 for the PPP/3G connections, but it didn't get picked up by the devs. I also manually tested mpd4 with my 3G modem, and it failed to connect due to some problem with the mpd4 code. mdp5 exists, but isn't included in pfSense, and from my testing a couple months ago, won't even compile cleanly in FreeBSD 8.0.

    I've contributed a little code to the PPP implementation, but it needs a little more . . .

    Hope this helps.

    Gabriel



  • Also,
    I'd stay away from the ATOM especially if you're going to use a PC ATOM board with a GM945 or Nvidia ION graphics chip. It takes way more power than necessary. The ALIX embedded board can push enough data to saturate any 3G connection, and you get to spend less on solar panels and batteries. ATOM boards with the embedded chipset instead of the '945 series draw a lot less power.

    I have an ATOM motherboard with an Nvidia ION chip and I measured the power draw after the AC/DC converter at around 18W. The ALIX draws 5-6W. I'm guessing/betting the Cradlepoint box draws at least 12W.

    Cheers.
    G


  • Rebel Alliance Developer Netgate

    I heard today's snap performed better with 3G thanks to it being built against 8-STABLE, and there is improved usb and tty code that came in after 8.0-RELEASE. Not sure if any measurements have been taken to compare though.



  • Many thanks to all who replied - especially gnhb for his detailed information.

    More background:
    –------------------
    A buddy of mine has a company whereby he installs remote sensors for some industrial equipment out in the field.  His challenge is getting the sensor data back-hauled to a central monitoring facility.  These locations are very remote, and in many cases, the only reliable connection is via Verizon 3G cards.  When he asked me for options, I immediately thought of pfSense not only because it is a firewall (ipSec, PPTP, port-forwarding, etc) but also because it can be run from very small embedded systems like the ALIX or ATOM-based boards (low power, etc).  Since pfSense has multiwan capability, it could use any existing wired network connection as the primary link and 3G for the standby link.  Given pfSense has the ability to install other apps and utilities, his monitoring app could easily be ported over to FreeBSD.

    I am wondering if the issues noted by gnhb could easily be solved using monit and some intelligence checking.  For example, we could have monit monitor the outbound network via ping or http and either reboot the box or reset the 3G card automatically.  I think this would be a perfect marriage between his app, pfSense, and monit.

    BTW - this project is expected to draw millions (yes, millions) of dollars if deployed in the field.  So, this is more than just a "wouldn't it be cool if" kind of project.  This is the real-deal.

    Please let me know your thoughts...



  • Commercial support can help you with these items.  Please open a ticket.

    Thank you.



  • Will do…



  • I had 1.2.3-beta (don't remember which version) running about a year ago with a Millenicom/Verizon EV-DO USB key running into the little 3G router they sell (which I don't recommend–flaky firmware).  I had this set up to load-balance between that and a partial T-1 we had coming in (the rest went to actual DS0 phone lines).

    There was a problem with the load balancer losing packets while testing whether the EV-DO connection was up and so it would flip up and down, or just stay down quite often, especially when Verizon's network was under load during rush hour (we're right next to a major freeway).  I BELIEVE this has been fixed by the time 1.2.3 went into final release, but I'm not positive and haven't tested it--we dumped both and went to a WiMax connection from a local provider that, once finally set up, has been very reliable. (And why not DSL or something?  Qwest/U.S. Worst wouldn't give it to us, saying we couldn't have that and the T-1 running.  We're at the very limit of the loop, several miles from the CO and NO DSLAMs in between!)

    I guess one problem is, do you really need the hard-core filtering a full pfSense setup provides or do you just need the connectivity?  Considering that, by your words, this is in the middle of nowhere, perhaps security isn't important enough to really worry about compared to, say, keeping power consumption low and having a reliable box with firmware that doesn't crash--this last bit being the most important since there won't be anyone around to reset the box!  Price tags mean nothing in that department unless you go carrier-grade (expensive Cisco stuff, say).

    Mike


Locked