Does pfSense do what I want (hardware, routing, shaping all in one)

  • Hi,

    I've seen there are deticated forums but I have a bunch of everything here. Please feel free to move me whereever I might belong.

    I plan on using pfsense but I'm confused about hardware. I can get a decent server (2x4core Xeon with hyperthreading and 24GB of ram) off ebay for a next to nothing but I wonder if it even does what I want.

    TL;DR at the bottom.

    I’m a designer/creator/PR guy, not an IT guy. I know a lot, but not everything. So if what I wrote down below doesn’t make any sense please feel free to offer a better solution. Maybe I just don’t know enough.

    I want to run a total of 8 ethernet interfaces (2x intel quad server nic) and route between different vlans and prioritize vlans when it comes to accessing the internet via a dual wan interface.

    I’ll give you the current situation and what I want to do in the future.

    WAN now and in the future
    WAN1 is our current 100down/40up (Mbits) vdsl line.
    WAN2 will be a “lan client” type situation where we “borrow” internet from another company in the same building on different occasions.

    Current office network
    As far as internet goes we have a couple of clients (5) doing web browsing, owncloud (offsite, mainly text, pdf, etc), email and stuff like that.
    On the network side we currently have a windows server running an active directory, exchange, a fileserver, a surveillance server, a printer/scanner/copier, some more devices (label printer etc) and some remote desktop stuff for managing heat, ac and stuff like that. We’re currently still rocking 10/100 and earnestly, we’re fine. That part so far has nothing to do with pfSense, I know – it’ll come.
    I myself do a lot of audio/video stuff and store everything on usb drives. I’m now to a point where I’m afraid of losing data because of the number of projects scattered around different drives. So we’re getting a second NAS to store my media stuff on it with proper raid and everything.
    Also we have a couple of event rooms of different sizes we rent out on a day to day basis.
    We’re running Ubiquiti Unifi AC-Pros (about 10) and we’re completely fine with those (and we can’t affort anything else anyways). We don’t manage that network at all, everything’s default, one SSID, one shared key, everyone gets the key when they rent the space. They basically use our internet connection as well. Still – everything’s fine, lots of clients, yes, but mostly phones checking email, nothing special. Keep in mind everything’s 10/100 – still fine. In the future I want to run a dedicated unifi controller to do all the cool stuff it can do.
    Now once a year we have an event we organize ourselves. It’s an it conference, we use every room we have, we do four concurrent livestreams, we have speakers doing live hacking, intense RDP sessions and what not and the wifi clients actually use their phones (mostly for twitter, emails and IM but still…) and that’s exactly where we’re not fine anymore. This event kinda grew by itself but our network didn’t grow with it.

    My network plans for the future:
    Our big event is the main reason for upgrading the network. But it’s also something that doesn’t really have much to do with our business and doesn’t give us any financial return. In fact, we’re losing money. We don’t have money for contractors or “professional grade” hardware so we’re trying to go open source and used as much as possible. Everything will be self-serviced.
    I want to open a couple of vlans to give me control over who does what and who has access to what. Basicly I want:

    VLAN 1 (not actually default vlan but just the first out of them)
    Our internal network as described above. We’ll get a new gigabit managed switch (more on that later) and all clients are gigabit anyways. We’ll get a used server and run as NAS, internal webserver (for some services) and a proper unifi controller (again more later) on it. Also we’ll get a dedicated render machine for the video projects which auto-pushes to youtube (we record our own events and upload them to youtube, this will greatly reduce workload for me).

    Our old outdated firewall and our consumer-grade “wireless router” will go, a pfsense firewall on a used (maybe overkill dual xeon server), a proper vdsl modem and another unifi AP AC PRO for the office will come.
    We’ll run a DHCP/24 on pfsense and use a dedicated Ethernet port on the machine for this network.
    I want to prioritize this VLAN/Network to have the second highest priority when it comes to accessing the internet on WAN1.

    VLAN 2 (streaming)
    This is the network where our livestreaming pcs are gonna connect (we’re streaming UP, to youtube, not DOWN, people in other forums got that wrong). Our event space has its own room for patch panels and switches and it’s connected to the “main office” via a multimode fiber line. We’ll probably have a gigabit connection over fiber since we can’t afford most switches with sfp+ and also most of what will go over that connection is internet anyways and that’s far away from reaching gigabit speeds for us (we’re pretty remote, 100-200Mbit is the absolute max we can get for at least another 5-10 years).
    This network will also be connected to a hardware port on the pfsense machine and it will be a /24 network with a little bit of dhcp (200 and up) so that I can plug a client directly into it without setting anything up on the client side.
    I want to prioritize this network over everything else on WAN1 to a point where it can take the full bandwith and basicly “lock out” everything else, including the office. A lot of people watch our livestreams and this is top priority for us. And when doing events, we’re not at the office anyways.

    I want to setup a route so that this network is able to access the Media NAS to store its recordings onto it. Since I have two individual ethernet interfaces for these networks, I wouldn’t bottleneck anything, right? Will a continuous transfer of a huge files for an hour or so impact my “VLAN1” to a point where clients would notice? I assume the answer is yes and I also assume I would be way better off with a layer 3 switch for this since I would be using the way higher internal switching capabilities of that switch instead of the gigabit/gigabit capabilities of the pfsense firewall right?
    (Keep in mind that this is a scenario what might happen 5 times a year at max and can be done over weekends)
    But on the other hand would a layer 3 switch just route 2 vlans together or would I actually be able to for example “allow ALL OF vlan 2 to access ONE IP in Vlan 1” because I know I need to setup rules for that in pfsense. Does a layer 3 switch also have such “rules” (I know a “router” is nothing but a layer 3 switch but I also know they come with different amounts of features – is this something every layer 3 switch will do?)

    VLAN 3 (management)
    This is gonna be the vlan that I want to manage everything in (switches, APs, unifi controller, Hypervisor, pfsense [again, separate interface]. Again I want to be able to access what network from two machines in VLAN1 (my bosses and my machine). /24, no dhcp.

    VLAN4 (visitors)
    I want to pass this vlan through to the unifi APs in the event spot. I want a /16 dhcp running on pfsense and this network to have a lower priority than all others when it comes to internet access. However, on huge events I want to be able to switch this network over to WAN2. The vlan needs to be able to access the controller in VLAN3 in order to do stuff like vouchers and terms of service (as I plan to professionalize our wifi solution and actually take money for offering wifi).  I want to limit what people can access to standard web services like http and email. A lot of things (individual speed limits, timeouts, wall gardening, captive portal) can be done using the unifi controller and I purposely WANT unifi to do this since there might be occasions where let’s say an event requires the clients to see each other (a computer class maybe, we had that) and I’m not inhouse. With Unifi I’m able to walk people through stuff like this on the phone, with other stuff I might not be, although it might have more functionality. I’m willing to trade that, though. We’re talking 300 clients maximum here and I know our vdsl is underpowered for that. That’s why we borrow that line from the other company (I don’t know specifics but they have over 50 employees doing constant online intensive work). And if it’s too slow still, it’s too slow. We tried, we’re sorry.

    VLAN5 (speakers)
    This vlan will also be passed to the APs. It’s gonna be a /24 network with dhcp. It will also need access to the controller and the internal webserver (more on that later). This network should be prioritized above VLAN4 and under VLAN1, also being able to be switched over to WAN2 in certain cases.

    TL;DR?  This keeps me awake at night:
    I’m reading a lot about the traffic shaper/limiter and how it basically kills little embedded style machines. I wanna reliably limit bandwidth for at least 4 networks on 2 wans. How much horsepower is really needed for this? I can go as high as a dual Xeon quadcore with ht and 24gb of ram, but do I need to?

    Also the amount of layer 3 routing we plan on doing is not ideal I guess? So should I go with a layer 3 switch for routing and leave dhcp and traffic shaping to pfsense?
    Since Unifi has been a success for us and the controller is nicely laid out and easy to manage for non tech people (read above) I’d like to go with Unifi switches since they can be managed through the controller. However those are layer2 switches.
    Ubiquitis Edge series switches offer the same POE functionality (for the APs) and they are layer 3 (and we can actually afford them, keep that in mind please). But they come with a rather complicated web interface which for me is not a problem but for others it might be (again, read above).

    So yeah… Your 2 cents?

    ps. my native languages autocorrect might have screwed something up. I'm sorry in advance ;)

  • VLAN4 (visitors)
    I want to pass this vlan through to the unifi APs in the event spot. I want a /16 dhcp running on pfsense and this network to have a

    so you are expecting 65k clients and wish them to have good interwebs on a 100mbit connection ? this is a no-go

  • I doubt there'll be 65k users in that vlan, though… He's just using a /16, nothing wrong with that. I'm using a /8 at home(

    I think yes, it will do what you want, but traffic shaping is always difficult unless you really know what you're doing.

    First lesson in QoS: to fix your issue, get more bandwith...

  • @heper:

    so you are expecting 65k clients and wish them to have good interwebs on a 100mbit connection ? this is a no-go

    No, but I expect to run out of IP's fast due to dhcp lease time (I'll set it to 24h but I don't wanna take chances). Thanks for reading it all btw  :)

    I doubt there'll be 65k users in that vlan, though… He's just using a /16, nothing wrong with that. I'm using a /8 at home(

    I think yes, it will do what you want, but traffic shaping is always difficult unless you really know what you're doing.

    First lesson in QoS: to fix your issue, get more bandwith...

    Well I can't really get more bandwidth - except getting a third WAN which at this point is not an option.

    So really, what type of hardware should I get? Gimme a ballpark please.

  • @SoulChild:

    He's just using a /16, nothing wrong with that. I'm using a /8 at home(

  • Well, let's do the easy part first:

    Routers vs layer 3 switches
    Routers and layer 3 switches do the same basic function: route IP packets. Routers usually have tons of ancillary functionality like routing protocols, DHCP servers, GRE/L2TP/IPIP tunnels, ipsec, etc. (although layer 3 switches can offer those things). Routers also usually offer interfaces other than ethernet (T1, coax DS3, etc). It doesn't sound like you need a router.

    Layer 3 switch vs pfsense for inter-VLAN routing
    No brainer - you absolutely don't want to use pfsense to do your inter-VLAN routing. Use a layer 3 switch for this, unless you have complex access control lists, or your L3 switch can't do access control lists.

    Traffic shaping
    You should be able to accomplish some traffic shaping in pfsense, but perhaps not quite to the level of detail you are asking for.

    Multiple WAN providers
    I think this will be you biggest hurdle. To be honest, I've never looked at multi-WAN configuration of pfsense, but I suspect it may be difficult to do what you want to do. I suspect you may need to use two pfsense instances and two layer 3 switches (one set for your VLAN4 users, the other set for everything else) in order to flip VLAN4 users over to the other WAN. In the Cisco world, you'd use policy based routing for this, but I don't know if pfsense has policy based routing.

  • @heper:


    He's just using a /16, nothing wrong with that. I'm using a /8 at home(

    What's the point of subnetting a /8 if I don't even need more than 1 subnet and only have a grand total of 20 IP devices at home, including cellphones, tablets and a NAS? I don't do DMZ or hosting or anything.