Converting from a Single Interface to a LAGG

guardian · Jul 5, 2018, 10:46 PM

I have a pfSense setup with many VLANs and firewall rules. I would like to convert my connection from pfSense to the switch (TRUNK) to a LAGG without having to recreate my configuration which is quite complicated.

The current set up uses em1 as a TRUNK connecting pfSense to the core switch so pfSense acts as a router on a stick for inter-VLAN routing and also as a gateway to the Internet. I have upgraded my Internet connection to 500Mbits/s, so the the single interface is likely to be a significant bottleneck. I was thinking that converting it to a LAGG with em1 and em2 (which is currently unused) would help the situation.

Is my thinking sound about using a LAGG?

Can I swap em1 for LAGG0 with the following procedure?

Procedure:

Create a LAGG with em2 in FAILOVER mode on pfSense
Create a single port LAGG Trunk on the switch with all VLANs
Connect the LAGG with a cable from em2 to the Switch
Navigate to pfSense - Interfaces/Interface Assignments
Change LAN, and all VLAN interfaces to LAGG
Click Save
Disconnect the cable on em1 and the original TRUNK on the Switch
Navigate to pfSense - Interfaces/LAGGs - Select Edit for the LAGG
Select em1 in Parent Interfaces to add em1 to the LAGG and switch to LACP
(Since it is no longer assigned, I'm assuming em1 should be available for selection)
Convert the TRUNKPORT on the switch to a member of the LAGG and enable LACP

Does this procedure look sound? (The switch is a Cisco SG300)

Any advice/suggestions/hints would be much appreciated.

johnpoz · Jul 6, 2018, 2:20 AM

How many vlans do you have? Do they all talk to each other.. You do understand lag doesn't mean that traffic will still not hairpin. Only way to make sure your not hairpin is to put the different vlans on different interfaces for uplink.

guardian · Jul 6, 2018, 3:44 AM

Thanks for the quick response @johnpoz - I just want to make sure there are not some serious gaps in my understanding.

@johnpoz said in Converting from a Single Interface to a LAGG:

How many vlans do you have? Do they all talk to each other..

I've got about 2 dozen VLANs. A lot of them just call out to the Internet through some very restrictive firewall rules, but there is some interaction which is moderated by firewall rules on the various interfaces.

You do understand lag doesn't mean that traffic will still not hairpin.

I haven't heard this expression before so I'm not sure what you are referring to. I'm not a network engineer by profession.

Only way to make sure your not hairpin is to put the different vlans on different interfaces for uplink.

I think I've done that. The system is operating but with a single cable between the core switch and pfSense. On the Interfaces / Interface Assignments tab I currently have a series of interfaces that look something like this:

LAN                      em1
VLANx                    VLANx on em1 - VLAN
VLANy                    VLANy on em1 - VLAN
etc.

and each of these assignments gives rise to a firewall rules tab where I define inter-VLAN rules.

Is my thinking correct that if I follow the procedure above I can systematically swap LAGG0 for em1 and then configure em1 to be part of LAGG0?

Or am I missing something?

johnpoz · Jul 7, 2018, 3:27 AM

You talk about intervlan traffic - and your 1 connection being a bottleneck.. So how is it you don't know what a hairpin is?

So box on vlan 1 wants to talk to box on vlan 2.. These are using the same physical interface em1, so traffic between vlan 1 and vlan 2 have to travel over the wire connecting em1 to your switch twice - its a hairpin.. In out the same interface.

Yes this can be a bottleneck, but has zero to do with what your internet speed is.. Your internet is 500mbps - have to assume em1 is gig. All your boxes talking to internet is not going to be a bottle neck.. Only when you want to go over 500mbps intervlan and 500mbps to the internet will your em1 be a problem.

Your problem is that even if you created a 4 port LAGG. And vlan 1 wants to talk to vlan 2.. You can not be sure that you will not hairpin over em1 still. The only way to make sure that you do not see hairpin traffic is if vlan 1 is on em1 and vlan 2 is on em2, etc..

guardian · Jul 7, 2018, 3:27 AM

@johnpoz said in Converting from a Single Interface to a LAGG:

You talk about intervlan traffic - and your 1 connection being a bottleneck.. So how is it you don't know what a hairpin is?
So box on vlan 1 wants to talk to box on vlan 2.. These are using the same physical interface em1, so traffic between vlan 1 and vlan 2 have to travel over the wire connecting em1 to your switch twice - its a hairpin.. In out the same interface.

From the description, I understood what was going on (the hairpin), but I have never heard that term before. I'm not a network engineer--self taught for home network.

Yes this can be a bottleneck, but has zero to do with what your internet speed is.. Your internet is 500mbps - have to assume em1 is gig. All your boxes talking to internet is not going to be a bottle neck.. Only when you want to go over 500mbps intervlan and 500mbps to the internet will your em1 be a problem.

This was the case I was considering, A large/fast download on VLAN3 slowing down traffic between 1 and 2.

Your problem is that even if you created a 4 port LAGG. And vlan 1 wants to talk to vlan 2.. You can not be sure that you will not hairpin over em1 still.

can not be sure
Are you saying that 3 or the 4 ports could be busy when the session starts, so the traffic has no choice but to hiarpin?

Are there times when the traffic will not hairpin, or is it always going to do so?

Does a tcp session always use the same port once it begins, or does the LAGG swich between ports depending on load?
(I'm assuming that if I do this I should use Cisco LACP?)

The only way to make sure that you do not see hairpin traffic is if vlan 1 is on em1 and vlan 2 is on em2, etc..

It's a small home network, and traffic is very bursty... sits fairly idle till I get some large file copies and downloads going on at the same time.

I'm assuming that all traffic on the same VLAN/sub-net wouldn't see pfSense as the switch would handle it.

The reason for all the VLANs is for control/security as I don't have much trust of most consumer hardware/software, so I've tried to compartmentalize as much as possible to allow very tight control and monitoring, and if I do get some sort of infection make pivoting harder.

So I've got:

IoT, on it's own isolated vlan with no direct internet access,
voip on it's own vlan with tight fw rules that only allow it to talk with the service providers
printers on their own isolated vlan with no internet access
Remote Access VLANs secured by OpenVPN with User and TLS keys
separate VLANs for Media players, Guests, and using the cell phones at home on WiFi (I treat cell phones as potentially hostile devices-they get network access or local access, but not both simultaneously, and local access is limited. Given that data is metered on the cell network any significant volume of ex-filtration would show up pretty quickly-still could steal creds from the phone, but there's not much I can do about it).
A Management VLAN
None of this is going to give rise to any substantial volume/speed traffic.

Where the majority of the traffic comes from is 2 VLANs with different firewall rules for the PCs (wife uses Windows and conference software like Skype/GotoMeeting etc.) and she isn't super PC savy, so I keep her as isolated from me as is practical -- and then there is the FreeNAS server for media and backup (there might be some heavy traffic between these)

From what you are telling me, it seems that I might be better to break my trunk from the switch into two separate trunks to service traffic flows from these potentially high traffic VLANs.

Comments/Suggestions?

johnpoz · Jul 7, 2018, 10:09 AM

@guardian said in Converting from a Single Interface to a LAGG:

Does a tcp session always use the same port once it begins, or does the LAGG swich between ports depending on load?

Depends on what your using.. LACP, FEC, Failover, etc. What switch are you connecting the lagg too - and what sorts of setups does it support? It s more complicated than oh this tcp session what connection to use.

From what you are telling me, it seems that I might be better to break my trunk from the switch into two separate trunks to service traffic flows from these potentially high traffic VLANs.

There you GO!!! It clicked! ;)

I have lots of vlans myself for control... I run some of the vlans shared over 1 physical interface, but the others that have higher traffic are on their own uplinks. So when vlan 1 talks to vlan 2 - it will for sure use 1 physical paths and not hairpin traffic. Lighter traffic like the wireless is not a big deal, there is really never any intervlan traffic between them anyway. While a device on a wireless vlan will talk to vlan nas is on streaming a movie, music etc. Those are 2 different physical interfaces involved. While for example me guest wifi vlan and iot vlan never going to talk to each other anyway.

Derelict · Jul 9, 2018, 8:33 AM

I would not create the failover LAGG first then change to LACP.

I would create a two-port LACP LAGG on the switch and a single-port LACP LAGG on pfSense with em2 as the only member. Patch em2 to the switch and make sure the LACP LAGG comes up with just the one link. Maybe make a temporary VLAN between the two and be sure they're passing traffic. This should all be pretty much out-of-band with everything else.

Then add all of the VLANs to the LACP on both the switch and pfSense sides.

!!! warning: Probable traffic disruption follows.

In Interfaces > Assignments, switch the pfSense interfaces to the LACP VLANs.

When they are all moved to the LACP, delete the VLANs from em1 then you should then be able to add em1 to the LACP LAGG and patch it over to the second LACP port on the switch.

I would set a maintenance window for this if necessary in your environment. I wouldn't expect a great deal of downtime for this but it will probably be somewhat disruptive.

guardian · Jul 11, 2018, 6:45 AM

Thanks @johnpoz @derelict thanks for the info. I guess I know how to set this up, but from what @johnpoz has said, it seems as if this might not be worth the effort. I would might be much better off to reconsider network design.

I have a FreeNAS which has 3 VLANs connected to the switch with a 2 port LAGG. From this discussion I'm now wondering if this makes any sense. My thinking was that this traffic should stay on the switch and bypass pfSense since the source and destination should be on a VLAN that is on the LAGG trunk. Any easy way to check if my assumption is valid?

Any comments/hints/suggestions are most appreciated.

Derelict · Jul 11, 2018, 7:04 AM

NAS/Storage traffic should pretty much not go through the firewall at all especially if it's heavy. It should be on the same subnet as the consumers if possible.

But if it does have to route, LACP would probably be better. Especially if you have multiple clients so you can actually take advantage of the load balancing hashing.

I have LACP lagg from FreeNAS to my switch, but it is only used for (same-subnet) traffic to my hypervisor for multipath iSCSI shared storage. There is another interface to the NAS for "general" storage traffic.

I also have LACP from my firewall to my switch and from my switch to my desk switch. It has come in handy a couple of times when rerouting cables. I really don't need it. I do it just because.

I also had a mini GBIC go bad and didn't know it for a couple of days. One is none. Two is one.

johnpoz · Jul 11, 2018, 8:55 PM

@guardian said in Converting from a Single Interface to a LAGG:

Any comments/hints/suggestions are most appreciated.

Understand the tech your wanting use - make sure it works the way you think it does. And understand your network flow to how to best optimize it while allowing for the filtering you need/want through your firewall.

I see it all the time with hairpins and bad uplinking.. in line with Derelict's Two is one, lets also remember that 1 + 1 does not equal 2, it is just 1 and 1 ;)

You do not instantly get 2 gig when you lagg/etherchannel/portchannel/lacp whatever term you want to use two gig connections. Just means you have 2 1 gig connections.

guardian · Jul 12, 2018, 1:14 AM

@johnpoz said in Converting from a Single Interface to a LAGG:

Understand the tech your wanting use - make sure it works the way you think it does.

This is much easier said than done because of the way this stuff is taught. I've cherry picked though a few of online CCNA/Networking courses and they concentrate far too much on how to set something up as opposed to "why" set it up a particular way and don't teach enough first principles.

The courses tend to have way too much material that is too basic (this is binary, this is a subnet), and then often very quickly go way too complicated (multi-building with several layers of distribution), and concentrate on details on the negotiation (active/passive-which modes are compatible etc.) and the command line commands to setup LACP in all the different modes.

Details about how the traffic gets divided, when hairpinning would be an issue, and a lot of the "why" set things up a given way is skipped over.

[Possible suggestion for whoever creates the content for the Gold Membership:
----BEGIN SUGGESTION
Please consider this a feedback/a suggestion for possible improvement, not a complaint-I appreciate your subscribers could be mostly network engineers, if so my comments may not apply.

There is a real need for some good tutorials why/how/gotcha's to watch out for on how to set up a system following best practices for the advanced home user/small business user (Think startup/small development shop with a couple of programmers who have a decent understanding of network basics, but no IT department/network engineer -- They need a good/secure network, but can't pay for support).

Case studies on how to set up a network with VOIP, Cameras, Media Players/TVs, PCs, OpenVPN LAN gateway (some traffic through the LAN, some thought the VPN), OpenVPN remote access server(s). There are setups for some of these individually, but often they don't explain enough to make it easy to combine the elements. It's often much easier to figure out how to remove an element from a more complex design than add one in - especially if it's documented.

Planning/setup/workflow when combining elements and applying good security practices and getting routing correct is very hard to figure out from scratch, but once you have a basic blueprint (and enough understanding of why it's done that way), it's not that bad. A visual model of the interaction between Interface Rules, Floating Rules, Policy routing/tagging and NAT would have saved me days of struggle (i'm still a bit unclear about some of it.)

Setting up remote access to view security cameras and to be able to access files (I assume this requires two separate remote access servers / firewall rules to do safely.) is very difficult to figure out based on what's out there online.

For my level of experience/needs I'm sad to say many time that Tom Lawrence's YouTube videos were more helpful than the Gold content I paid for. A deep dive/case study approach on 2, 3, 4 (whatever is needed to cover the topics) would be far more useful than the current book for many like myself. Case studies allow the right advanced topics to be covered and in sufficient detail to get the job done properly without having to become a CCIE! I don't know if the community thinks this is a good idea, I just know what would have saved me weeks.
----------------END SUGGESTION]

I see it all the time with hairpins and bad uplinking.. in line with Derelict's Two is one, lets also remember that 1 + 1 does not equal 2, it is just 1 and 1 ;)

You do not instantly get 2 gig when you lagg/etherchannel/portchannel/lacp whatever term you want to use two gig connections. Just means you have 2 1 gig connections.

It all comes back to education and experience (don't know if you learned this from experience or formal training.)

This thread caused me to do a bit of research/review and for the benefit of others I thought I would put this out for the benefit of others/comments.

----REVISED----

I just came across these so I have edited this post to correct/improve it:

http://ieee802.org/3/hssg/public/apr07/frazier_01_0407.pdf

http://www.ieee802.org/1/files/public/docs2017/ax-rev-haddock-conversation-sensitive-collection-distribution-0517-v1.pdf

A LAGG bonds 2 (or more) links together into single interface with common ingress and egress queues so they act as one. Data goes into the ingress queue, then some sort of hash based on the Layer 2 header which determines which link get used, and then the packet comes out the egress queue on the other end.

What fields the hash is derived from is dependent on the vendor/age/quality of the hardware.

The important points are:

As @johnpoz has suggested LAGG does not increase the bandwidth for a single conversation and in this case the only value is graceful failover in the event of a link failure.
LAGG achieves high utilization only when carrying multiple simultaneous conversations since all packets associated with a given “conversation” are transmitted on the same link to prevent mis-ordering
The load sharing is generally based on a hash derived from the Layer 2 header and the algorithm will vary with manufacturer/equipment. As a result it is difficult or impossible for a network administrator to predict or control which frames are transmitted on which links. The resulting load‐sharing between the links in the LAG may be very imbalanced.

Anyone have anything to add?

Derelict · Jul 11, 2018, 9:22 PM

You are getting into the realm of "real" network design. If you want to do that you should hire someone who knows how to do it or know/learn how to do it yourself.

No documentation is going to acceptably cover every use case and the people who don't know how will need a step-by-step walkthrough for their specific circumstances and will likely be unable to adapt the docs to their set of criteria.

Those who know how to google can get all of the information from many sources because at the end of the day, we are just talking about IEEE 802.3ad.

All you do is create a LAGG, add the member interfaces, and assign it. That is all covered in the pfSense book. You asked for a specific case of migrating an existing connection to a lagg. You received, what I feel, is valid feedback for that scenario.

Whether or not to use a lagg is not a "pfSense" question. Entire books are already written on all of this. Industry certifications and high $ courses are available, etc.

guardian · Jul 12, 2018, 1:26 AM

@derelict said in Converting from a Single Interface to a LAGG:

You are getting into the realm of "real" network design. If you want to do that you should hire someone who knows how to do it or know/learn how to do it yourself.

I agree, and the point I am trying to make is there is a huge gap in the knowledge/training sources available out there. I want to learn enough to do the job - I have no interest in becoming a CCNA as I'm not working in the industry. The person who want a good/secure network for a home/small shop with a handful of PCs, a few phones, a few cameras, small file server and smart TV or two that needs to be kept under control has no need for 90-95% of the stuff that is taught in professional courses.

No documentation is going to acceptably cover every use case and the people who don't know how will need a step-by-step walkthrough for their specific circumstances and will likely be unable to adapt the docs to their set of criteria.

I agree that no documentation will cover every use case, but I disagree with your comment about adaptation to particular circumstances. Anyone who is going to set up pfSense has got to know networking basics

I would also suspect that someone who works in the field/knows it well would have a pretty good idea of a handful of reference designs that would fit 80-90% of the use cases with a bit of adaptation. Using Google for the last 10-20% is a whole lot more manageable than using Google for 80-90%!

I also suspect that a small collection of well documented/commented reference designs would improve pfSense Gold membership. Over the years I've dealt with a number of programming reference manuals - the ones that include a short but sensible/meaningful example of the command being describe are infinitely more valuable/useful than than line after line of text that didn't describe as well as a few lines of code. (I'm not suggesting getting rid of the reference manual - but in MY PERSONAL EXPERIENCE I've found the the GUI is so well done that it is almost self documenting. For the issues I had, the reference manual didn't add very much. If I couldn't figure it out from the GUI, the manual didn't get me there-I needed google for basic information (if I as missing a basic network concept) or I needed to ask a question in the forum (if it was more of a pfSense specific issue).

Those who know how to google can get all of the information from many sources because at the end of the day, we are just talking about IEEE 802.3ad.

Actually hasn't it changed to 802.1AX? 802.3ad got me close enough-Thanks for that comment, that gave me an idea of what to google to fill in a couple of missing bits so I'll edit my post above.

Let's face it if someone isn't willing to google for answers then clearly pfSense or any open source project isn't for them. Often when you don't know, you also don't know WHAT to google and that's where the fourm can help a lot.

All you do is create a LAGG, add the member interfaces, and assign it. That is all covered in the pfSense book. You asked for a specific case of migrating an existing connection to a lagg. You received, what I feel, is valid feedback for that scenario.

You are correct, I feel I received excellent feedback that was exceptionally helpful. If I was starting from scratch, I wouldn't have even asked the question. I found that the GUI was all I would need.

I am VERY grateful that @johnpoz injected the question why/are you sure you want to do that? I think based on his comment I'm going to leave things alone until I get time to go over my design in detail. I put LAGG on my FreeNAS, and it didn't seem to matter (and now I know why).

Whether or not to use a lagg is not a "pfSense" question. Entire books are already written on all of this. Industry certifications and high $ courses are available, etc.

I'm wandering a bit and my feedback/comments were more general comments that likely should be in a different sub-forum, but they would lose of of their significance if removed from the context of this discussion. I put the comment out there, since I see it as a need in the marketplace that Netgate might be able to take advantage of to increase their Gold adoption.

You are quite right Whether or not to use a lagg is not a "pfSense" question, and in the context of a good reference design for a small network it likely would not come up at all.

Again, thank for your input--it was the catalyst for filling the the missing pieces and correcting my earlier post.

johnpoz · Jul 12, 2018, 10:53 AM

@guardian said in Converting from a Single Interface to a LAGG:

Anyone who is going to set up pfSense has got to know networking basics

You would think huh ;) Clearly not the case...There are people that try and setup pfsense that don't understand why they can not have 192.168.1/24 on both their wan and lan networks, etc.

Another scenario that comes up ALL the time - is why can pfsense not filter traffic between devices on the lan.. So the concept of what a router/firewall is doesn't seem to click to why you can not filter traffic between 192.168.1.100 and .101 on the gateway device..

So basic network understanding is clearly not a requirement for trying to setup pfsense ;) The one that gets me the most is not understanding the basic difference between a resolver and a forwarder in dns.

Derelict's points are very valid though about "not a pfsense question" But thats what we have the forum for - to discuss and exchange information between people that know and have experience and people that do not.

I have been doing network design/support for years. I see many people with years and years in the biz still not grasp some concepts for good traffic flow.. And still think oh will just through a lagg on there when need more bandwidth. And then don't get why there is still issues with congestion, etc.

I still see home runs all the way back to the edge router vs doing routing internally to remove traffic from the uplinks, etc. etc. Teaching this sort of stuff is not something you could expect to get out of documentation for a specific product "pfsense"

Another problem I see is you have people that shouldn't be putting together guides - putting together guides. The amount of BAD information out there is horrible.. They try and latch onto pfsense to draw traffic to their site so they put together some guide on how to do XYZ, and its just horrible advice, etc..

I always attempt to ask the user why they want to do something, because more often then not they are going down the wrong path to the correct solution.. But they get something in their heads and its sometimes impossible to get them to listen they just want to know how to do X, when they should be doing Y. Bridging router interfaces vs a using s switch is a big one that comes up over and over and over again. They don't see anything wrong with it...

Hang out here and sure there will be many discussions on network design in general. But putting together a use case that goes into all the details of why you did this network decision vs another way is a lot of work, and will go WAY over the heads of the vast majority of users here anyway.

The vast majority of users here are just new users to pfsense asking questions - that tend to be the same questions over and over and over again. But there are a few here that know what they are talking about - and that is when it gets fun and interesting is when you get a discussion going where the person wanting to learn actually gets it - your ahah moment was fantastic to me knowing I help light the lightbulb above someones head.. Or discussion about the how's and why of something - Derelict and JimP are very very good just in IT in general and then when it comes to pfsense they are just gods amoung us.. So yes they can smack down some good wisdom, etc..

if you have a networking question that is design related - just ask and sure lots of people will chime in with different views on the subject. Design is not always black and white as you think it might be.. Example I see no problem with running a native untagged vlan with tagged traffic. While Derelict is of the mind that if going to running tag then all should be tagged, etc. Both work - nothing wrong with either way, etc.

But if I have a native interface with no vlans, and then I want vlan on that interface I am not going to change the first naked network to tagged - its just extra work ;) heheh

Derelict · Jul 13, 2018, 4:58 AM

@johnpoz said in Converting from a Single Interface to a LAGG:

But if I have a native interface with no vlans, and then I want vlan on that interface I am not going to change the first naked network to tagged - its just extra work ;) heheh

I would never do that in the first place if it was something like a port to a managed switch. Tagged vlans all day every day no days off from the get-go. And if I was to make that change I would set a window to change the pfSense interface and the single(!) port it's talking to to tagged. It's worth it to do it right. It only takes a minute.

johnpoz · Jul 12, 2018, 1:08 PM

^ see my point exactly ROFL ;) His point is valid, but there is nothing wrong with have 1 untagged vlan on your trunk. It is a valid configuration - why they have the native vlan commands in cisco, and use of PVID etc. But Derelicts point is also valid - and if setting up a greenfield network that would prob be the best option.

There are many ways to skin a specific cat to be sure - while there might be a few ways to do something that are correct technically. It might go against standard best practice, and we learn bad habits and sometimes they are hard to shake.

Another example where me and Derelict conflict is using a not rule as more explicit allow in a firewall rule. Where he sees it as blocking with an allow. I don't see it this way, I just see it as more explicit allow, and you have to be very comfortable with your understanding and do valid testing of your rules. And yes above this not rule you should always put in place explicit blocks if that is what your looking to do, etc.. Its just another example where something can not always be black and white in a field that is pretty much always black and white ;)

If it was always the same questions over and over again - I wouldn't be here as long as I have. Now and then something new comes up, or a good discussion gets going on some good techie topic. Not always directly related to pfsense - but in general the board is here to help the pfsense user community and not really a general networking/IT discussion forum. There is the general section for those - and they added the vlan/layer2/switch section - which prob could of been added quite some time ago - but now that some of the appliances are coming with switch ports and not just router interfaces I think that section will see more and more threads going forward for sure. And in that section you are going to be very likely to get into discussion that go deeper into the lan network that moves away from actual pfsense stuff. How you handle uplinks of your vlans to either other switches or your router is not really a "pfsense" discussion - but can be very informative and fun to discuss.

guardian · Jul 13, 2018, 4:58 AM

Hi @johnpoz @derelict thanks for the replies - good discussion

@derelict said in Converting from a Single Interface to a LAGG:

@johnpoz said in Converting from a Single Interface to a LAGG:

But if I have a native interface with no vlans, and then I want vlan on that interface I am not going to change the first naked network to tagged - its just extra work ;) heheh

I would never do that in the first place if it was something like a port to a managed switch. Tagged vlans all day every day no days off from the get-go.

I agree your way is best practice, but other than sloppy what's the major concern? Security/VLAN hopping?

@johnpoz said in Converting from a Single Interface to a LAGG:

@guardian said in Converting from a Single Interface to a LAGG:

Anyone who is going to set up pfSense has got to know networking basics

You would think huh ;) Clearly not the case...There are people that try and setup pfsense that don't understand why they can not have 192.168.1/24 on both their wan and lan networks, etc.

Another scenario that comes up ALL the time - is why can pfsense not filter traffic between devices on the lan.. So the concept of what a router/firewall is doesn't seem to click to why you can not filter traffic between 192.168.1.100 and .101 on the gateway device..

@johnpoz You just made my point even clearer - that would be 2 points in the "gotchas" section for noobs. Having that type of material well organized would save a ton of time and heartache. I would not expect much greater detail either. The info would be laid out as "here's what works/must do/must not do (and in some cases could do/could not do where there are multiple choices). If you want to know more then google is your friend. They sell plans for "prefab" houses that show you how to build the design they have provided, but they don't teach plumbing and carpentry.

So basic network understanding is clearly not a requirement for trying to setup pfsense ;) The one that gets me the most is not understanding the basic difference between a resolver and a forwarder in dns.

Why is this coming up so much? Are people using a forwarder and wondering why pfBlocker DNS block isn't working?

Derelict's points are very valid though about "not a pfsense question" But thats what we have the forum for - to discuss and exchange information between people that know and have experience and people that do not.

Agreed, but wouldn't it be nice if we could raise the "staring level for the discussion."

I have been doing network design/support for years. I see many people with years and years in the biz still not grasp some concepts for good traffic flow.. And still think oh will just through a lagg on there when need more bandwidth. And then don't get why there is still issues with congestion, etc.

I think it goes back to how this stuff is taught. It's mostly about how to configure stuff, not why/why not do things a certain way. A lot of the training creates robots, not thinkers. A certain percentage will have a good mentor, or take a lot of time to really think about/experiment with things and really learn the craft. The vast majority will learn just what they need to get by and that's it.

I still see home runs all the way back to the edge router vs doing routing internally to remove traffic from the uplinks, etc. etc. Teaching this sort of stuff is not something you could expect to get out of documentation for a specific product "pfsense"

Again it wouldn't be teaching per se. I'd be calling is something like the pfSense Cookbook - Recipies for Home/Small Office Networks. Define the scope, and include the caveat to use at your own risk and google is your friend.

Again, provide what to do, and maybe include a paragraph in the "got ya section (or not)". Its about providing a blueprint. Either follow the blueprint or do some research to learn what you need to learn. It's often much easier/more fun to learn by doing.

Another problem I see is you have people that shouldn't be putting together guides - putting together guides. The amount of BAD information out there is horrible.. They try and latch onto pfsense to draw traffic to their site so they put together some guide on how to do XYZ, and its just horrible advice, etc..

Even more reason to have one set up by the community. I wouldn't have been looking for that type of stuff if I could have gotten it here.

I always attempt to ask the user why they want to do something, because more often then not they are going down the wrong path to the correct solution.. But they get something in their heads and its sometimes impossible to get them to listen they just want to know how to do X, when they should be doing Y. Bridging router interfaces vs a using s switch is a big one that comes up over and over and over again. They don't see anything wrong with it...

I have a teacher (in another field) who says "You can't fix stupid.", and this doesn't sound like something that would be in the scope of complexity
that I had imagined. Would this show up in a home or a small office (and I
don't mean a branch office)?

Hang out here and sure there will be many discussions on network design in general. But putting together a use case that goes into all the details of why you did this network decision vs another way is a lot of work, and will go WAY over the heads of the vast majority of users here anyway.

And for what I was suggesting that isn't necessary. Present ONE/TWO WELL DESIGNED examples of a network that includes the most common things that a SOHO/Home user would need with a few examples of how to make common modifications.

The reason for the design(s) is(are) to illustrate how to set up firewall rules (interface/floating) and NAT when working with Multiple VLANs and OpenVPN clients/servers. I know of nowhere that is properly illustrated/explained - and that is pfSense not general networking.

What I find most difficult is visualizing how the various elements interconnect with pfSense. I got as far as diagramming the interface and floating rules. I still haven't got lo, NAT, and OpenVPN clear. This is what I did as a personal reference (I posted it, but the forum conversion has totally mangled it) Ascii art is a bit lacking for this type of task.



LAN     ----*FW--(192.168. 1.1)--|
                                 |
VLAN10  ----*FW--(192.168.10.1)--|
                                 |            WAN
VLAN20  ----*FW--(192.168.20.1)--|         Interface              Public  or 
                                 |            IP                  Gateway IP
VLAN30  ----*FW--(192.168.30.1)--@@@--NAT--(x.x.x.x)-FW*---WAN--- (p.p.p.p) 
                                 |
VLAN40  ----*FW--(192.168.40.1)--|
                                 |
VLAN50  ----*FW--(192.168.50.1)--|
                                 |
VLAN60  ----*FW--(192.168.60.1)--|

                    Sample Firewall with Multiple VLANS
                       and a Single Internet Gateway



              ----LOCAL I/F----       *FW       ---GATEWAY I/F---
               IN    (I)   OUT                   OUT   (O)   IN    
         ---> ~F>R*********F<R<        |        >F>R*********F<R<  <---
   LAN        ~L>U*  -->  *L<U<        |        >L>U*  <--  *L<U<
   or   ******~O>L*  I/F  *O<L<~@~~~~~~|~~~~~~@~>O>L*  I/F  *O<L<****** WAN
   VLANx      ~A>E* RULES *A<E<        |        >A>E* RULES *A<E<
              ~T>S*********T<S<        |        >T>S*~~~~~~~~T<S<  


                            Detail of FW Blocks with 
                                 Floating Rules

DEFINITIONS

IN/OUT:         Direction is always with respect to the interface.
SOURCE:         The Initiator of the session to be blocked.
DESTINATION:    The recipient of the session to be blocked
I/F RULES:      Rules applied on the Firewall / Rules / (LAN,WAN etc.) 
                Tab for a given interface.
FLOAT RULES:    Rules applied on the Firewall / Rules / Floating Tab.

The IP for an interface sits inside the firewall block (FB) and access
to it can be blocked with a firewall rule

Floating Rules are processed before I/F rules, so they can be used to
bypass I/F rules.

Once the firewall block is opened for a session, the rules are no 
longer evaluated for that session. 

When creating rules that block an open traffic flow, the states 
associated with that traffic flow must be cleared before the new rule 
will be effective.  So either the traffic flow must stop and the 
state must time out and clear by itself, or you must use thw WebGUI
to either clear the chosen states, or to clear all states. (Clearing
all states will disrupt streaming traffic and downloads.)

Rules can only block traffic in the direction shown by the arrows
shown with the particular rule.  Interface/Normal rules can only block
traffic going into that interface - either from the WAN into pfSense
box, or from a (V)LAN into pfsense as shown.

Floating rules may use one rule to appear on multiple interfaces 
simultaneously, and will work as if they were placed on the diagram
as shown.

Anyway that's my $0.02 for what it's worth.

Derelict · Jul 13, 2018, 5:04 AM

@guardian said in Converting from a Single Interface to a LAGG:

pfSense Cookbook - Recipes for Home/Small Office Networks

If I wrote that I would sell it, not give it away. Just sayin'.

guardian · Jul 13, 2018, 4:09 PM

@derelict said in Converting from a Single Interface to a LAGG:

@guardian said in Converting from a Single Interface to a LAGG:

pfSense Cookbook - Recipes for Home/Small Office Networks

If I wrote that I would sell it, not give it away. Just sayin'.

Who said anything about giving it away?

I was suggesting that this is what would provide REAL value for Gold Membership for the hordes of neophytes that attempt to set up pfSense. I did a gold membership for awhile as a "thank you", but I didn't feel that I got much value out of it other than supporting the project.

It could also be a stand alone product - As I said the education out there sucks for someone who wants to learn enough to do good small networks, but not become a CCNA. I was suggesting Netgate might do something like to raise the profile of the project, and make a few $$. The big obstacle with setting up pfSense is the configuration. I would suspect a publication like that could bolster the sales of their appliances if promoted correctly. Think the person who knows a bit, is reasonably handy with computers, knows that even the high end consumer routers out there are untrustworthy/not updated etc, and need a hand to get to the next level.

I don't know if they sell their lower end appliances to "educated consumers" or just enterprise, but that's the market I see it serving. When I was getting started I would have paid $100 for that in a heartbeat - and depending on the contents, I might still be in the market.