A good PRIQ Howto?
-
Hey all,
I have a network setup that lends itself very well to basic per IP prioritization.
I figure I'd set it up like this:
Priority: (highest to lowest)
1.) VOIP devices
2.) HTPC's
3.) Default for new clients
4.) SFTP Server
5.) Other misc File transfer server
6.) Cloud Backup ServerIn my setup, due to virtualization, most things have their own dedicated IP, I could call out in PRIQ. 1 & @ are very unlikely to crowd out 3, and I don't care if anything below 3 gets temporarily starved.
So I guess my question is, has anyoen seen a good step by step guide for setting up PRIQ? I've been googling but can't find a good one. All the guides out there seem to go through the HFSC setup, not the PRIQ setup.
I'd imagine I can just run the wizard and see what happens, but I'd prefer to know what's going to happen in advance. How do I define which clients belong to which queue?
Also, is it possible to nest queues, such that - in my example above - PRIQ is the overall priority ordering, but that clients within PRIQ queue 3 share bandwidth using HFSC between eachother?
Appreciate any help, links, guides and suggestions!
Thanks,
Matt -
A better approach might be to use the wizard, and THEN try to understand what the wizard did and why by asking questions and experimenting. For your case, I would use the wizard with a bunch of shaping options such as VoIP, P2P, etc so that it creates all of the queues you will need. From there, you can go and add/remove floating rules to direct the traffic you want into the specific queue you want.
-
I agree with KOM. The PRIQ wizard is pretty good at setting things up. I would create aliases for your voip servers before you start and use those in the wizard. Makes it a bit easier to manage if you have alot of voip addresses.
-
Thank you both for your thoughts.
While this isn't exactly an enterprise production environment, I am still a bit concerned about downtime, which is why I was hoping to get an understanding of what to expect before starting.
I was considering installing it in a VM and playing around, but it would be a pain to set up a bunch of guests to serve as clients behind it…
-
I was considering installing it in a VM and playing around, but it would be a pain to set up a bunch of guests to serve as clients behind it…
Well, no not really. Install VirtualBox or VMware Workstation. Install pfSense. Download a Lubuntu ISO, install & configure it, then clone it to a few other clients. Done. My home lab uses this approach with 1 pfSense instance with WAN, LAN and DMZ, 3 Lubuntu LAN clients, 2 Win7 LAN clients, 1 Ubuntu Server DMZ web server. I can simulate a lot of things this way. It only took a couple of hours to create and configure everything, and it's been invaluable when testing pfSense or just playing around.
-
I have a small Form factor dell PC that I have it loaded on. I use that behind my cable modem and just bridge out the wireless router. I got the PC for like $50 added in two Intel NIC's I had laying around and bam - insta router!!
-
@KOM:
I was considering installing it in a VM and playing around, but it would be a pain to set up a bunch of guests to serve as clients behind it…
Well, no not really. Install VirtualBox or VMware Workstation. Install pfSense. Download a Lubuntu ISO, install & configure it, then clone it to a few other clients. Done. My home lab uses this approach with 1 pfSense instance with WAN, LAN and DMZ, 3 Lubuntu LAN clients, 2 Win7 LAN clients, 1 Ubuntu Server DMZ web server. I can simulate a lot of things this way. It only took a couple of hours to create and configure everything, and it's been invaluable when testing pfSense or just playing around.
I guess you are right. Not THAT much of a pain.
I already have an ESXi box (which my pfSense currently runs on), so I guess I could just clone that, and work from there…
-
Well,
I spent some time playing with guests on my ESXi server, trying to figure out traffic shaping and I got pretty much nowhere, and exhausted myself to where my head was spinning.
I've been doing this stuff for 20+ years, and I just can't seem to figure out the traffic shaper in pfSense. it just makes no sense. (which is amusing, because the name pfSense is derived from making BSD's pf make sense)…
I'm starting to gather that the number of people who truly understand how it all comes together are a pretty small and elite crowd.
Hopefully some of you are among that group.
So I think I understand the basics.
In order for traffic shaping to work, I first have to create a few queues, and then firewall rules to send the appropriate traffic through the rules. That part is fairly easy to understand.
Though the practical implications of this aren't. For instance, does this mean that anything that has an existing firewall rule (like a port forward) bypasses the queue? Do I have to choose to have a firewall rule like a port forward, or a queue, not both?
What I want to do OUGHT to be relatively straight forward.
my desire:
1.) Set up a few PRIQ queues (for arguments sake, one at each of the 7 priority levels)
2.) Create firewall rules to direct traffic to and from the priority level of my choosing.
3.) Create one last firewall rule that sends traffic from any other IP through my chosen default priority queue.
4.) profit.Staring at the damned menus in pfSense I just can't figure out how to do it though. It's making me feel dumb. And it really shouldn't be this difficult to figure out.
I feel like I probably could if there were detailed documentation spelling out how EXACTLY every field works and interacts with others, but there isn't, just this terribly vague traffic shaper guide, which IMHO is next to useless.
Anyway… I'm done ranting.
Does anyone have any pointers, suggested reading or anything like that to help get me in the desired direction?
Thanks,
Matt -
https://www.dropbox.com/s/6loxfax6k4xr78u/LANPARTYPRIQSLSW.zip
Download this - extract it. Restore it to your pfsense. You might have to rename interfaces or adjust the gateway on some queues but outside of that this should work out of the box.
This is a single WAN / single LAN PRIQ config.
-
Sideout, i haven't loaded the configs but I scanned through by hand and see you've got OPT1 and OPT2 interfaces defined in the ruleset. What are those for? Different LAN segment, wifi…?
TIA
-
I am only using WAN and LAN in this one . I will look at it again but I am pretty sure it does not include OPT1 and OPT2 unless it just picked them up from the machine I was using as it had a 4 port PCI Express NIC in it.
-
I see where you are showing OPT2 , it is picking up the other NIC's in the box so you can either edit the XML to remove OPT1 and OPT2 or do it in the GUI.
-
https://www.dropbox.com/s/6loxfax6k4xr78u/LANPARTYPRIQSLSW.zip
Download this - extract it. Restore it to your pfsense. You might have to rename interfaces or adjust the gateway on some queues but outside of that this should work out of the box.
This is a single WAN / single LAN PRIQ config.
Thank you for this!
I downloaded and played with these settings in a double natted guest, with three duplicated ubuntu servers behind it for testing purposes on my ESXi server the other day.
I am still rather confused, but I am moving in the right direction. I plan on posting some follow-up questions here (with screen shots) in the near future.
Thanks,
Matt -
Alright,
so maybe no screenshots at first, but here are a few questions:
1.) The config files you provided. Shaper config appears to be the shaper queues, filter config appears to be the supporting firewall rules, but what is aliases?
2.) When setting up HFSC you need to tell it how much bandwidth you have up and down to make sure that prioritization occurrs locally, rather than remotely. I can't seem to find where this setting is in your PRIQ example. Is it not required for PRIQ? Only one of your queues has a "Queue limit" of 500, and it is qLink, which doesn't appear to be assigned to anything in rules.
3.) I'm gathering from your rules that traffic rules should be floating rules? What is a floating rule?
4.) Some of your queue's are assigned to WAN and some to LAN. Does this correspond to incoming and outgoing traffic? Which is which? If I had to wager a guess upstream would be on the LAN side and downstream on the WAN side. Is this correct?
5.) Clicking through all of your queues, I can't seem to find where I tell the queue if it is HFSC or PRIQ? How do I define this?
6.) Do you recommend starting with the wizard and modifying the queues as needed from there, or creating them manually?
7.) I can see how I can assign hosts to each queue using rules. How do I tell the system to send all other clients that have not been manually assigned to a "Default client" queue? Is it just like other firewall rules, where I create an ALL rule at the bottom, that assigns everything that hasn't been otherwise specified to my "default" queue?
8.) In your example, you have specified UDP or TCP for all of your rules. Is there any reason I can't just tell it to apply to all protocols for the specific host?
9.) It would seem all of your rules are associated with the WAN interface. Some specify the source and some the destination. I'd imagine that this is to create rules for upstream and downstream for each. Is that accurate? I would have expected based on the observation in #4 above, that downstream would need to be assigned to WAN, and upstream to LAN. Is this not the case?
Anyway, I REALLY appreciate your help. This is some of the best information I have found on this topic to date. Thank you very much for your guidance!
–Matt
-
So i have typed a reply to this like 5 or 6 times and then erased it and started over.
I am at a loss on how to reply to some of these.
-
So i have typed a reply to this like 5 or 6 times and then erased it and started over.
I am at a loss on how to reply to some of these.
Sorry, didn't realize I was asking such tricky questions.
If you know some of them, but not all, I'd love to hear as much as I can to learn it!
-
It is not that your questions are tricky. It is that alot of the questions you asked would already be answered if you read some of the documentation.
-
It is not that your questions are tricky. It is that alot of the questions you asked would already be answered if you read some of the documentation.
Ahh,
Thank you. Which documentation are you referring to?
I read the official pfSense Traffic Shaping Guide which is a decent view from 30,000 feet, but doesn't provide much in the way of detail, which I was hoping to get to in my questions above.
Or maybe there is just a level of assumed knowledge in that guide that I am lacking?
Is there another document that would be helpful you recommend reading?
Thank you,
Matt -
1.) The config files you provided. Shaper config appears to be the shaper queues, filter config appears to be the supporting firewall rules, but what is aliases?
Seriously you don't know what aliases are after reading the tab in PFSense?
2.) When setting up HFSC you need to tell it how much bandwidth you have up and down to make sure that prioritization occurrs locally, rather than remotely. I can't seem to find where this setting is in your PRIQ example. Is it not required for PRIQ? Only one of your queues has a "Queue limit" of 500, and it is qLink, which doesn't appear to be assigned to anything in rules.
Multiple forums post on this - HFSC does not use the priority setting but the wizard puts it in there. Also if you look at all the check marks on qLink you would see it is the default queue on the LAN interface so you would know that typically if there is not a rule allowing or disallowing something then it goes to the default rule.
3.) I'm gathering from your rules that traffic rules should be floating rules? What is a floating rule?
Again - you dont know what a floating rule is after reading the tab in PFSense? Plus if you went here https://doc.pfsense.org/index.php/Category:Firewall_Rules then you will see the very same question you asked answered already.
4.) Some of your queue's are assigned to WAN and some to LAN. Does this correspond to incoming and outgoing traffic? Which is which? If I had to wager a guess upstream would be on the LAN side and downstream on the WAN side. Is this correct?
All the queues on the floating rules tab should be assigned to the WAN interface only. There are specific rules that get assigned to the LAN for things like the limiter.
5.) Clicking through all of your queues, I can't seem to find where I tell the queue if it is HFSC or PRIQ? How do I define this?Again you can only have HFSC or PRIQ not both. That is defined on the interface so if you go under Traffic Shaping and read what the drop down box says , you know what you have set.
6.) Do you recommend starting with the wizard and modifying the queues as needed from there, or creating them manually?I recommend creating them manually unless you dont know what you are doing then start with the wizard and choose a very basic simple setup and modify it from there.
7.) I can see how I can assign hosts to each queue using rules. How do I tell the system to send all other clients that have not been manually assigned to a "Default client" queue? Is it just like other firewall rules, where I create an ALL rule at the bottom, that assigns everything that hasn't been otherwise specified to my "default" queue?
https://doc.pfsense.org/index.php/Firewall_Rule_Basics
8.) In your example, you have specified UDP or TCP for all of your rules. Is there any reason I can't just tell it to apply to all protocols for the specific host?
In my experience I have found that using a combo rule for TCP/UDP with HFSC shaping does not work that well in high packet situations. I prefer to separate them as when using floating rules with TCP you need to define qACK but with UDP you do not need qACK.
9.) It would seem all of your rules are associated with the WAN interface. Some specify the source and some the destination. I'd imagine that this is to create rules for upstream and downstream for each. Is that accurate? I would have expected based on the observation in #4 above, that downstream would need to be assigned to WAN, and upstream to LAN. Is this not the case?
https://doc.pfsense.org/index.php/Firewall_Rule_Basics
-
1.) The config files you provided. Shaper config appears to be the shaper queues, filter config appears to be the supporting firewall rules, but what is aliases?
Seriously you don't know what aliases are after reading the tab in PFSense?
2.) When setting up HFSC you need to tell it how much bandwidth you have up and down to make sure that prioritization occurrs locally, rather than remotely. I can't seem to find where this setting is in your PRIQ example. Is it not required for PRIQ? Only one of your queues has a "Queue limit" of 500, and it is qLink, which doesn't appear to be assigned to anything in rules.
Multiple forums post on this - HFSC does not use the priority setting but the wizard puts it in there. Also if you look at all the check marks on qLink you would see it is the default queue on the LAN interface so you would know that typically if there is not a rule allowing or disallowing something then it goes to the default rule.
3.) I'm gathering from your rules that traffic rules should be floating rules? What is a floating rule?
Again - you dont know what a floating rule is after reading the tab in PFSense? Plus if you went here https://doc.pfsense.org/index.php/Category:Firewall_Rules then you will see the very same question you asked answered already.
4.) Some of your queue's are assigned to WAN and some to LAN. Does this correspond to incoming and outgoing traffic? Which is which? If I had to wager a guess upstream would be on the LAN side and downstream on the WAN side. Is this correct?
All the queues on the floating rules tab should be assigned to the WAN interface only. There are specific rules that get assigned to the LAN for things like the limiter.
5.) Clicking through all of your queues, I can't seem to find where I tell the queue if it is HFSC or PRIQ? How do I define this?Again you can only have HFSC or PRIQ not both. That is defined on the interface so if you go under Traffic Shaping and read what the drop down box says , you know what you have set.
6.) Do you recommend starting with the wizard and modifying the queues as needed from there, or creating them manually?I recommend creating them manually unless you dont know what you are doing then start with the wizard and choose a very basic simple setup and modify it from there.
7.) I can see how I can assign hosts to each queue using rules. How do I tell the system to send all other clients that have not been manually assigned to a "Default client" queue? Is it just like other firewall rules, where I create an ALL rule at the bottom, that assigns everything that hasn't been otherwise specified to my "default" queue?
https://doc.pfsense.org/index.php/Firewall_Rule_Basics
8.) In your example, you have specified UDP or TCP for all of your rules. Is there any reason I can't just tell it to apply to all protocols for the specific host?
In my experience I have found that using a combo rule for TCP/UDP with HFSC shaping does not work that well in high packet situations. I prefer to separate them as when using floating rules with TCP you need to define qACK but with UDP you do not need qACK.
9.) It would seem all of your rules are associated with the WAN interface. Some specify the source and some the destination. I'd imagine that this is to create rules for upstream and downstream for each. Is that accurate? I would have expected based on the observation in #4 above, that downstream would need to be assigned to WAN, and upstream to LAN. Is this not the case?
https://doc.pfsense.org/index.php/Firewall_Rule_Basics
Thank you. I do appreciate you taking the time, and having a little patience with me.
I think part of my problem is a terminology gap. Been doing a lot of googling and browsing around the pfsense documentation, but obviously not for the right terms!
This - hopefully - should point me to the right reading to understand all of this.
Thanks!
