HFSC - Lan Party shaping for 150 / Multiple Cable Modems - Reference Topic
-
Here is a list of all my posts about shaping. I will keep this post updated with each LAN Party.
Goals
1. The goal is to provide LAN party admins a guide on how to config PFSense for best results at LAN Parties.
2. Provide a living knowledge base for referenceSystem / Requirements
1. I run PFSense on primarily an enterprise class server or similar hardware.
a. Dell R610 1U Server with 16GB RAM / 6 core Intel Xeon CPU / 12 NIC's - 4 onboard / 8 offboard / 76GB RAID 1 SAS on 10K drives / dual 750Watt PS
2. I use an 8 port amplified modem amp to provide clean / amplified signal to all modems
3. I use primarily Motorola SB6141 modems
4. I use a Dell 5448 switch for my core switch currently as it has some Layer 3 capability and 96Gbps backplane.
5. I use PRTG or Obserium to monitor the LAN. I setup netflow from PFSense to the PRTG server.Shaping priorities
1. Gaming traffic has priority over all requests but qDNS and qACK
2. TCP streams are limited with LAN interface rules.
3. Floating rules are utilized for WAN interfaces to shape traffic
4. TCP rules are implemented on the LAN to limit traffic
5. A load balancing group is created with 2 or 3 modems in this configuration.
6. Apinger is disabled and all gateways are considered live.
7. Custom DNS is used and each WAN is assigned DNS.
8. Sticky connections are utilized.Traffic Queue Setup
1. Queues being utilized:
A. qInternet - 150Mb
1. qGames - all gaming and high priority traffic / 30% bandwidth / 500 queue depth / 25% realtime
2. qHTTP - web traffic / downloads / 30% bandwidth / 500 queue depth / 50% Linkshare
3. qACK - ack traffic / 15% bandwidth / 500 queue depth / 15% realtime
4. qDNS - DNS traffic and other medium traffic / 15% bandwidth / 500 queue depth / 15% realtime
5. qLink / qDefault - all other traffic / 8% bandwidth / 500 queue depth / 8% max2. A limiter is setup as foxdale did with a Download / Upload queue to set bandwidth limit and a sub queue to define how it does it.
A. Apply this to a LAN firewall rule right above the default ANY / ANY Rule.Floating Rule Setup
1. Remember that floating rules are applied differently than LAN rules .
2. Choose only the WAN interfaces when using floating rules.
3. Use aliases to help with keeping rule set clean.
4. If using a large number of ports in a TCP or UDP group - make a separate rule for TCP and UDP.
5. If you are using aliases for IP destinations place those rules first and again use separate for TCP and UDP.General Tips for keeping Internet traffic optimized during the LAN
1. Watch who has uPNP connections and what type they are - you will find torrenters in this group.
2. Monitor bandwidth out or upload on each WAN - if a WAN interface is getting maxed out then someone is running a torrent.
3. Try to release and renew on your modems to get different gateways if possible.
4. Adjust the limiter download amount during tourney's to give max amount to game traffic. Can use schedules maybe.List of previous topics
https://forum.pfsense.org/index.php?topic=92938.0 - Nexus LAN Stats April 25 2015
https://forum.pfsense.org/index.php?topic=91545.0 - HFSC config March 30th 2015
https://forum.pfsense.org/index.php?topic=99405.0 - HFSC Shaping LanOC v17 Sept 12 2015
https://forum.pfsense.org/index.php?topic=99503.0 - HFSC Lan Party Gold Config Sept 15 2015
https://forum.pfsense.org/index.php?topic=100342.0 - HFSC Shaping in Action NeXusLAN Oct 2015
I hope this helps people out with LAN parties and making it easy to use HFSC for shaping.
-
All your queues are RT except default to the tune of 75%. How does that work in practice? I would have thought that when everything is special, nothing is special. I wonder if you would get the same results by making them all LS instead of RT.
-
Sorry that is my mistake - late night edit with a copy paste gone wrong. Only qGames / qDNS / qACK have RT.
I corrected the topic
-
OK, that makes more sense to me. After banging my head against the HFSC wall, I think my tiny brain is finally starting to understand it.
-
@KOM:
All your queues are RT except default to the tune of 75%. How does that work in practice? I would have thought that when everything is special, nothing is special. I wonder if you would get the same results by making them all LS instead of RT.
Technically, removing all uses of real-time and using exclusively using link-share would yield virtually the same results as setting link-share & real-time to the same values. I doubt the over-usage of RT is a problem since the original HFSC implementation only had a "service-curve" parameter that simultaneously set both LS & RT to the same values (IIRC).
-
Edited doc with link to NeXusLAN graphs
-
Added a tested MultiWAN HFSC config with alias.
-
Next LAN is coming up on March 18th / 19th. Since I have had issues with the cable modems all pulling the same gateway , I purchased 3 Linksys wired routers to place in front of Pfsense and behind each modem.
Each Linksys is configured with a different IP range , Linksys 1 is 192.168.10.1/24 , Linksys 2 is 172.16.10.1/24 , Linksys 3 is 10.10.10.1/24 .
I turned off all firewall and other features on these routers including DHCP. PFSense WAN interfaces will be 192.168.10.10/25 , 172.16.10.10/24 and 10.10.10.10/24
I am changing the monitor IP for each WAN to match DNS as well. WAN 1 will be 4.2.2.2 , WAN 2 will be 4.2.2.3 , WAN 3 will be 4.2.2.4 .
I am creating manual NAT rules on PFSense for the WAN's as well.
-
@sideout are you using the latest version of pfSense or staying on 2.1.5 for LAN party use?
-
Using the latest version right now. I have another firewall on an older version for backup use.
-
After looking at some conversations around Snort and OpenAppID , I am going to run Snort at the next LAN and use OpenAppID to block unwanted applications from running. Attached is my custom list of Snort rules to apply. You would do this after you install Snort and assign it to an interface. You would also need to assign it to all WAN interfaces if running multiWAN. You would choose custom rules after enabling OpenAppID for Snort.
Just copy and paste this list in the window and hit save.
Some things to consider - Change how Snort filters based on your hardware. If you dont have a large swap file for PFSense you might want to reinstall and choose custom install and make a large swap file partition.
-
Changes coming for the config. Will post up zipped files at a later date. Adding alias's for new games and a few other changes.
-
Hello @sideout, really appreciate for your uploaded config files. It gave me a lot of knowledge. But one thing giving me headache is I can't make the queue work alone with floating rules. I always needed to add those queue to LAN rule to work. Is it normal or am I missing something? I am using 2.2-RELEASE (amd64).
thanks.
-
The floating rules should work with just choosing the WAN interfaces. You should not have to choose the LAN on floating rules .
-
Awesome work man, completely excellent reference post, very detailed and easy to follow. One thought, do you think you might get better bufferbloat conditions with a buffer queue depth lower than 500? I'm at 50 for most of mine, and it made quite a difference when speed testing with DSLreports.
Do you find that when you're leveraging sticky connections that the traffic is still fairly well balanced across all the WAN links? I'm not seeing that in my small scale testing, but perhaps I just don't have enough endpoints yet. I saw weird behavior where it was like all the states shifted from one WAN to another, then back. Fixed when I disabled sticky connections…but I'm thinking I'm going to need them for games like Battlefield that burp when you change public IPs. Image of weirdness with (2) 50Mbps modems.
Seemingly fixed with sticky connections removed…
I've recently been tuning my (your) config for a LAN this weekend, will be doing 3 modems exactly as you've mentioned, as a practice run for the next large one. One of the changes I made was different TCP download limiters for guest DHCP addresses and the lancache box, so that the caching box gets a bigger piece of the download pie rather than an individual user downloading. Super pumped to try that bad boy out…10Gb networking via a cheap 10Gb switch and a Mellanox 10Gb adapter.
-
Yes I would lower it to like 100. I havent noticed the sticky connections thing. Let me get the config from the LANOC firewall that I ran a bit ago as it has the most up to date Alias's and firewall rule configs along with some NAT changes that you will need. I will update this topic with it so you can download and look at it and import what you want.
Yea super jelly of 10G. I want!!! Good luck at ForgeLAN and thanks for the appreciation. Much mutual respect for what you do as well. Hopefully one of these days I can make it out to ForgeLAN.
-
The floating rules should work with just choosing the WAN interfaces. You should not have to choose the LAN on floating rules .
I got your point and followed according to your config files. But what I am trying to say is that I have to apply the exact same rules inside the LAN tab to work with the queue.
- Now I have rules in floating tab, choosing WAN interface. (But the queues don't work)
- And I applied exact same rule (pass rule) in LAN tab with appropriate Ackqueue/Queue. ( It's working )
I am just wondering why floating rule alone isn't working. ??
Thanks for the reply sideout. Always appreciate it. :D
-
I would update to the latest version and then re make the traffic shaping and make sure you clear all the states before you test. To test I would do this:
1. Log into PFSense and look at the queues page.
2. Open a new browser window - start browsing sites. Check the HTTP queue and see that it is getting traffic.
3. Open a game you know is defined under the rules and see if that is putting traffic into the qGames queue.Other than that , you should not have to apply the queues to the LAN rules at all.
Screenshots of your config would be helpful in troubleshooting.
-
Here is the latest config for multi modem's. This is the list of updates:
1. Added in Alias's for:
A. The Division , Battlefield 4 , Rocket League , Warframe , Wargaming Family of Games (WoT , WoS)
B. Cleaned up a few Alias's as well.
2. Cleaned up Floating Firewall rules to a more condensed list.
3. Made a generic password - pfsense111 so that you can use my System config which has modifications to it.
4. Added back in apinger with Gateway monitoring of Level 3 external DNS for the IP so that when getting same gateway on cable modem's you will get a true RTT now instead of using default gateway.
5. Modified Traffic Shaper so all queues are set to 100.
6. Modified Traffic Shaper for the following split:
A. qACK - 20%
B. qHTTP / qGames - 35%
C. qDefault / qCatchAll - 10%
7. Added NAT configs so that static port mapping is enable for all WAN's to help with console use at LAN Parties - this is just for generic console use on your tables. This is not going to fix Halo 5 issues on Xbox One problems with Teredao IP and Strict NAT.
8. UPnP is enable by default.So to use this config do the following:
1. Download the Zip and extract.
2. Login to PFSense and restore
3. Remember the password is pfsense111
4. Rename the WAN's as you desire. If you need more than 2 then enable Traffic Shaping for them as it is not checked right now. There are 4 WAN's in this config.
5. Modify DNS under General if you dont want to use who I have set there.
6. CHANGE THE LIMITER UNDER FIREWALL / LIMITER to what limits you want. - right now this is set at 5Mbits for Down and 2Mbits for Up as I was testing.As always backup your config before you put mine on your system. Remember to reset your states. Enjoy the config and happy LAN partying!!!! if you have suggestions please post in a different topic as I want to keep this clean for edits and updates of the config.
Thanks.
