HFSC Config used at 150 person LAN Party

[sideout]

Just had our 150 person LAN party this weekend.  We had 3 TWC 50/5 modems hooked up.

Used my HFSC shaping config with Codel queuing .

At one point I had the limiter set for 90Mbit and we had about 85Mbit sustained HTTP / Web traffic going and were still getting under 100MS pings in LoL (League of Legends) games and other internet games we had going. We even had a twitch TV stream going the whole time streaming the event and doing some spectate casting of games with commentary.

This is a 3 WAN / 1 LAN config with a ton of floating rules and Alias's used.

if your interested in getting the config - PM me with your email and I will send it to you. I am working on getting a clean Single WAN / LAN config and will put that up in dropbox for people to download.

Ran this on an Intel Core i3 quad core CPU with 8G RAM on a 256G SSD with a 4 port Intel ToE Server NIC in it.  That feed into a Dell PowerConnect 2824 switch.

For me at least HFSC works out great and does what I need.

[sideout]

here is the config for single wan / single lan use.

A couple of things:

1. Limiter is set at 40Mbit so be aware of that.
2. This is optimized so that qGames, qLoL , qDNS have reserved bandwidth for those queues.
3. qCatchall and qDefault have lower bandwidth.
4. Read over my floating rules and redo them as needed.

Again this is optimized for LAN party use where you have people trying to game on mostly UDP packets versus all the people who forgot to update or install games at home trying to download on TCP.
This works for me at my LAN parties . I do 4 a year of 150 people each and I have practically zero issues using it unless it is past the modem.

if you get new games and they run like crap , you will need to research ports , make a floating rule and assign it to qGames and that should resolve it.

Enjoy.

https://www.dropbox.com/s/drih6ls87l0iaoh/singlewansinglanconfig.zip?dl=0

[Nullity]

Have you noticed whether HFSC "Priority" actually works or not?

[sideout]

No I haven't but then I don't really care because I am setting real time limits on my high level queues anyways so they have reserved space on the meter.

I use the load balancing limiter so it shares my bandwidth that I set equally for the most part across all the people trying to download.

[sideout]

For me , with having qDNS and qACK with reserved bandwidth along with my other queues , like qLoL and qGames , I am able to put my own level of priority into it.  LAN parties are unique situations to shape and deal with.

You have all kinds of games running from Plants versus Zombies running TCP to LoL and Blizzard games using UDP then throw in Steam downloads and LoL downloads and you can have a mess real fast without some control.

This config is probably my best one yet and the result of about 6 LAN parties of playing with along with running it at home on my network.

I am far from a technical HFSC person however , I know that I can run it on my PFSense and get it to do what I need it to do.  Hopefully now that I have golden config , I won't have to mess with it alot and be able to enjoy the LAN's more than in the past.

[Nullity]

If it works, it works. I am mostly theory and simulation, with very little real-world experience.

Dealing with bulk download traffic is something I am trying to learn more about.

[Derelict]

Is there a specific reason you don't have qACK configured with any realtime?

[KOM]

Thanks for the post.

Limiter is set at 40Mbit so be aware of that.

Where is that?  I didn't see an actual limiter defined if that's what you mean.  I do see the WAN set to 5.5 Mbps, which I believe is the only thing that needs to be modified to suit the bandwidth?

[sideout]

Traffic shaper  > Limiter > Download queue - set to 40Mbit

[sideout]

Here it is.

[sideout]

@Derelict:

Is there a specific reason you don't have qACK configured with any realtime?

In the lan party config i have it set to realtime. This is my home / lan party config and i changed it back to just link share here.  I set it to realtime of 10 at the LAN.

[KOM]

OK, thanks.  I thought it was something that was missing from the shaper config.xml.  If you have a 40 Mbps limiter, why do you have WAN set to 5.5 Mbps?

[sideout]

That is upload limt.  The Download queue in the shaper is used in the LAN rule to limt TCP streams.  This shares whatever you set in there equally with all machines wanting a TCP stream.

[sideout]

If you want to test it , set it 10Mbit , run a speed test.  Close all browser windows .  Set it to 5Mbit , run a speedtest again.

[KOM]

What I mean is, why do you have a 40 Mbps limiter when the WAN queue in the shaper is set to 5.5 Mbps?  I thought that if your WAN is set to 5.5 Mbps then your limiter would have to be less than that…?

[sideout]

If you look at the rule the limiter is only applied to the LAN. You are looking at the WAN bandwidth on the Traffic shaper which is what you set  from your ISP limits.

[Derelict]

@Nullity:

Have you noticed whether HFSC "Priority" actually works or not?

Doesn't the fact that the priority, if set on an HFSC queue, doesn't make it into the pf altq queue config at all pretty much settle that?

[Nullity]

@Derelict:

@Nullity:

Have you noticed whether HFSC "Priority" actually works or not?

Doesn't the fact that the priority, if set on an HFSC queue, doesn't make it into the pf altq queue config at all pretty much settle that?

Yes, that along with the fact that the m1/d/m2 parameters are technically how HFSC's prioritization is configured, but I thought I would ask anyway.

[KOM]

If you look at the rule the limiter is only applied to the LAN.

OK< that's the part I was missing.  You would think I would have picked that up by the names, Download_LAN, Upload_LAN, but no…

[sideout]

Yes I try and apply the KISS principle to any of my naming conventions . That way if I get hit by a bus , someone should be able to come in behind me and pick it easily.

Glad you found it.

[Harvy66]

@Nullity:

@Derelict:

@Nullity:

Have you noticed whether HFSC "Priority" actually works or not?

Doesn't the fact that the priority, if set on an HFSC queue, doesn't make it into the pf altq queue config at all pretty much settle that?

Yes, that along with the fact that the m1/d/m2 parameters are technically how HFSC's prioritization is configured, but I thought I would ask anyway.

"Prioritization" is kind of a loaded word, but is the closest concept with a lot of abstract level overlap. I'm not sure the best way to say it, but I guess you could say "m1/d/m2 is how HFSC manages minimum bandwidth and maximum latency". I'm not correcting you, just thinking out loud about the best way to word it to people less familiar with HSFC's concepts.

P.S. Been busy lately, need to check out your HFSC thread again.

[Derelict]

The problem is this language in the GUI:

For hfsc, the range is 0 to 7. The default is 1. Hfsc queues with a higher priority are preferred in the case of overload.

I have no idea why that is there.

[KOM]

From what I have read, priority is not part of the official HFSC spec but some implementations have added it in.

[Nullity]

@Harvy66:

"Prioritization" is kind of a loaded word, but is the closest concept with a lot of abstract level overlap. I'm not sure the best way to say it, but I guess you could say "m1/d/m2 is how HFSC manages minimum bandwidth and maximum latency". I'm not correcting you, just thinking out loud about the best way to word it to people less familiar with HSFC's concepts.

Yes, I agree, but it is pertinent as an argument againt the useless "Priority" section in the HFSC shaper GUI.

[mcwtim]

Thanks again for sharing your configs and info sideout.

Did you have any other issues moving from 2.1.5 to 2.2?

[sideout]

No I didnt have any issues upgrading with this config.

[Nullity]

Do you have any tips about multi-wan traffic-shaping?
Did you run into unexpected difficulties, if so, what were they?
Do you ask the LAN party attendees to attempt to keep their downloading,/uploading to a minimum, or do they use the bandwidth however they want (torrenting?).

[sideout]

@Nullity:

Do you have any tips about multi-wan traffic-shaping?
Did you run into unexpected difficulties, if so, what were they?
Do you ask the LAN party attendees to attempt to keep their downloading,/uploading to a minimum, or do they use the bandwidth however they want (torrenting?).

Multi-Wan traffic Shaping:

1. Use LAN Rules above the default any any rule to get traffic to go where you want it to go , ie we had a twich.tv stream running the whole time that we streamed the LAN and tourney's over so I dedicated one modem to the static IP's of the streamer PC and camera's and did not let them hit the limiter.  I set up a group for all modems at Tier 1 for the default any any rule.  Then you can set groups for two modems for gaming with the third as tier 3 or something like that.

2. At this point I had most of the bugs worked out of it.  One thing is that more and more games are starting to go back to TCP ports instead of UDP for game traffic so you have to put LAN interface rules in for those to not get hit by the limiter. You also need to make sure that you put separate DNS server entries in for each interface so it will be able to get to the Internet.

Also realize that PFSense will spread traffic out across your modems so if you set your limiter to say 50MB when you have 3 50MB modems then your already capping yourself even more.  At first I thought that I had to set the qInternet to what just one modem is but then realized that I needed to set it to what all 3 were.  The same applies to the limiter as well.

3. We dont ask them to limit bandwidth since we do that for them.  I generally open it to like 90Mbit on the limiter until we get ready to start our first online tourney then I knock it down a bit.  I do monitor it and see if someone is hogging a lot of traffic consistently and then I do this:
        a. Get their MAC and make a static DHCP lease
        b. Make a LAN rule that blocks all traffic to and from that lease
        c. Kill all their states and remove the current DHCP lease. 
        d. Sit back and wait for them to come up and find out what they are doing.

We put Steam backups of all the games we are playing for tourneys on a server so no one has to update Steam.  We have played with a Steam caching server but with 150Mbit to the Internet - it is easier to just throttle the downloads as needed.

As stated above with have real time parameters on the important traffic and the multiple modems , I can let people download what they want for the most part without penalty.  I think at one point someone was downloading Heroes of the Storm at like 4mbit or 5mbit a second yet gaming pings to LoL were still at 95ms.

I have another LAN Party at the end of April with about 155 people so I will try and save some graphs and stuff from that and post them up.  I think I will have 3 modems for this one as well.

[JackPettit]

@sideout:

here is the config for single wan / single lan use.

A couple of things:

1. Limiter is set at 40Mbit so be aware of that.
2. This is optimized so that qGames, qLoL , qDNS have reserved bandwidth for those queues.
3. qCatchall and qDefault have lower bandwidth.
4. Read over my floating rules and redo them as needed.

Again this is optimized for LAN party use where you have people trying to game on mostly UDP packets versus all the people who forgot to update or install games at home trying to download on TCP.
This works for me at my LAN parties . I do 4 a year of 150 people each and I have practically zero issues using it unless it is past the modem.

if you get new games and they run like crap , you will need to research ports , make a floating rule and assign it to qGames and that should resolve it.

Enjoy.

https://www.dropbox.com/s/drih6ls87l0iaoh/singlewansinglanconfig.zip?dl=0

Hi, thanks for sharing your configs. I've downloaded this one and restored it to my 2.2 box however I'm not convinced it's shaping correctly. Attached is a screenshot of my Queue Stats whilst downloading via Steam.

I also noticed that the config in that zip doesn't include the limiters you mention. It's quite possible I've gone wrong somewhere, just can't get my head around it.


[Harvy66]

You have some wrong traffic going into your qACK. Based on average bandwidth and PPS, your average packet in that queue is 1,495.51 bytes. Nothing should be assigned directly to that queue, it should only be used as an ACK queue. When you select which queue to place traffic in, there are two options, and ACK queue, and a regular queue.

Only the left drop down should be set to qACK, NEVER the right one

[JackPettit]

No rules have the qAck queue in anything other than the left dropdown. What seems really odd is that I've only give 60Mbit/s to the Lan queues and the qAck is given 15%, yet in the stats I posted its states its pushing 50Mbit/s or more.

[Harvy66]

Bandwidth you assign is the minimum. Unused bandwidth is shared evenly.

[ilumos]

Thanks for posting this config sideout - been learning / experimenting with traffic shaping and its great to have a look at a tried and tested config specifically for LAN parties.

In your learning process over the last 6 LANs you put on were there any disastrous configurations that caused massive problems for your attendees? The reason I ask is in the past I've done more harm than good implementing traffic shaping incorrectly but never really understood where I went wrong…

[eri--]

@Nullity:

Have you noticed whether HFSC "Priority" actually works or not?

Just FYI since i read this, it does nothing :S

[sideout]

@ilumos:

Thanks for posting this config sideout - been learning / experimenting with traffic shaping and its great to have a look at a tried and tested config specifically for LAN parties.

In your learning process over the last 6 LANs you put on were there any disastrous configurations that caused massive problems for your attendees? The reason I ask is in the past I've done more harm than good implementing traffic shaping incorrectly but never really understood where I went wrong…

Yes I made plenty of mistakes. Sorry for the delay in reply but i was on a 12 day business trip.  Here are my tips for implementation of shaping live at the party

1. Apply the rule of least restrictive when making changes - ie person X says they cant get an online game to work.  So I would do the following:
    a. Run a capture on PFSense from the LAN interface using their IP and see where they are going and on what ports.
    b. Put in a floating rule for those ports / protocol - have them reboot and test again.
    c. If that doesnt work then you can apply a LAN rule with their IP and those ports and protocols and send it out a specific WAN.

2. You just made a change to the shaper and things went to crap -
    a. document your changes before you do it and backup your config so you have a good config to restore if it goes to crap.
    b. again dont make drastic changes unless you are majorly affecting tourneys and such.  If xyz cant download this new game to play as long as it is not a game at the
        LAN then it is not a priority.

3. Try and find out the IP's of the game servers being used on the Internet and use Alias's and floating rules to get traffic to them.

4. When you have online tourney's going on , like LoL , give it more bandwidth and take it away from something else.  With having the qLoL in my config that is way
    easier now to do than it used to be.

5. TEST TEST TEST TEST TEST at home.  Use PFSense at home on your LAN , test with your computers .  I cannot stress enough - TEST IT OUT at home first. 
    a. The more you test at home the better it is.  You can test one game specifically to see how changes affect game play.
    b. Use VM's  if you can as well to test to get more clients on the LAN or do what I do and limit the bandwidth to a lower number to simulate the load.

6. Never be satisified with your config and what you did - always look for ways to improve on it and use the monitoring tools in PFSense or others - Observium , PRTG ,
    etc to provide you live feedback of what is gong on with the network.

to sum it all up - KISS - Keep It Simple and Stupid when doing any changes.

Now to prep for the 155 person LAN party this weekend in Miamisburg OH - www.nexuslan.org .  Going to try a new monitoring package . will try and post up some detailed stats for everyone.

[gratis.obake]

am trying this one out. however I have a few question(s), although my wordings below might be very confusing, just try to re-arrange them :(

see screenshot and look at "qCatchAll"
I have deleted LoL and increased "qCatchAll" to 15%

I am using something like a diskless computer (iscsi) and therefor it will have too much activity on the LAN.
I have created qLink (995Mb bandwidth and link share) and qCCBoot (90% bandwidth and 90% link share) for example

a local computer acting as the iscsi server serves on port 3260 (192.168.1.222)
diskless clients are in the range of 192.168.1.10x
on a client computer, when I do netstat -o, it will indeed point to the iscsi server 192.168.1.222:3260, port on local computer is 49152 (but may change)

created something like on the LAN rules:
IPV4/TCP, LAN Net, 3260, LAN Address, *, *, qCCBoot
IPV4/TCP, LAN Net, *, LAN Address, 3260, *, qCCBoot

I have tried both LAN rules and did not insert them at the same time.
also tried any source/destination and switching 3260 from source/destination.

I cant seem to make it go to qCCBoot.

my main concern is that qCatchAll will be heavily saturated if something like any other activity (youtube/playing games/etc) and I see it have a QLEN on console a value of 2~4 when playing youtube instance in 1 computer, how much more if 5 computers are also doing http and stuff.

am not sure as well if I need to create a new thread on this one.

anyone be able to help, I would appreciate it, thanks in advance

[Harvy66]

gratis.obake, please make your own thread instead of hijacking someone else's

[gratis.obake]

ok sir, am really sorry, will take note.