@meruem:

Is there anything in the traffic shaper configuration to say "ignore this local ip" completely ?

Traffic is assigned to a particular queue by firewall rules. All unassigned traffic uses the "default queue".

If you are using CODELQ (I think it is just a single " default" queue), I dunno if you can avoid the queue.

The bigger problem is that Plex does not halt playback mid-stream for an unavailable, external resource. Additionally, pfSense should not even be encountering the LAN-to-LAN traffic… your switch should be handling that traffic.

If you could share more details about your LAN toplogy and your pfSense setup, it would help.

Did you reset pfSense's states?

If you still have problems, you can use pfSense's firewall logs or tcpdump to see what is happening from pfSense's perspective.

Do you need to use DSCP? Could use standard source/destination IP/port filtering?

You can use HFSC to create queues and assign minimum bandwidths, but I think there is a limit of 16 queues. The biggest issue with with anything that is "per IP address", is it is algorithmically complex and will have at least O(n) scaling. There is no good way to solve this issue and anyone who does have a "solution" is going to have performance issues at high speeds.

If you need to supply an SLA, there is no cheap solution except for low speeds. If all you care about is best effort, you're better off looking into stateless algorithms like CoDel, fq_CoDel, or Cake. Cake is the only one that can actually near perfectly distribute bandwidth among devices. CoDel and fq_CoDel just fight buffer bloat which in and of itself makes a world of difference.

I know I didn't answer the question at all, but I gave some food for thought.

Ok, i've got various gateway definitions (2 cablemodems being T1, one being T1, ADSL being T1, etc) as a way of distributing traffic between interfaces. To catch all traffic for a given interface, without messing all the other rules, how should i build the rule? And being that not a single one of those wan links is symmetrical, where CODELQ asks for bandwith, does it mean download or upload?

If you need more than a few PRIQ priorities, then you should not use PRIQ…

https://doc.pfsense.org/index.php/Traffic_Shaping_Guide#Priority_Queueing_.28PRIQ.29
Read the "Cons" section. (Starvation)

You may have a legitimate bug, but the bigger issue is that you should not be using PRIQ.
Use CBQ, FAIRQ, or HFSC.

@bwf.it35218:

Thanks for the help so far

The reason I want to also mark packets depending on VLAN is

On the LAN side there are two VLANs - VLAN 1 is the office network, VLAN 2 is the guest network

Both VLANs use WAN2 as the default gateway, but VLAN 1 has a failover to WAN1

Now I would like to use Limiter dynamic queues to equally share the available bandwidth with all the clients. So the office network traffic (VLAN1) needs to go into the same queues - for WAN2 - as the guest network (VALN2) until it fails over. Then the office network (VLAN1) traffic needs to go into queues for WAN 1

@Nullity - So what I'm thinking is, first mark packets from VLAN1, and then only match those marked packets in the two rules you suggested

VLAN2 will be directly added to the queues for WAN2 as that is the only gateway it uses.

If WAN2 (default GW) fails, you want the guest network (VLAN2) to be completely blocked?

If so, you could mark/tag the VLAN2 originating packets, then create a floating rule on WAN1 that blocks/rejects said packets.

I guess you could do the inverse and explicitly PASS only office (VLAN1) packets through WAN1, but I am too tired to comprehend whether that makes sense…

@moikerz:

I got fed up with the same problem - it seems the web page can't keep up with the queue updates, and it backlogs something chronic.

Instead, try connecting to pfSense using PuTTY, and call the pfTop command: pftop -s1 -v queue. It updates a lot faster.
(reference: https://doc.pfsense.org/index.php/Traffic_Shaping_Guide#View_Queues_with_pfTop)

Here's a pretty picture (attached). I have one WAN, two LANs, which is why the existing 5 Queues are repeated 3 times, once for each interface.

Yeah, I have never trusted the queue stats either.

FYI, you can view pftop via the GUI. Diagnostics->pfTop :)

the e/book "freeradius beginner's guide" will probably be your best source of help. Read it a few years back and it was quite useful.

@devlin016:

How can I install tcp cubic into pfsense?

https://tools.ietf.org/id/draft-rhee-tcp-cubic-00.txt

This is a guess… :)

TCP congestion algorithms like CUBIC rely on connections which are point to point. Therefore, the TCP congestion algorithm of an intermediary network node (your pfSense router or any other router/switch between the TCP connection's end-points) is a non-issue because they are simply relaying/forwarding the IP packets. The TCP congestion algorithm is only used when you are the creator or receiver of a TCP connection.

Example: Your TCP connection is connecting from localhost to Google. You are not creating a TCP connection from localhost to pfSense, from pfSense to the next node, etc, etc, until you connect to Google.

tl;dr - the TCP algo of your router is moot unless your router is the initiater of the TCP connection.

Hi Thank you so much for your reply. I will do some more research on your answer.

Depends on whether you're shaping on the OpenVPN client or the server.

If the server, set the queue on the firewall rule that passes OpenVPN traffic into the firewall.

If on the client set a floating match rule on WAN out for the OpenVPN client (UDP/1194?) and set the queues there.

There is not anything in the wizard for this. Use the wizard to establish basic queues and manually tweak from there.

@kfkehua:

Nope. My initial guess was correct.
see here: https://www.youtube.com/watch?v=nMJnp7GMwcg

In 2.2 they still had the descriptions. In 2.3 they removed all the description.  >:(

the first screen is where you spec your pipe bandwidth.
the second screen is where you reserve or guarantee the bandwidth for your VOIP.

thanks.

That's exactly what I said.

Thanks.

Ideally we want to be able to apply traffic shaping rules ONTO different types of web traffic.

To do this would require a layer7 filter, which has unfortunately been removed from 2.3 (it was removed because it wasn't working correctly in 2.2.x) I have already submitted a feature request for a replacement option - nDPI from ntop - https://redmine.pfsense.org/issues/5813 - maybe add your vote to it

@nightanole:

Not to hijack a thread, but upgrading to 2.3 ended up blocking gdrive from connecting.  Connected directly to the cable modem works great.  It wont connect at all behind pfsense, dropbox and onedrive are not affected.  Im running a very min setup for psfsense since im a beginner.  Its setup like a consumer router from one cable modem, and the only thing else im running is a CODELQ traffic shaper to cut down on buffer bloat.

Use tcpdump or logs to confirm that pfSense is blocking. Unless you have a specific rule blocking outgoing packets, the packets should be passed and NATted like anything else.

Thank you for the fast answer. Sadly this rule does not work, neither a rule having set the private address of the host as source in an equivalent floating rule.
I am by far no professional but for my understanding this cannot work as the WAN rules are applied after (for traffic originating in the LAN) NATing took place, hence simply match private LAN IP addresses on the WAN interfaces does not work. Feel free to correct me if my understandings are wrong.
Of course I would appreciate any other hints.

Wouldn't it be an option to apply queuing on LAN interface in IN direction? At least it seem to work and sends the traffic to the correct queue, when i setup a floating rule on LAN interface in IN direction matching source IP address as the host i want to prioritize by its IP address.
I often read things like "only create floating rules for the WAN interfaces for traffic shaping". Are there any side effects?

sMau

If you want to limit interface bandwidth, then do traffic shaping, not bandwidth limiting.

Any help guys?

Are you certain it only happens there? From the error it sounds like the disk can't be written for one reason or another.

Is this a full install? NanoBSD? Is the disk read-only for some reason?

moikerz,

Thank you - it took me until tonight to be able to re-test with your suggestions. I tested after each of the following steps:

Setting the bandwidth in Kbps - PRIQ 50000 down 5000 up; VOIP 1000 down 1000 up. Tested.

Reset state table. Tested.

Unchecked Explicity Congestion Notification on all queues, then reset state table. Tested.

In each case the bandwidth tests through speedtest.net show between 8Mbps down and 4+ Mbps up.

Here is the output from pftop:

pfTop: Up Queue 1-6/6, View: queue, Cache: 10000                        21:16:42 QUEUE              BW SCH  PR  PKTS BYTES DROP_P DROP_B QLEN BORR SUSP P/S  B/S qACK                  priq  6    0    0      0      0    0            0    0 qDefault              priq  3 87069  33M    126 133488    0            69 8168 qVoIP                  priq  7    28 17724      0      0    0            0    0 qLink                  priq  2  168K  81M    184 196856    0          242  94K qACK                  priq  6    0    0      0      0    0            0    0 qVoIP                  priq  7    56 29484      0      0    0            0    0

I then ran through the shaper with the same settings as above, but enabling the "Raise or lower other Applications" page and giving Higher Priority to Http/Https and RDP traffic. With or without "Explicit Congestion Notification" checked we still get 8Mbps download and 4.3 Mbps upload. But we do see some traffic in teh qACK queues. Here is the pftop output at the end:

pfTop: Up Queue 1-10/10, View: queue, Cache: 10000                      21:30:32 QUEUE              BW SCH  PR  PKTS BYTES DROP_P DROP_B QLEN BORR SUSP P/S  B/S qACK                  priq  6  2187  120K      0      0    0            37 2312 qDefault              priq  3 24690  11M    60  85001    0          351 388K qVoIP                  priq  7    8  5064      0      0    0            0    0 qOthersHigh            priq  4  2232  976K      0      0    0            23 8680 qOthersLow            priq  2    0    0      0      0    0            0    0 qLink                  priq  2 52395  27M    136 194579    0          379  72K qACK                  priq  6  1975  111K      0      0    0            19 1168 qVoIP                  priq  7    16  8424      0      0    0            0    0 qOthersHigh            priq  4  2519 2052K      0      0    0            54  64K qOthersLow            priq  3    0    0      0      0    0            0    0

I'm attaching a screen clip of the floating rules (could not figure out how to filter from "pfctl -vvsr").

Any other ideas?

pfsense_floating_rules.jpg
pfsense_floating_rules.jpg_thumb

I take the whitelist approach instead of the blacklist. What I mean is I identify the traffic I want to by normal or high priority and all unknown traffic goes to low/idle priority. I use the term priority loosely because I use HFSC and all it really means is more or less bandwidth.

Don't forget to enable CoDel on your child disciplines. At some point in the far future, PFSense should get Cake, once it's done and ported. Then you will probably never need to configure priorities again.