SG-1100 and Bridge LAN/OPT w limiter

[SmokinMoJoe]

Hi All,

This firewall is used on an internal network with a pc plugged into the OPT port and an upstream switch on the LAN port. The PC gets a DHCP address from somewhere beyond the switch the pfSense fireall is plugged into. Traffic pases fine.

I have no ip addresses on the LAN, OPT or BRIDGE interfaces. I am configuring the firewall from the WAN side.

My issue is that I cannot seem to get limiters to work right. On a bridge do I put the down limirter on one side and the up limiter on the other? Can I put both limters on the bridge interface? Can I use both the up and down limtier on the same firewal rule? Can I make it so all IPv4 and IPv6 traffic get limted and not just the workstation ip or the test server ip?

I would like to run iftop once I get the developer using this setup. The developer will be testing WAN latecy, packet loss and a slow WAN conenction via dummy net.

Thanks,
Joe

[SmokinMoJoe]

Any suggestions?

Well I have it working but traffic in the both directions is terrible. With no limiter I get 400/400 across this bridge interface and I tried with TSO and LRO enabled and it had no effect, still about 400/400.

lan(em0) - connected to upstream switch
opt(em1) - connected to test pc

firewall rule LAB1(em1) Destination IP 192.168.200.29(internal speed test server)
with limiters on in(DownQ=50Mbit/s - out(UpQ=100Mbit/s)

LimiterDown 50/CoDel/FQ_CODEL/ECN=yes
LimiterDownQ CoDel/ECN=yes
LimiterUp 100/CoDel/FQ_CODEL/ECN=yes
LimiterUpQ CoDel/ECN=yes

From the windows 10 cleint I get 19Mbit/s download and 3Mbit/s upload

I get very similar/identical results with a SG-1100 or Intel Atom D510, so I think it is a bug or I am doing something wrong.

Thanks,
Joe

[SmokinMoJoe]

here is a snapshot showing the traffic on the upload going to the WAN, yes I have NAT disabled :
login-to-view

[SmokinMoJoe]

So my WAN is now on a dedicated network/subnet and I no longer have the issue where traffic goes out the WAN interface.

I do still have an issue with the limiting doing a bad job and I can only put the limiter on the interface that the PC is on, if I put it on the other interface it does not work and firewall rules show no traffic going to that NIC.

I now have no WAN and an ip on the bridge interface to manage the firewall and still the poor trafic. This is the limiter at 50/5 using OpenSpeedtest :

login-to-view

[SmokinMoJoe]

here is my limiter info from the diag page :

Limiters:
00001: 5.000 Mbit/s 0 ms burst 0
q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0
GRED w_q 1.000000 min_th 0 max_th 1 max_p 1.000000 (ecn)
sched 65537 type FIFO flags 0x0 0 buckets 0 active
00002: 50.000 Mbit/s 0 ms burst 0
q131074 50 sl. 0 flows (1 buckets) sched 65538 weight 0 lmax 0 pri 0
GRED w_q 1.000000 min_th 0 max_th 1 max_p 1.000000 (ecn)
sched 65538 type FIFO flags 0x0 0 buckets 0 active

Schedulers:
00001: 5.000 Mbit/s 0 ms burst 0
sched 1 type WF2Q+ flags 0x0 0 buckets 0 active
Children flowsets: 1
00002: 50.000 Mbit/s 0 ms burst 0
sched 2 type WF2Q+ flags 0x0 0 buckets 0 active
Children flowsets: 2

Queues:
q00001 50 sl. 0 flows (1 buckets) sched 1 weight 1 lmax 0 pri 0
GRED w_q 1.000000 min_th 0 max_th 1 max_p 1.000000 (ecn)
q00002 50 sl. 0 flows (1 buckets) sched 2 weight 1 lmax 0 pri 0
GRED w_q 1.000000 min_th 0 max_th 1 max_p 1.000000 (ecn)

[SmokinMoJoe]

Here is Iperf3 from a windows 10 client that should be 75/25 and then from the pfSense box itself to the TrueNAS Core 12.0-u3 box.

Server listening on 5201

Accepted connection from 10.39.151.100, port 57344
[ 5] local 10.39.151.29 port 5201 connected to 10.39.151.100 port 57345
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 294 KBytes 2.41 Mbits/sec
[ 5] 1.00-2.00 sec 354 KBytes 2.90 Mbits/sec
[ 5] 2.00-3.00 sec 431 KBytes 3.52 Mbits/sec
[ 5] 3.00-4.00 sec 416 KBytes 3.41 Mbits/sec
[ 5] 4.00-5.00 sec 483 KBytes 3.96 Mbits/sec
[ 5] 5.00-6.00 sec 368 KBytes 3.01 Mbits/sec
[ 5] 6.00-7.00 sec 463 KBytes 3.80 Mbits/sec
[ 5] 7.00-8.00 sec 422 KBytes 3.45 Mbits/sec
[ 5] 8.00-9.00 sec 84.1 KBytes 690 Kbits/sec
[ 5] 9.00-10.00 sec 180 KBytes 1.47 Mbits/sec

---

[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 3.41 MBytes 2.86 Mbits/sec receiver

Server listening on 5201

Accepted connection from 10.39.151.100, port 57346
[ 5] local 10.39.151.29 port 5201 connected to 10.39.151.100 port 57347
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.55 MBytes 13.0 Mbits/sec 207 5.70 KBytes
[ 5] 1.00-2.00 sec 2.90 MBytes 24.3 Mbits/sec 386 5.70 KBytes
[ 5] 2.00-3.00 sec 2.90 MBytes 24.3 Mbits/sec 382 4.28 KBytes
[ 5] 3.00-4.00 sec 2.90 MBytes 24.3 Mbits/sec 385 5.70 KBytes
[ 5] 4.00-5.00 sec 2.90 MBytes 24.3 Mbits/sec 384 5.70 KBytes
[ 5] 5.00-6.00 sec 2.90 MBytes 24.3 Mbits/sec 383 5.70 KBytes
[ 5] 6.00-7.00 sec 2.90 MBytes 24.3 Mbits/sec 383 2.85 KBytes
[ 5] 7.00-8.00 sec 2.89 MBytes 24.3 Mbits/sec 388 2.85 KBytes
[ 5] 8.00-9.00 sec 2.91 MBytes 24.4 Mbits/sec 379 5.70 KBytes
[ 5] 9.00-10.00 sec 2.90 MBytes 24.3 Mbits/sec 385 4.28 KBytes
[ 5] 10.00-10.00 sec 11.4 KBytes 23.7 Mbits/sec 1 2.85 KBytes

---

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 27.7 MBytes 23.2 Mbits/sec 3663 sender

Server listening on 5201

Accepted connection from 10.39.151.117, port 13141
[ 5] local 10.39.151.29 port 5201 connected to 10.39.151.117 port 38602
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 10.8 MBytes 90.6 Mbits/sec 0 352 KBytes
[ 5] 1.00-2.00 sec 39.6 MBytes 332 Mbits/sec 0 385 KBytes
[ 5] 2.00-3.00 sec 39.8 MBytes 334 Mbits/sec 0 385 KBytes
[ 5] 3.00-4.00 sec 39.6 MBytes 333 Mbits/sec 0 385 KBytes
[ 5] 4.00-5.00 sec 39.8 MBytes 334 Mbits/sec 0 385 KBytes
[ 5] 5.00-6.00 sec 39.7 MBytes 333 Mbits/sec 0 385 KBytes
[ 5] 6.00-7.00 sec 39.6 MBytes 332 Mbits/sec 0 385 KBytes
[ 5] 7.00-8.00 sec 39.7 MBytes 333 Mbits/sec 0 385 KBytes
[ 5] 8.00-9.00 sec 39.7 MBytes 333 Mbits/sec 0 385 KBytes
[ 5] 9.00-10.00 sec 39.8 MBytes 334 Mbits/sec 0 385 KBytes
[ 5] 10.00-10.74 sec 29.3 MBytes 333 Mbits/sec 0 385 KBytes

---

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.74 sec 397 MBytes 310 Mbits/sec 0 sender

Server listening on 5201

Accepted connection from 10.39.151.117, port 61357
[ 5] local 10.39.151.29 port 5201 connected to 10.39.151.117 port 49785
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 12.6 MBytes 106 Mbits/sec
[ 5] 1.00-2.00 sec 47.9 MBytes 402 Mbits/sec
[ 5] 2.00-3.00 sec 47.9 MBytes 402 Mbits/sec
[ 5] 3.00-4.00 sec 47.8 MBytes 401 Mbits/sec
[ 5] 4.00-5.00 sec 47.7 MBytes 400 Mbits/sec
[ 5] 5.00-6.00 sec 47.5 MBytes 399 Mbits/sec
[ 5] 6.00-7.00 sec 47.9 MBytes 402 Mbits/sec
[ 5] 7.00-8.00 sec 47.8 MBytes 401 Mbits/sec
[ 5] 8.00-9.00 sec 48.1 MBytes 403 Mbits/sec
[ 5] 9.00-10.00 sec 47.7 MBytes 400 Mbits/sec
[ 5] 10.00-10.73 sec 35.0 MBytes 402 Mbits/sec

---

[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.73 sec 478 MBytes 374 Mbits/sec receiver

[SmokinMoJoe]

Well I figured it out. You cannot rename a Limiter Queue. I deleted all my limiters and settings and started over, creating the upload limiter first, saving it, then creating the Queue under that limiter and saving.

I must have messed up the settings when renaming the limiter and queue.

Now my uploads and downloads are very perdictable with dummynet! This is a 100/25Mbit/s example:
login-to-view

[SmokinMoJoe]

how do I mark the ticket as closed, I created it and fixed it.