Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense®?

w0w

Looks like Netflix improved their code to production ready state, what should be done to implement this code into pfSense? This should be very helpful in certain scenarios like VPN channels etc...
https://cgit.freebsd.org/src/commit/?id=5d8fd932e418f03e98b3469c4088a36f0ef34ffe

Sergei_Shablovsky

@w0w said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

Looks like Netflix improved their code to production ready state, what should be done to implement this code into pfSense? This should be very helpful in certain scenarios like VPN channels etc...
https://cgit.freebsd.org/src/commit/?id=5d8fd932e418f03e98b3469c4088a36f0ef34ffe

Vote for this by both hands!

w0w

https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/

Available only on the Freebsd 13 and additional options needed when compiling the kernel. I am pretty sure that Netgate uses the Freebsd 12.3 and I have seen somewhere that there are no plans currently to change the version.

Sergei_Shablovsky

@w0w said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/

Thank You for news!

Available only on the Freebsd 13 and additional options needed when compiling the kernel. I am pretty sure that Netgate uses the Freebsd 12.3 and I have seen somewhere that there are no plans currently to change the version.

Recently I asking here on forum about pfSense shift to the FreeBSD 13, but no any great news from NetGate.

Really, not good news.

Because pfSense looks outdated on FreeBSD 12.X due most end-users traffic today (80%+ in US/EU and around 90%+ in Asia) are classified as “with big latency, packet loss” where BBR2/QUIC are much better CC solution that all we have before.

w0w

@sergei_shablovsky
RACK and BBR will mostly have an effect running on endpoints, like streaming servers or tunnel endpoints. Since pfSense is a firewall there are not so many situations when BBR or RACK will give any benefit, but we need more test to be done, before making statements and conclusions.

Sergei_Shablovsky

@w0w said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

@sergei_shablovsky
RACK and BBR will mostly have an effect running on endpoints, like streaming servers or tunnel endpoints. Since pfSense is a firewall there are not so many situations when BBR or RACK will give any benefit, but we need more test to be done, before making statements and conclusions.

Agree with You more than 99,999% ;)

Last 0,001% is in question: what are behind the pfSense?

Databases (and with pictures also);
Audio/Video streams (because post-COVID era push a lot of employees work from home, and we make a lot of video calls, not audio-only calls, add video surveillance here for home or for government needs);
Real-time data streams from equipment (wide range from medical to industrial and energy);
Scientific distributed calculating;
AI and machine learning distributed systems;
....

We quickly jump in a world where internet are equal to stream, where even static web page no more exist anymore.

P.S. As example, even Your streaming server use BBR2, with TCP/UDP stack tuned exactly for delivered content, You need to make appropriate tuning the pfSense. Otherwise - Your server tuning not working at all.
Agree?

w0w

TCP congestion control is managed by endpoints (sever and/or client e.g. web browser and web server), so anything not placed on the firewall is not using cognestion control, like newreno or any other.
Endpoint means that firewall iself is an endpoint, then congestion control is applied, otherwise all other traffic is just passed to upstream/downstream interface. For example any congestion controll will work when you have configured VPN on the pfSense.
Since RACK is a TCP STACK and not just congestion control, I am not sure if those statements above are applied in that case.

Sergei_Shablovsky

@w0w said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

TCP congestion control is managed by endpoints (sever and/or client e.g. web browser and web server), so anything not placed on the firewall is not using cognestion control, like newreno or any other.
Endpoint means that firewall iself is an endpoint, then congestion control is applied, otherwise all other traffic is just passed to upstream/downstream interface. For example any congestion controll will work when you have configured VPN on the pfSense.

Thank You for great explanation!

BTW, most of pfSense / TNSR installations (no matter metal or cloud) work as main gateway-firewall nowadays. So, for 80%+ of usecases nowadays, when due pandemic limitations/restrictions most of stuff working remotely thru VPN (a lot of video calls, a lot of screen sharing, access to databases, etc.) ** congestion control always positively impact on work**.
Isn’t?

Since RACK is a TCP STACK and not just congestion control, I am not sure if those statements above are applied in that case.

I read a little bit more about correct realization and write here a little bit later...

BTW, which BBR2 or QUIC port You recommend to test in conjunction with pfSense (mean FreeBSD 12.2...) ?

Thank You!

w0w

@sergei_shablovsky
If your VPN tunnel is configured on the pfSense side, then yes, it should work better with congestion control.
Concerning BBR2 or QUIC, I am not sure, because the first it the TCP cognestion control part and the QUICK is user level protocol on top of TCP/UDP, so theoretically you can test both simultaneously or independently.

Sergei_Shablovsky

@w0w said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

Concerning BBR2 or QUIC, I am not sure, because the first it the TCP cognestion control part and the QUICK is user level protocol on top of TCP/UDP, so theoretically you can test both simultaneously or independently.

After reading a lot of research, test results, and discussions with pro in networking, I conclude that BBR2 is more for netflow with a media streaming, and QUIC is like “one tool for all kind of traffic”.

BTW, BBR (and BBR2) more pushed by Netflix (due they need effective netflow with less latency for their server farms), and QUIC are more pushed by Google (due they need effective netflow with less latency & big quantity of packet drops because last 8-9 years traffic goes more “mobile”).

Am I wrong?

Sergei_Shablovsky

@w0w said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

@sergei_shablovsky
If your VPN tunnel is configured on the pfSense side, then yes, it should work better with congestion control.
Concerning BBR2 or QUIC, I am not sure, because the first it the TCP cognestion control part and the QUICK is user level protocol on top of TCP/UDP, so theoretically you can test both simultaneously or independently.

And another one reason to enable QUIC on FreeBSD behind pfSense - the netflow nowadays become more, let to say, "infrastructure-oriented": this mean that as Network Architect you may choose, on what infrastructure Your network may live, - CloudFlare, Google, Akamai, Amazon....

So, as example please read about how CloudFlare support modern protocols Understanding Cloudflare HTTP/2 and HTTP/3 Support

Most of Sys Admins & Network Architects in small or middle projects using CloudFlare or Google or Amazon infrastructure

yon 0

support add this. i am using bbr2 in ubuntu now.

Sergei_Shablovsky

@w0w said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

Looks like Netflix improved their code to production ready state, what should be done to implement this code into pfSense? This should be very helpful in certain scenarios like VPN channels etc...
https://cgit.freebsd.org/src/commit/?id=5d8fd932e418f03e98b3469c4088a36f0ef34ffe

Good News!

HowTo

Anyway, only CDG CC is better than any other in pfSense / FreeBSD 12.2... (and this list THE SAME AT LEAST 8+ YEARS!!!!)

ls -l /boot/kernel/cc_*

/boot/kernel/cc_cdg.ko
/boot/kernel/cc_chd.ko
/boot/kernel/cc_cubic.ko
/boot/kernel/cc_dctcp.ko
/boot/kernel/cc_hd.ko
/boot/kernel/cc_htcp.ko
/boot/kernel/cc_vegas.ko

And VERY INTERESTING press release about Fujitsu work on something like BBR2/BBR/QUIC more early than Netflix/Google.

JeGr

@sergei_shablovsky said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

@w0w said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/

Thank You for news!

Available only on the Freebsd 13 and additional options needed when compiling the kernel. I am pretty sure that Netgate uses the Freebsd 12.3 and I have seen somewhere that there are no plans currently to change the version.

Recently I asking here on forum about pfSense shift to the FreeBSD 13, but no any great news from NetGate.

Really, not good news.

Because pfSense looks outdated on FreeBSD 12.X due most end-users traffic today (80%+ in US/EU and around 90%+ in Asia) are classified as “with big latency, packet loss” where BBR2/QUIC are much better CC solution that all we have before.

About that: Don't understand that nonsense about "pfSense looking old on FreeBSD 12.3". It's simply not true.

FreeBSD 12.2 and 13.0 are the current production ready/stable versions listed on the project page. So talking about "old" or "outdated" is simply false. That Negate currently is staying on 12.x with the current 2.5.x release tree is completely normal and understandable as they aim for a STABLE release. Not bleeding edge. And as we are talking about a border gateway, router, gateway device, that's a good approach. FreeBSD 13 is still young and was only released on April this year. So just about half a year of age and as a new release it wasn't even immediatly pushed to -stable but to a -current/-release state.
I don't see the sense in rushing to new releases as that always requires a complete rebasing and updateing of all components of pfSense and its base system. That doesn't work with the wish for more stable releases per year as is currently planned for pfSense plus. With 3 releases per year you aren't simply adapting a completely new base system every few weeks and can include testing for all bells and whistles.

I'm really all for new things that make sense. Hands down. But a stable and more tested release is far better than stupid running around after every new driver and release to include the latest and greatest commits there are.

Additionally it was already talked about in several blog posts, that pfSense will get FreeBSD 13 (potentially with 2.6 or 2.7 depending on when/what 2.6 will include) later on. So I see no sense in downtalking the use of a stable base OS :)

Cheers

Sergei_Shablovsky

@jegr said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

@sergei_shablovsky said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

@w0w said in Any chances to get Netflix 's Open Connect Appliance (OCA) TCP code (RACK and BBR) into pfSense?:

https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/

Thank You for news!

Available only on the Freebsd 13 and additional options needed when compiling the kernel. I am pretty sure that Netgate uses the Freebsd 12.3 and I have seen somewhere that there are no plans currently to change the version.

Recently I asking here on forum about pfSense shift to the FreeBSD 13, but no any great news from NetGate.

Really, not good news.

Because pfSense looks outdated on FreeBSD 12.X due most end-users traffic today (80%+ in US/EU and around 90%+ in Asia) are classified as “with big latency, packet loss” where BBR2/QUIC are much better CC solution that all we have before.

About that: Don't understand that nonsense about "pfSense looking old on FreeBSD 12.3". It's simply not true.

FreeBSD 12.2 and 13.0 are the current production ready/stable versions listed on the project page. So talking about "old" or "outdated" is simply false. That Negate currently is staying on 12.x with the current 2.5.x release tree is completely normal and understandable as they aim for a STABLE release. Not bleeding edge. And as we are talking about a border gateway, router, gateway device, that's a good approach. FreeBSD 13 is still young and was only released on April this year. So just about half a year of age and as a new release it wasn't even immediatly pushed to -stable but to a -current/-release state.
I don't see the sense in rushing to new releases as that always requires a complete rebasing and updateing of all components of pfSense and its base system. That doesn't work with the wish for more stable releases per year as is currently planned for pfSense plus. With 3 releases per year you aren't simply adapting a completely new base system every few weeks and can include testing for all bells and whistles.

What I say on this? ;)
My experience in IT after more than 20+ years told me, that You are absolutely right.

There is one BUT: the world running faster and faster. And this speed more and more impact on the ”STABILITY vs NEW FEATURES” balancing that we, as network engineers and SysAdmins need to keep well. And of course, this balance vary depend on network environment, client goals, and many other factors.
And the features that we was very septic about, become more valuable and more needed by our clients.
And this is no “bells and whistles”, this is the protocols and technologies that 3-5 years before not exist, but now become a key for a business.
Media streaming coming (Amazon, Netflix, etc...) - and modern and ** more effective CC** like BBR/BBR come in!
Social networking + better cameras in smartphones coming (with a lot of photos and videos) - and modern effective CC QUIC coming and stay as standard RFC protocol in near a future.

I'm really all for new things that make sense. Hands down. But a stable and more tested release is far better than stupid running around after every new driver and release to include the latest and greatest commits there are.

With this I more than agree: in security sector we need to be more traditional, more stable.

Additionally it was already talked about in several blog posts, that pfSense will get FreeBSD 13 (potentially with 2.6 or 2.7 depending on when/what 2.6 will include) later on. So I see no sense in downtalking the use of a stable base OS :)

I am not sure that coming as fast as we need: after recent issue with error in code (that mean no effective code quality check system exist in Netgate) I am not sure there are so much resources to keep both CE and TNSR versions and (as You good write) ** that always requires a complete rebasing and updateing of all components of pfSense and its base system. That doesn't work with the wish for more stable releases per year as is currently planned for pfSense plus. With 3 releases per year you aren't simply adapting a completely new base system every few weeks and can include testing for all bells and whistles.**.