HAProxy Websockets - Frigate
-
Good day, all. I have spent days looking at the posts regarding websockets on HAProxy, but I cannot seem to get it to work at all. I know my HAProxy is working, because I have 8 other web sites that are working just fine. I'm trying to set up a reverse proxy to Frigate, which is a security camera monitoring package. It is accessible via web browser at ip:5000. According to the Frigate documentation, if I were using Apache as a reverse proxy, I would need the following parameters:
RewriteEngine on RewriteCond %{HTTP:Upgrade} =websocket [NC] RewriteRule /(.*) ws://frigatepi.local:5000/$1 [P,L] RewriteCond %{HTTP:Upgrade} !=websocket [NC] RewriteRule /(.*) http://frigatepi.local:5000/$1 [P,L]For whatever reason, I cannot seem to translate the documentation I have seen on websocket setup into something that works for me. I'm not sure if I'm trying to put the values in the wrong place (frontend vs. backend) or what. I would be very grateful if anyone can help shed some light. Thank you very much in advance!
For reference: pfsense+ 24.03 and haproxy 0.63_4
-
Or, if you speak nginx, here is the suggested setup:
proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $http_connection; proxy_http_version 1.1; access_log /data/logs/proxy-host-40_access.log proxy; error_log /data/logs/proxy-host-40_error.log warn; location / { proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $http_connection; proxy_http_version 1.1; } -
@phipac
I use websockets quite a bit.
Put your websocket servers into their own backend, websocket_servers
Ensure that the correct parameters are met in order to upgrade to websocket by placing these rules
For health checks I use GET and this URL:
HTTP/1.1\r\nHost:\ yourdomain.com\r\nConnection:\ Upgrade\r\nUpgrade:\ websocket\r\nSec-WebSocket-Key:\ haproxy\r\nSec-WebSocket-Version:\ 13\r\nSec-WebSocket-Protocol:\ echo-protocolAlso on the backend add these rules to 'Backend pass thru'
timeout tunnel 1h
http-check expect status 101
timeout http-request 10s
timeout http-keep-alive 2s
timeout queue 5s
timeout server-fin 1sOn your frontend dedicated https offloading set these 2 rules for the connection upgrading to websocket.
with ACTION:
hdr_connection_upgrade hdr_upgrade_websocket
USE backend websocket_servers. -
@davecummins Dave this is great information, but I can't see the full config value of the ACL's. Could you post the haproxy config with all of it included, I'm not familiar with some of these haproxy commands. Thanks in advance
-
@rpm5099 my config contains a lot of frontends and backends and sensitive information but here is the backend websocket info which you can't see from the screen shots. Hope this helps.
backend webapp_websockets_app_ipvANY
mode http
id 119
log global
http-response replace-header Set-Cookie "^((?:(?!; [Ss]ecure\b).)*)$" "\1; secure" if { ssl_fc }
http-check send meth GET uri /wssapp ver HTTP/1.1\r\nHost:\ mydomain.com\r\nConnection:\ Upgrade\r\nUpgrade:\ websocket\r\nSec-WebSocket-Key:\ haproxy\r\nSec-WebSocket-Version:\ 13\r\nSec-WebSocket-Protocol:\ echo-protocol
balance roundrobin
timeout connect 5000
timeout server 50000
retries 1
load-server-state-from-file global
option httpchk
timeout tunnel 1h
http-check expect status 101
timeout http-request 10s
timeout http-keep-alive 2s
timeout queue 5s
timeout server-fin 1s
acl hdr_websocket_key hdr_cnt(Sec-WebSocket-Key) eq 1
acl hdr_websocket_version hdr_cnt(Sec-WebSocket-Version) eq 1
acl hdr_connection_upgrade hdr(Connection) -i upgrade
acl hdr_upgrade_websocket hdr(Upgrade) -i websocket
acl ws_valid_protocol hdr(Sec-WebSocket-Protocol) echo-protocol
http-request deny deny_status 503 if !hdr_connection_upgrade !hdr_upgrade_websocket !hdr_websocket_version !hdr_websocket_key
server websocketServer-37 172.18.80.237:443 id 112 ssl check inter 5000 weight 10 verify none
server websocketServer-36 172.18.80.236:443 id 120 ssl check inter 5000 weight 10 verify none -
This is very helpful.
Since you seem very knowlegeable, may I please ask for your advice as well:
My situation is that I have a substantial number of applications hosted in my home lab that are proxied through pfSense's integrated HAProxy.
All of a sudden, some two or three of them require websocket support.
Now I'm wondering how to implement that. Is it possible to upgrade more than one backend connection to websocket or can that be done only for one?
You seem to be suggesting to put more than one server in one backend. That way, it would seem possible to overcome a potential websocket backend limitation. But how would I distinguish between the individual apps within one backend?
Thank you very much in advance.
-
@sensewolf
If you really have a LOT of domains you probably need to do some custom HAProxy scripting or other configuration that you'd want to contact the HAProxy developers for help with. For a reasonable amount, i.e. <30, this is what I do:Name Expression Value ------------------------------------------------------------------------------- is_websocket Host starts with: ws. is_websocket Custom acl: hdr(Connection) -i upgrade is_websocket Custom acl: hdr(Upgrade) -i websocket domainA_acl Host matches: domainA.yourdomain.tld domainB_acl Host matches: domainB.yourdomain.tld domainC_acl Host matches: domainC.yourdomain.tld Action Parameters Condition Acl Name -------------------------------------------------------------------------------------------------- Use Backend:domainA_backend See below is_websocket domainA_acl Use Backend:domainB_backend See below is_websocket domainB_acl Use Backend:domainC_backend See below is_websocket domainC_acl -
Thank you.
I'm still very new to the concept, so please excuse my asking naive (aka stupid) questions:
This looks to me like we look for three conditions to be fulfilled (plus the host match) and if they are fulfilled, we forward to the backend that matches the host. And if requirements are not met, we don't forward (because the Action part implies an AND operator). In other words, some backends can only be reached, if a websocket upgrade is requested.
My situation is that I have a number of hosts and some require the websocket upgrade and others don't.
Can't I just forward the connection request as is to a backend that matches the host (say, app1.example.com) and let the host deal with it, i.e. if the connection request contains a websocket upgrade request, either upgrade or not and if the connection request doesn't contain an upgrade request, don't upgrade.
Why would I filter out certain requests to certain hosts?
Sorry, if the answer is obvious... (to me it isn't).
Thank you.
-
@sensewolf if your backend server upgrades to websocket, then it will open a new tcp port for that connection. So you need to add a new backend, call it ws_servers, add the same server again but with the websocket tcp port, call it ws_version for example. This is then the backend server you reference for websocket upgrade requests coming in your front end.
-
Thank you.
So, for each of my apps that may want to upgrade a connection to websocket, I should create a second backend with the same app but a different port number, right?
And from the frontend I should forward traffic that does not request an upgrade to the normal backend and traffic that does request an upgrade to the websocket backend for the same app, right?
Okay, so far I get it (I'm hoping). That explains why I should filter the traffic for upgrade requests.
But how do I know the port number to use for the websocket backend? The docs of those apps that (may) need an upgrade do not mention another port number. In fact, I almost exclusively host docker apps and they only require one port to be forwarded into the container (for traffic; maybe another one for metrics). This suggests to me that they don't use another port.
And I do have (so far only) one app that does require websocket and for which I have set up HAProxy (with the help of some forum thread from the internet) in a way that makes it work that does not use a separate backend (or port number) but the same one. So, I believe that this one app does not open another port but uses the normal http port.
If the apps open another port, it all seems so straight forward. But what do I do, if the apps don't open another port but use the same port for websocket as they use for normal http(s)?
Sorry for all the questions and thank you!
-
@sensewolf All of the applications I have that run websockets use the same port, so I do not create separate backends for them and it works fine. If your application does use separate ports then you will need to create a separate backend. I'm not sure how common this is, but I have I think 5 domains with websockets and none of them use a separate port.