Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Websockets configuration in HAProxy

    Cache/Proxy
    8
    12
    5041
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      Peque last edited by

      Hi Forum
      I having setup a HAproxy on Pfsense - for handling our incomming request to webpages.
      One of the sites are running Qlik

      I'm struggling since the server is using websockets.
      I can find other guides on using HAProxy directly - as described Here

      I'm not running as Loadbalancing since the setup is a testsetup - for development on our Qlik server.

      But what I do not get and do not know where to configure - is these lines:

      ## routing based on Host header
        acl host_ws hdr_beg(Host) -i ws.
        use_backend bk_ws if host_ws
      ## routing based on websocket protocol header
        acl hdr_connection_upgrade hdr(Connection)  -i upgrade
        acl hdr_upgrade_websocket  hdr(Upgrade)     -i websocket
        use_backend bk_ws if hdr_connection_upgrade hdr_upgrade_websocket
        default_backend bk_web
      

      I cannot find where and how to create these Actions since i cannot se the option for how to create these settings. Since Qlik is using websockets - we get a timeout - can refresh and the page are loading - But if I press refresh it'll load the page.
      I've look in generel, frontends and backends but cannot find anywhere I can configure these lines

      Buit I would like the setup to run as smooth as possible without these timeouts

      P 1 Reply Last reply Reply Quote 0
      • P
        PiBa @Peque last edited by PiBa

        @peque
        You should be able to create the acl's like this:
        e541967d-709f-4ae1-8d4b-a9d27e12de62-image.png

        The 'hdr_beg' looks a bit different than what is generated in the config, but should have the same effect as the '-m beg'. As for the 'use-backend' actions, i guess you can figure that part out? If not ill make a little screenshot that part as well.

        clumbo 1 Reply Last reply Reply Quote 0
        • clumbo
          clumbo @PiBa last edited by

          @piba

          Thankyou

          This is just what I have been looking for I have been trying to workout how to do this with bitwarden_rs

          Any chance you could share your a more full screenshot of the backend part too?

          Thanks

          D 1 Reply Last reply Reply Quote 0
          • D
            dfir_dk @clumbo last edited by

            @clumbo Did you happen to make it work with bitwarden_rs? I'm here for the same reason struggling with this issue :)

            S 1 Reply Last reply Reply Quote 0
            • S
              sgtpepperaut @dfir_dk last edited by

              @dfir_dk said in Websockets configuration in HAProxy:

              a

              hey fellow bitwarder...looking for the same. i havent been able to get this to work :(

              since im also ssl offloading i get a WSS connection failed in the frontend. :(

              D 1 Reply Last reply Reply Quote 0
              • D
                dfir_dk @sgtpepperaut last edited by

                @sgtpepperaut
                Hey dude, I've found my solution here: https://github.com/dani-garcia/bitwarden_rs/wiki/Proxy-examples

                I've started to use "HAproxy (by @williamdes)" and it looks like this:
                82076c3b-e948-46ea-b456-72bf58581259-image.png
                Hope this helps..

                S 1 Reply Last reply Reply Quote 0
                • S
                  sgtpepperaut @dfir_dk last edited by

                  @dfir_dk

                  yes it does thanks!

                  but what I cant get my head around is why I need two different backends...

                  how do your backend configs look like? specifically the ws backend. Isnt it the same port?! or is there some magic going on ?

                  D 1 Reply Last reply Reply Quote 0
                  • D
                    dfir_dk @sgtpepperaut last edited by

                    @sgtpepperaut
                    Yes, it is built upon two different services. One that handles everything except notifications, which is handled by a different service on port 3012.

                    Take a look here:

                    https://github.com/dani-garcia/bitwarden_rs/wiki/Enabling-WebSocket-notifications

                    I have two backends that point to the same IP address but different port 80 and 3012.

                    If you do not enable notifications then you do not need the additional config :)

                    1 Reply Last reply Reply Quote 0
                    • P
                      Peque last edited by

                      Well still struggling a whole lot with this websocket and how to configure my proxy for using the Qlik server
                      As configured Backend - it looks like this:
                      Backend.PNG

                      And the frontend are configured as this
                      frontend2.PNG Frontend1.PNG

                      But are not getting through to the server at all - just getting error 503
                      But am I doing this wrong - is the custom ACL placed under the backend or frontend ( The same input are available both places) - and since this is the first I have a site using websockets - so not sure if this is configured totally wrong ?
                      Please advise

                      P

                      senseivita mthx 2 Replies Last reply Reply Quote 0
                      • senseivita
                        senseivita @Peque last edited by

                        @peque if you haven't fixed this, I think I might be able to help. — or maybe help someody else. :)

                        According to Websockets Load Balancing with HAProxy all the connections can be made in HTTP mode just fine but when the 101 HTTP status code is passed the connection is "upgraded"* and from then on it's not processed anymore, much like TCP or SNI traffic. So it needs to be setup like that but at the same time offloading what needs a new certificate (I assume this is the reason). Leave your old config as is and first find what exactly needs to be passed through.

                        Find the switchover point

                        Connect directly to your backend using your browser:

                        • Modify it and your hostsfile/DNS temporarily so it thinks you're connecting through a proxy if need be
                          • Remember to clear system's local DNS caches and your browsers' caches. Safari is the easiest one for this (⌘⌥e). Overall it's really annoying. As a refresher:
                            • Windows: ipconfig /flushdns
                            • macOS: sudo killall mDNSResponder mDNSResponderHelper
                            • Linux: JK--You don't need my help. systemd-resolve --flush-caches maybe?
                        • Open a new browser window in private session mode
                        • If it loads anything automatically, go to "about:blank" no quotes, no scheme (i.e: http:// https:// ws://…)
                        • Open the dev tools and go to the Network tab:
                          • Firefox has a direct shortcut: ⌘⌥e () ⌃⇧e (Windows)
                          • Chromium-based/Safari: ⌘⌥i () ⌃⇧i (Windows) to open the dev tools, then click on Network.
                            • Safari needs to have the Developer menu enabled first: ⌘, then Advanced then check Show Develop menu in menu bar
                        • If for some reason it's not blank, look for the icon that clears it. A little trash icon or something being crossed over…
                        • Look for the icon to disable the caches and enable it.
                        • Look for the option to persist the logs and enable it (in FF it's in the little ⚙︎ icon)
                        • Load your site and scroll to the top of the entries and select it to view the headers, use the arrow keys to scroll down the list until you find the 101.

                        Screen Shot 2022-02-26 at 08.15.31.png

                        Divert the right traffic to the right place

                        As other mentioned earlier, your app is serving content from more than one server; if it doesn't use a another port for the other traffic, an internal proxy must be redirecting the websockets connection based on the URL's path.

                        Duplicate your existing backend for that server and on the server section uncheck the SSL box. Give it a new new (it will have "-copy" at the end if you duplicated it) by appending/prepending something like SNI or websockets, whatever matches your naming scheme if you have one.

                        With the information you learned using your browser, edit your frontend to add the rules (as separate rules, not replacing) you found before (or use the link tot the article in this post at the beginning) so it matches the traffic to be redirected to the new backend which is exactly as the one you had before except without the SSL checkbox –– because HAProxy won't process any of it, it'll only pass it through.

                        Position the rules and double check it "stuck"

                        According to the rule itself slide it into position. Let me elaborate:
                        Let's assume your app uses pathnames to separate the traffic, it's reachable at [https://…]/app and you found out the websocket connection is at the root /. Since you need to offload the /app connection the rule to divert the traffic for the / connection should then go after /app's because it's broader/less specific. Since you'd be appending preexisting frontends, it should be alright to just add it at the end.

                        Because HTTP is also served from the backend, it's more likely that the websocket connection it's served at a deeper path level though, so using the previous example /app continues to be your HTTP traffic and the websocket connection is served at /app/ws. In this case you would need to move the rule above /app's. (click its check box and click on the little anvil/anchor icon of the other rule).

                        ✔︎, ✔︎, ✔︎ but still doesn't work? Double check your saved config

                        Recently I moved back to the built-in HAProxy on pfSense. I had moved out to a dedicated machine where I was forced to learn all of this and then some. Setting up the health checks on pfSense I noticed some wouldn't pass regardless they were configured correctly. I knew this I just went though hell learning about it.

                        Then one time when I hovered the cursor on the server name on the stats page the little tooltip (or whatever it's called) appeared with the server's IP address and the port, both of which weren't what I entered in the configuration. Adding new servers to the backend then deleting the older ones didn't work either. I needed to add a completely different backend then switch the frontend to this new backend and only then delete the broken backend. The config sometimes just won't stick, I have noticed this happens all over pfSense so you have to "nudge" it into working. It's not a big deal but it's not obvious either so it can go unnoticed long enough for you to screw up other parts of the system thinking they're at fault.

                        Strangely enough now that I have all greens the tooltip won't appear in any of the backends… That's another story though.

                        *: I use quotes because everywhere else in the HAProxy docs to upgrade a connection is the opposite, e.g; TCP → HTTP → H2 → QUIC. Here you're going backwards TCP ← HTTP/X.

                        1 Reply Last reply Reply Quote 0
                        • mthx
                          mthx @Peque last edited by

                          @peque Got it working with the Help of this Guide ( https://github.com/dani-garcia/bitwarden_rs/wiki/Enabling-WebSocket-notifications ) , but wanted to say that i removed ACL1 & ACL 2 cause this guide assumes bitwarden is the only thing running on it and will lead to every Domain configured in Haproxy loading Bitwarden. since i have allready configured Bitwarden without WS and had it working, i left that backend as is and took those first 2 ACLs away

                          1 Reply Last reply Reply Quote 0
                          • M
                            mvern last edited by

                            Thank you for this! Got my application to work. Much appreciated.

                            1 Reply Last reply Reply Quote 1
                            • First post
                              Last post