@ironwood Ok, I found the solution, or rather, ChatGPT found the solution. Under System > Advanced > Admin Access, there is a setting called WebGUI Login Redirect. This is the description:
When this is unchecked, access to the webConfigurator is always permitted even on port 80, regardless of the listening port configured. Check this box to disable this automatically added redirect rule.
The redirect is enabled for port 80 by default and was conflicting with the http to https redirect I had set up in HAproxy a long time ago. I check the box to disable it, saved, enabled my redirect and voila, it works!
I'm guessing this was either a new feature in 23.09.1 or it I had it checked before and it "unchecked" itself? Would be interested in finding if that setting exists in earlier versions if anyone hasn't upgraded.