Okay I ran out of ideas so I grabbed a backup of the config file from before I installed HAProxy and ACME and restored it.
Access is again granted to port 443. I will have to assume it was HAProxy but I only had set it up for port 80 and it was working. I was starting to work on 443 but everything for those backends and frontend was disabled. Also I completely disabled HAProxy and no difference.
Oh well. I will just start again with ACME and HAProxy and see what happens