Cloud HAProxy Wireguard to pfsense Wireguard HAProxy

djwopasadjlk

The goal is to be able to host services behind a CGNAT. I've setup HAProxy on pfsense if I'm on my LAN, HAProxy is working after I remove accept-proxy in the Frontend HAProxy external listener.. Wireguard tunnel between the pfsense and the cloud vpc is also working. From the VPC I can ping and connect to local servers thru the terminal.

I'm testing on my mobile thru my phone. I've tested with Chromium, Firefox & Brave

when I try http://gitea.MYDOMAINNAME I get error

503 service unavailable
No serve is availabe to handle this request

which is expected because I'm not using port 80

when I try https://gitea.MYDOMAINNAME I get error

This site can't be reached
gitea.MYDOMAINNAME unexpectedly closed the connection
Try:
Checking the connection
ERR_CONNECTION_CLOSED

I've also tried adding DNS = 192.168.22.222 #pfsense wireguard ip to my oracle cloud VPS wireguard and then the error changed to

This site can't be reached
gitea.MYDOMAINNAME took too long to respond. 
Try:
Checking the connection
ERR_TIMED_OUT

Those are the errors I'm getting and I guess HAProxy is not working properly proxying the traffic to the local haproxy thru the wireguard tunnel . So please help :slight_smile:

Cloud

I've got Oracle clouds always free tier account and setup an ubuntu 22.04 instance there.

on my cloud virtual network I've opened ports:
- 51820/udp
  80/tcp
  443/tcp
  ssh/tcp

wireguard cloud part

in my instance I've installed wireguard

sudo -i
apt update && apt upgrade && apt install wireguard wireguard-tools resolvconf -y

created my keys
cd /etc/wireguard umask 0077; wg genkey | tee privatekey.key | wg pubkey > publickey.pub
configured my interface
vim /etc/wireguard/pfsense_lab

[Interface]
Address = 192.168.22.22/24 # oracle_wireguard tunnel IP
MTU = 1500
SaveConfig = true
PostUp = iptables -A FORWARD -i %i -j ACCEPT;
PostUp = iptables -A FORWARD -o %i -j ACCEPT;
PostUp = iptables -t nat -A POSTROUTING -o enp0s3 -j MASQUERADE
PostDown = iptables -A FORWARD -i %i -j ACCEPT;
PostDown = iptables -A FORWARD -o %i -j ACCEPT;
PostDown = iptables -t nat -D POSTROUTING -o enp0s3 -j MASQUERADE
ListenPort = 51820
PrivateKey = REDACTED # privatekey of the oracle_wireguard

[Peer]
PublicKey = 76iLP1IxfvPeGmqn0GbaS7b9i5doIrcJcUhOUoNUUhA= #pfsense wireguard
PresharedKey = REDACTED # preshared key from my pfsense wireguard
AllowedIPs = 192.168.22.0/24

enabled the wireguard tunnel in systemd
systemctl enable --now wg-quick@pfsense_lab
enabled the ports in the firewall

ufw allow 22/tcp
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow 51820/udp
ufw enable

added the ip_forward for ipv4
echo "net.ipv4.ip_forward = 1" > /etc/sysctl.d/100-ip_forward.conf

HAProxy cloud

after that I've installed haproxy
apt install haproxy -y
this is my haproxy config:
vim /etc/haproxy/haproxy.cfg

global
  log /dev/log  local0
  log /dev/log  local1 notice
  chroot /var/lib/haproxy
  stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
  stats timeout 30s
  user haproxy
  group haproxy
  daemon
  maxconn 10000

defaults
  log   global
  mode  http
  option        httplog
  option        dontlognull
  timeout connect 10m
  timeout client  10m
  timeout server  10m
  timeout http-request 10m

# I know you can use front and backend separate but I've gone with the listener and kept all in one place
listen https
  bind *:443
  mode tcp
  option tcplog
  server default 192.168.22.222:443 send-proxy

listen http
  bind *:80
  stats enable
  stats show-node
  stats uri /REDACTED #uri for the stats to be shown
  stats auth $USERNAME:$PASSWORD # set your own username and password for haproxy stats
  server default 192.168.22.222:80 check

check to see if there are any problems with the configuration:
haproxy -f /etc/haproxy/haproxy.cfg -c
start and enable haproxy on boot
systemctl enable --now haproxy.service

If needed I can provide everything that I've done on the lan pfsense part also.