pfSense, VLANs, HAProxy, Web/Email/Cloud Servers, OpenVPN, Public VPS, WireGuard Tunnel
-
Hey All,
I'm not sure if this is the right place for this post, but I figured it's the best place to start.
So, I've been working on a solution for hosting a private, secure email system on my hardware in a SOHO setup behind a dynamic IP. Thanks to a relatively stable IP address from my ISP, I have been routing all internet traffic through my pfSense box to the server VLAN via the HAProxy package. All works GREAT! HAProxy uses ACL/SNI exact match rules to pass HTTP/HTTPS traffic as intended. For remote access, I use an OpenVPN "Road Warrior" VPN server also configured in pfSense.
To get around the dynamic IP issue for hosting email servers, I secured a cheap public VPS with a static IP, for which I control the PTR record. The VPS is running HAProxy and WireGuard Server on Ubuntu 20.04. The local email server runs the WireGuard client, tunneling all traffic through the public VPS. HAProxy is proxying all IMAP/POP/SMTP/HTTP/HTTPS traffic, and all is well from an operational perspective.
My problem is now accessing any host connected to the WG server via SSH from within my LAN. When I start the WG client service, it connects to the VPS and disconnects any LAN access. As soon as I stop the client (it's an ESXi VM, so I still have console access), I can reaccess it via the local IP. If I want to log into a client via SSH while connected to the WG server, I have to first log into the VPS, and then I can SSH into the email server or other client via the static IP configured in WG.
I'm sure I'm missing a stupid IP Tables/NAT rule or something, but I can't figure out how to regain connectivity to the WG clients from within my LAN. Any assistance would be greatly appreciated!
-
@pits_king Hi there. I find it amazing that you were able to accomplish that much. Have you written a blog post or tutorial for the newbies like me to follow and check out?
-
@djwopasadjlk, I don't consider myself much more than a newb either! LOL
Unfortunately, no, I do not have a blog. On the plus side, there are countless blogs out there to cover all of what I've done and more!
For my pfSense setup, nguvu and Lawrence Systems taught me a lot! You can find a good baseline guide here!
LinuxBabe.com is an excellent source for information. This guide paved the way to self-host an email server and get around a dynamic IP and all the restrictions associated with it.
Don't be afraid to play around. I've broken a lot more than I've fixed/got running! If you can, pick up a used server. You can find them for pennies on the dollar. Typically, if you swap in some new hard drives, you'll get several solid years out of them, depending on how old they are. Turn it into a virtual host (VMware/Proxmox VE/KVM/Zen/etc.). Snapshots are your friend!!!
Oh, and for the record, I was able to get everything working. Now, instead of using HAProxy in pfSense and routing traffic through CloudFlare to my residential dynamic IP, all internet traffic comes through my public VPS with HAproxy and WireGuard, back to each local VM. The trick to making everything reachable was found in PolicyRouting.
I hope this helps you on your journey!