PfSense and traefik on TrueNAS Scale
-
hey guys. tired of ripping my hair out trying to get this system to just play nice (i was fighting with HAProxy for the longest time and just could not get SSL to work correctly)
Anyway, now im fighting trying to get PfSense and traefik to play nicey nice.
I dont know if i am screwing up the NAT or port forwarding rules or if its something on TrueNAS side causing this frustration. (See attached SS of current NAT settings, this is just the latest of a long string of attempts at getting this working, and i need a set of unstrained eyes looking at it).I want all traffic coming in as SSL to be sent to traefik to be split out and sent where its supposed to go (im hoping i can do the same in traefik as HAProxy would let me do and segment it out based on the subdomain being used (cloud.mydomain.org goes to nextcloud, etc) FYI: traefik secure listens on port 9443 and TrueNAS's IP is 192.168.2.2
Any help is MUCH appreciated
-
M menethoran referenced this topic on
-
@menethoran Yeah like I was saying in that older thread.
So you have a bunch of stuff on this 2.2 box that is running stuff all on 443 and you want to send it to specific dockers based on the name?
The way you have that setup is all the traffic would get sent to that 2.2 box if came in on 9443.. So like https://something.domain.tld:9443
From there you could have proxy listening on 9443 do something with that traffic. But what are you dockers actually listening on? different IPs, the docker host IP?
Is your plex also a docker?
Happy to help if I can.. I currently have only 1 docker exposed via haproxy. I do ssl offloading on haproxy, and using strict sni filtering. I also leverage my public IP 443 for a openvpn instance, etc.
So if you come in on https://host.domain.tld that resolves to my pfsense public IP the openvpn instance says hey that not vpn traffic and sends it to 9443 on haproxy frontend on loopback, this looks at the sni and sends it to my backend in the clear 192.168.9.10:5055
All the ssl is handled by haproxy via a acme cert.
If you come in with with just IP or some other fqdn, then your just not going anywhere..
I also run plex on this IP, but this is just typical port forward from external port (different than plex default) to my plex port on 192.168.9.10 on 32400..
Happy to help if I can if you give some details of your docker setup, etc. And what exactly do you want to happen when, etc.
-
@johnpoz ok, so definitely have my rules messed up.
I want all traffic coming in on 443, hitting the PfSense box then forwarded to 9443 on 192.168.2.2 (where all my Dockers are hosted).Once it hits 192.168.2.2:9443 traffic will take over and delegate where it's supposed to go (somehow. I assume magic or their little gerbil guy just forwards it, I don't know. Trawfik is supposed to be this ultra-easy system to use, but frankly, I find it's simplicity and lack of ability to let me pick and choose things to be more difficult, not less. Personally. I would prefer to stick with an all in 1 firewall and traffic shaper (ie: pfSense and HAProxy) but it seems that, at least for now, there isn't enough support for what I'm trying to do with the setup I'm using for someone else to have done it and for me to learn from... And having my services continue to be down is getting... Disruptive :) )
Anyway, yes, the 192.168.2.2 box is my TrueNAS Scale system which runs apps as Dockers (KVMs to be more precise). The closest I could ever come to getting the ssl traffic pointing at that 192.168.2.2 machine was to hit the UI for the NAS, and no further. From what I gather, it's the way that TrueNAS Scale handles its own reverse proxies for the applications.
Anyway, all of that said, the simple explanation is that I want all SSL traffic coming into my network to be forwarded to 192.168.2.2:9443
-
@menethoran said in PfSense and traefik on TrueNAS Scale:
I want all SSL traffic coming into my network to be forwarded to 192.168.2.2:9443
Then create your port forward for traffic hitting your wan address on 443 to be forwarded to 9443 at 192.168.2.2.
Looking at your port forwards this time vs just the wan rules - this is correct
If not working, that is on whatever you have setup on that 2.2 box..
-
@johnpoz are my wan rules still incorrect (as above)?
-
@menethoran no those look fine..
-
@johnpoz so, the fact that this doesn't work (doesn't get me hitting 'traefik', which resides at 193.168.2.2:9443 has nothing to do with PfSense or my settings... Correct? (Just making sure I cover my bases, so when I go and tell truecharts their things are broken, they can't try to blame PfSense).
-
@menethoran Not that I can see from that port forward.
I would test say going to can you see me .org and hitting port 443 on your wan IP. While sniffing, do you see that traffic?
Then doing the test again sniff on yoru lan side interface going to 9443 do you see it send on the traffic?
Keep in mind you really need to test from outside.. Testing from something on your 192.168.2 network hitting your wan IP could be problematic, have you setup nat reflection. Where are you testing from?
Keep in mind pfsense can not forward what it doesn't see.. I see hits on your plex wan rule, see that 40MB, but see no hits on the rule for your 9443 forward.
