Problem accessing internal webservers via external addresses



  • I have a bunch of webservers (6 to be precise) for my automation that I need to access in an easy way. Which is why I have DynDNS hosts for them. So for instance if I type www.automation.com (of course it's not the real address...) I get to 192.168.1.20:1234 (which is the IP for the pc connected to the switch from pfSense that runs all those machines, on different ports). And if I type www.automation2.com I get to 192.168.1.20:1235 (so another port, which means another webserver). I have NAT'ed all the ports, and it works flawlessly from outside (tested on my phone).

    What doesn't work is doing it from inside, not on my main net 192.168.1.x and not on the subnet for my rental flat, 10.10.11.x. I tried setting up NAT reflection, but that doesn't help. Can somebody please tell me what I'm doing wrong?

    Oh, I can't use split DNS because half of those webservers are webhops, so if I go to www.automation3.com it will go to www.automation.2.com:1235 I have to do it this way because of the way one of the programs I use is set up.


  • Rebel Alliance Global Moderator

    @mastiff said in Problem accessing internal webservers via external addresses:

    So for instance if I type www.automation.com (of course it’s not the real address…) I get to 192.168.1.20:1234

    No you don't dns has zero to do with some port.. Resolving www.automation.com would not return :1234 it would return your IP.. But if you have public dns returning rfc1918 address space 192.168.1.20 then its borked before you even get started.

    So your doing webhob on your dyn services.. Why not just run reverse proxy on pfsense sense so you do not have to use all these ports.

    So now www.automation.com returns your public IP on the internet. So you hit say 1.2.3.4 on port 80 or 443 the standard ports. Same when you resolve www.automation2.com it also returns 1.2.3.4 your public IP.

    Now your reverse proxy on pfsense says oh you want automation.com that gets sent to 10.10.11.x, oh you want to go to automation2.com you go to 10.10.11.y

    From internal you just resolve automation and automation2 to their actual rfc1918 addresses 10.10.11.x and .y or all the same box if they are doing virtual domains served up on the same machine via host headers.



  • Sorry, but you're wrong (at least the way I use it). Webhop can be set up to work like this:

    Client out: www.automation3.com (no port, so 80) --> DynDNS out: (my external IP from the ISP associated with www.automation.com) :1234 --> pfSense out: 192.168.1.20:1234

    So www.automation.com is the catch-all address for those servers, and the webhops control which port on my external IP this is sendt to. So I can't use DNSMasq or similar because that's not able to do the port magic, as far as I know. Or can it do this:

    Client out: www.automation3.com (no port, so 80) --> pfSense out: 192.168.1.20:1234


  • Rebel Alliance Global Moderator

    I know exactly what a webhop is - my point is not the correct way to do it.. But if you happy with it - have fun and good luck ;)



  • @mastiff
    Have you tried NAT reflection in "NAT + proxy mode"?



  • OK, sorry. You know what a webhop is. But is it possible in your way to do it the way I need it? I HAVE to be able to write in just www.automation3.com for it to be like this:

    Client out: www.automation3.com (no port, so 80) --> pfSense out: 192.168.1.20:1234

    I would love it to work only on the pfSense and keep it internal, believe me. I just didn't think it worked in that way. And if it doesn't, can you please tell me how to do a simple "loopback" (i know that's not the correct word, but I don't know the correct terminology) so that when I write www.something-associated-with-my-dyndnds.com it goes to dyndns and then comes back into the pfSense?

    Viragomann (riding a Honda Blackbird myself, but my son's into cruisers), I haven't. Can that change the ports?



  • @mastiff
    ☺
    It doesn't change the ports, but it makes responses go back the same way as requests.



  • Viragoman, it either chews forever (with a port number) or goes to the pfSense dashboard (without a port number).


  • Rebel Alliance Global Moderator

    Yes the correct way to do this you never have to worry about ports.. Using the reverse proxy on your wan you can send whatever fqdn to whatever httpd behind pfsense you want - all on 80 or 443. You setup your local to just use the name and go to the the local IP that site is being hosted on via its fqdn.



  • I am afraid that either I don't understand you or you don't understand my needs. So let me explain, to be sure: There are four different webservers on the same computer (VM, really, but that doesn't matter, it worked before pfSense, when I was using M0n0wall and an Asus router for what is now a pfSense box). That means that they need to be on four different ports. I have one on the standard 80 port, and the three others on non-standard ports (so 1234, 1235 and 1236 as an example).

    With DynDNS webhop I can go from the internet to the three nonstandard port webservers without using the port, because the webhop translates www.automation3.com to www.automation.com:1234 (so without the 3, but with the port). And it is absolutely necessary that this translation happens.

    On the pfSense box, the addresses www.automation.com, www.automation.com:1234, www.automation.com:1235 and www.automation.com:1236 are sendt to the same VM, with the ports intact. So when they arrive at the VM they will go to the correct webserver on that VM.

    When I do this from outside, it works. I just found out that for some reason it works on my secondary subnet as well (which goes to the rental flat), but it doesn't work on the subnet where the VM is. On that subnet the webserver with the standard HTTP port is turned into https and goes to the pfSense web interface, and any of the other webservers times out.

    Was that enough of an explanation, so maybe you can see if I am misunderstanding you or you are misunderstanding me?


  • Rebel Alliance Global Moderator

    @mastiff said in Problem accessing internal webservers via external addresses:

    That means that they need to be on four different ports.

    No they do not... I can run hundreds of domains o the same IP via host headers or virtual domains depends on what httpd your using what they call them. IIS calls them host headers while apache calls them virtualhosts.

    https://httpd.apache.org/docs/2.4/vhosts/examples.html

    Since the box is on rfc1918 you could also have all your different sites be on different IPs.

    You use the reverse proxy on pfsense to understand the fqdn to know where to send the traffic in vs just simple port forward..

    Here take a look
    https://github.com/PiBa-NL/pfsense-haproxy-package-doc/wiki

    You could also do it with the squid proxy as well.



  • But you are not using the Girder and EventGhost home automation webserver to run your domains on. I have for 10 years. :) Just trust me, they need to be on different ports for this stuff. And as I said, it works on the opt1/other subnet, the ping-pong to DynDNS and back just doesn't work on the same subnet as the VM with the webservers are on. Can you think of a reason why that is? I have used rules to exclude the optional subnet from accessing the pfSense interface, can that be the reason?


  • Rebel Alliance Global Moderator

    That you would open home automation anything to the public internet is nuts if you ask me. That it has to run on nonstandard ports is also nuts.

    You stated they are running on VM.. Then setup 6 vms and give them different rfc1918 IPs. So you do not have to deal with nonsense of webhops.

    or just use the uri with the port included.. And setup your nat reflections with the ports your using. Sounds like your trying to skin the cat with a dull rusty spoon vs a butcher knife..



  • I could give you the ip, and you couldn't get in. There are no known vulnerabilities (the webservers only does two things, and that is show a webpage to give commands and info), so it's virtually impossible to do anything without the correct username and password. So there's no danger. And I can't connect the physical hardware to more than one VM at the time, so I can't use more than one VM. Of course I could spend around 2000 dollars to get six of each of the hardware, but that would just be dumb.

    Seriously, I am fully aware that you know a LOT more than me about pfSense and networking. But I have been doing home automation for many years (I checked, and my first setup was from 1998, it turns out). And my current system runs perfectly, as long as the traffic is forwarded as it should. I have been running version of this system for 4-5 years, and this is the first time I have this problem.

    As I said, before I ran an Asus router that forwarded everything inbound to a M0n0wall firewall and split my subnet from the rental flat's subnet, and the M0n0wall sendt the stuff to the VM, with the ports. I never had any problems going from my home net with an address without a port to DynDNS, which changed the address and attached a port, and then back into the VM on my system. But I figured that I would leave M0n0wall (the father of pfSense) because it's too old, and I didn't need the Asus router when the pfSense could both split and forward. Or so I thought.

    So again, any idea why I can do my pingpong from the rental flat's subnet, but not on the subnet where the VM is?



  • @mastiff said in Problem accessing internal webservers via external addresses:

    but it doesn’t work on the subnet where the VM is. On that subnet the webserver with the standard HTTP port is turned into https and goes to the pfSense web interface, and any of the other webservers times out.

    So configure the pfSense web GUI to listen on another port than 443, also uncheck "WebGUI redirect" and activate NAT reflection with proxy mode.


  • Rebel Alliance Global Moderator

    @mastiff said in Problem accessing internal webservers via external addresses:

    There are no known vulnerabilities

    Oh that is funny!!!

    But yeah as viragomann stated your going to have to use nat reflection.. What non efficient way to so something... Here let me bounce all the way to some proxy running on the internet (webhop) so it can send my browser a redirect with the port on it. Then I can hit my actual public WAN ip on this port, to get reflected back into a box sitting on my network.. What a fantastic solution that is - vis just say using fqdn:port in your uri and having that fqdn resolve to your rfc1918 address local, and forward it on the public side.

    So this runs on windows, and its latest release is from feb of 2016?
    Latest release:

    EventGhost 0.4.1.r1722 [source], Feb 03 2016


  • For the avoidance of misunderstanding, in my opinion also the proxy solution is the better way to do that and offers more options in configuring the application servers. But Mastiff obviously want to get it work as it did for years before.


  • Rebel Alliance Global Moderator

    Many home routers nat reflect out of the box.. Anything that nat reflects is borked to be honest.. Nat reflection is just plain abomination if you ask me ;)

    Pfsense does not nat reflect out of the box - you have to purposely tell it, hey pfsense I like to do things the F'd way - let me hit you on your wan, just so you can send me back into a box right next to me.. hehehehe

    Here is my advice - if your thinking of nat reflection as a way of getting something work, your doing it wrong! ;) Back to the drawing board where you don't have to hairpin connections to get them to work.



  • Who runs on releases? The latest VERSION is 05.06.2018. And as far as I have heard nobody has managed to actually break in and do anything in the webservers of EventGhost and Girder when they didn't know the password and username. There's a difference betwenn no vulnerabilities and no KNOWN vulnerabilities. The known part means that there are nobody who has found it interesting enough to find whatever may be there. Also you need to know what IP and ports to attack, and what kind of program that's behind them too. Of course there are lots of vulnerabilities found in programs that they can actually make money on hacking! But who wants to spend the time hacking something which has few users and nothing of value to find.

    And I have asked more than once in this thread how I can do it with proxys when I need to get to the ports I am using. I have said what I need to send out (a regular web adress without a port, so the standard port) and what has to get into the VM from pfSense (another web address, and with a non-standard port). If you could tell me how to do that, I'm all ears. But so far you've only told me how I should be doing it when it isn't possible to do it this way with my programs.

    Oh, as for efficiency it doesn't matter. It's text and icons 64x64 pixels that's sendt, there is no discernable difference at all.

    Viragomann, THANK YOU!!!! I had the NAT reflection with proxy mode set, but changing the port of the webgui and disabling the "WebGUI redirect" fixed it, and I'm up and running!



  • johnpoz, synchronized posting. Like sync swimming, but without the bathing suit. At least I'm not wearing one, I have no idea what you're wearing...

    The thing is that I can only work my system this one way (a limitation in the programs, the webserver is just a tiny part of a what they do), and I don't want it to be visible to the end user. And that means no port in the URL. And the whole system has been built with so many hours, there is no way I'm changing programs to avoid something that may not be clean enough to you purists, but doesn't slow down anything. 😂



  • @johnpoz said in Problem accessing internal webservers via external addresses:

    you have to purposely tell it, hey pfsense I like to do things the F’d way

    😂

    But to be honest, I also use it for some purposes. 😉
    For me, it's the short way to reach my goal.



  • @viragomann For some reason I see with my inner eye a tall lady with strange clothing leading a naked queen down the street shouting "shame, shame"! 😈

    Oh, and I have no idea what happened, but suddenly my OpenVPN tunnel, which passes through the pfSense box to the server, stopped working! It links my cabin and house together and is an essential part of my system. Can any of the changes made here be responsible, or is it correlation and not causation?

    Edit: I've of course restarted everything, but nothing helps.



  • Activating NAT reflection and changing the the web interface port can not bring the vpn down as long as you use another port as the vpn instance.
    Don't know if you did some further changes except that.



  • I was sort of expecting that answer... ;) I guess my learning curve with pfSense is a bit steeper then I thought. I have been using M0n0wall since forever, and I thought that the fork would be quite similar. At least it's very stable...or stably annoying, mostly since I don't know what I'm doing! 😎



  • About the reverse proxy thing. Can Pound do reverse proxy and add ports? So when the address www.automation.com comes to pfSense and Pound, it will be translated to 192.168.1.20:1234? Because that is of course a better way, I just know that DNSMasq can't do port adding.



  • It sounds like it could work, here's a bit from the manual:

    BackEnd
    A back-end is a definition of a single back-end server Pound will use to reply to incoming requests. All configuration directives enclosed between BackEnd and End are specific to a single service. The following directives are available:
    
    Address address
        The address that Pound will connect to. This can be a numeric IP address, or a symbolic host name that must be resolvable at run-time. If the name cannot be resolved to a valid address, Pound will assume that it represents the path for a Unix-domain socket. This is a mandatory parameter. 
    Port port
        The port number that Pound will connect to. This is a mandatory parameter for non Unix-domain back-ends.
    


  • Why want you use different ports? Why not different IP addresses?
    You can assign multiple IPs to a single server and assign each to a specific service. So each service can listen on its default port.



  • @viragomann said in Problem accessing internal webservers via external addresses:

    Activating NAT reflection and changing the the web interface port can not bring the vpn down as long as you use another port as the vpn instance.
    Don't know if you did some further changes except that.

    Feel like an idiot... Now I found out why it didn't work. The previous setup was all ports forwarded to the server, which worked. Then I split it up into series of ports to have the home automation webservers directly accessible from pfSense, instead of via my server. At that time the tunnel was still up. I forwarded the standard OpenVPN port to the server, but forgot one thing: A few years ago I had two tunnels up and running, on different ports! And the one that I still use is on the non-standard port that I had the secondary tunnel on. 😱 But it was still working because the tunnel was already up, until I activated Nat reflection, since activating that probably triggered some kind of a reset that broke all connections. So now my tunnel's up and running again. 😁



  • @viragomann said in Problem accessing internal webservers via external addresses:

    Why want you use different ports? Why not different IP addresses?
    You can assign multiple IPs to a single server and assign each to a specific service. So each service can listen on its default port.

    Not completely sure I follow you there. I need to have everything running on the same VM. And the services are http, but it's of course not possible to have more than one http service listening on the same port on one computer, as far as I know.

    I could split up the webservers on six VM's (I thought a bit wrong about the number of servers I run, each address has two servers, so I'd need to have six VM's for this to work), but it just seems like crazy overkill when it has to be at least Windows 7 running them. I already have five VM's on my server, three of them Windows 7. So I'd have to get a more powerful server to do it. I stated earlier that I'd had to by several sets of hardware for the home automation, but I could of course set up a system with tcp commands between the VM's. Still it's very convoluted.

    But I do have one possible idea. The problem is that I don't know if this would work. Maybe some of you can tell me that? pfSense is very lightweight compared to a Windows installation, so running several of those is no problem. And I have two unused network ports on a server NIC. So maybe I could send the address www.automation1.com to the main pfSense, use NAT reflection to send that to internal pfSense 1 that does NAT wich converts www.automation1.com to 192.168.1.20:1234 and passes it on to the automation VM? And then www.automation2.com would be sendt to internal pfSense 2 that converts that to 192.168.1.20:12345, www.automation3.com would go to internal pfSense 3 that converts that to 192.168.1.20:12346 and so on.

    Can I have several pfSenses in parallel in this way, as long as I run the DHCP service on only one of them? Or wouldn't it work because the answer would be "confused" as to what pfSense it should go out on? or maybe that doesn't matter because routing out from LAN to WAN (in this case WAN is the outer LAN) always works? It would be two levels of pfSense, the outer level with one and the inner level with six.



  • 0_1530264560672_b7a0ec44-54b2-40f5-b0f1-ae0fd5e221a1-image.png


  • Rebel Alliance Global Moderator

    What kind of nonsense rabbit whole you going down??

    Do you think you could complex up something more? I have gone over the correct way to do this.. Have fun with such nonsense.



  • I've already told you several times why your way isn't possible with the necessary port setup of the home automation software I use and have several years of development sunk into. But if this is your way of saying "no, that is not possible", then please say that. If it's your way of saying "I have no idea if that's possible", then please say that too, in that case I will experiment with it. I just didn't want to experiment with something that wasn't possible.

    Edit: I actually thought that anything that M0n0wall could easily do would be just as easy to do with pfSense, but I was probably wrong.


  • Rebel Alliance Global Moderator

    Then you the ports int he url! why do you even need to use different domains... Just use the same one with the different ports on the end.. Its pointless calling out auto1 and auto2.com

    You can use whatever ports you want in your reverse proxy.. If you want to send auto1.com to ipaddress:1234 and auto2.com to ipaddress:4567 have at it.

    Hitting some outside url, to get sent to your public IP:port - and nat reflect then in if you want will work.. You just need nat reflection setup.

    be way simpler to just use the url with port in.. Then you could do split dns and not have to do any of this nonsense.



  • No, I don't (as I explained earlier) want to use ports in the URL because telling guests to tap in :1234 after the URL when they are going to access it (Airbnb guests that I rent my house out to in the summer and cover much of my mortgage with, much because it's a combination of a 1700's house with 2000's tech) is confusing them. I tried that first, that's why I started using webhops that adds the port.



  • Oh, and the auto1 and so on is just examples, so I don't put my actual addresses out here. ;) I have addresses that are directly connected related (that was ambigous when talking about networks) to the house and easy to understand (somewhat like www.whitehouse.com).



  • johnpoz, I was looking for the answer to another question (splitting into different severs from WAN, I thought that wasn't reverse proxy, but it seems it is), and I stumbled over an old post by you where you actually use the same argument as me:

    You can not run 30 http servers behind 1 public IP.  Unless you use different ports on the 1 public IP you have.
    
    public.IP.80 –> private.1.80
    public.IP.81 --> private.2.80
    public.IP.81 --> private.3.80
    
    But that method BLOWS because from the public side you would have to put the port in your url like http://www.domainA.tld:80 or http://www.domainB.tld:81, etc…
    

    Exactly like you I think putting ports in an URL BLOWS! 😂


  • Rebel Alliance Global Moderator

    If you had 30 server putting in ports blows yes.. Get 30 IPs, or use a reverse proxy where did I say use a freaking webhop and then nat reflection??

    You want to make it simple for your guest. Then send them to a bookmark page with the urls listed for them - they don't have to see the ports, they down even have to see the full urls - you have a link that says

    Light
    Fireplace
    Blinds

    Or whatever it is your wanting to control

    Other option is if you have 30 servers, then get 30 public IPs



  • Well, it was mostly the ports on the public side. And no, I can't use one address simply because it's to confusing. Three different zones in the house, and rental guests use one or the other, never the third, and never both the two first. ;) But I am going to work on my freakshow next week and see if I can get HA Proxy to work the way I'd like. I never give up until it's proven impossible. Oh, this is what my web radio page looks like. Pressing any button means that you start playing that radio channel in that zone, and the zones only have speakers, nothing more. Amps are all in the technical room. It's possible to use AirPlay and Bluetooth streaming on the same system, that's in another page. I can't really imagine how to make it more selfexplanatory. On a phone or another screen that can't show everything at once you scroll to the right, and the first column (were the room names are) follows.

    0_1530366219241_Web Radio.jpg



  • Today I had time to mess around with this, and it worked better than I thought it would. I have now converted the webhops to regular hosts, and I don't have to mess with ports at all. I have split the DynDNS hosts into auto.com/auto2.com, anotheraddress.com/anotheraddress2.com and so on. The first is always EventGhost, and 2 is always Girder. From both outside and inside this goes to the frontend of HA proxy on my pfSense box, 192.68.1.1 (from the inside I think it's going directly, but I haven't yet pulled the plug on the Internet connection to verify that). There the pages are split according to name (host matches auto.com and so on), into one of six pfSense VM's with one NIC in (from the 1.1 pfSense) and one pure host NIC (VMWare's virtual network cards that don't need a physical card) to 192.168.1.50-56, which has the internal networks 192.168.3.x-8.x

    These virtual NICs are also connected to the automation VM, so it has now 8 NICs (the two physical for contact with the hardware, and those to the VM pfSense). Only the VM pfSenses mess with ports, converting 80 to the necessary port for the webserver instances that are running inside.

    This actually works perfectly, with much faster response than before, when I had it hairpin through one M0n0wall and one Asus router.

    Dirty? Probably. Working? Indeed. I'm satisfied. ;)


  • Rebel Alliance Global Moderator

    @mastiff said in Problem accessing internal webservers via external addresses:

    I have now converted the webhops to regular hosts, and I don’t have to mess with ports at all.

    There you go - almost there.. Why are you bouncing your internal off your ha proxy? Why not just resolve to whatever the internal IP is?