Port Forwarding ESXi VM no joy !
-
I have successfully done eight or so Port Forwards... the concept is clear in my mind. However in this case I am unable to uderstand what is wrong. Let me give some history:
Existing DL380g7 running ESXi 6.7 I can forward a Linux Mint VM used as a web server to my PUBLIC IP address.
Wishing to upgrade my equipment to later version. I configured a DL360p Gen8 server running ESXi HPE version. I successfully created a BSD server and it works on the LAN IP.
However when I create a Port Forward of that LAN IP and PORT to the PUBLIC IP, it fails to work.
- Obviously there is nothing wrong with the BSD server
- ESXi is working properly as it puts the web content on the LAN
- pfSense is working properly as it has performed on eight other Port Forward configurations.
I am at a loss for the answer. I have even loaded ESXi ver 6.0 which was a disaster... my datastore was not found, and indicated "depreciated".
So... hopefully someone will know what this is all about. I have burned 40+ hours on this and still "no joy"There are NO firewalls set in ESXi
There are NO firwalls set in BSD
The gateway address is correct
The routing is correct pfSense reports the routing as ?.?.?.0 routed to ?.?.?.0:30 which is correct. As well everything else works flawlessly.Have given some thought to creating a new "port group" in ESXi. But that is my last possible answer.
-
pfSense is not running in ESXi here? It's upstream of that?
The server in ESXi is using an IP in the pfSense LAN subnet directly? Is it statically configured?
Do you see it in the ARP table in pfSense?@dhenzler said in Port Forwarding ESXi VM no joy !:
The routing is correct pfSense reports the routing as ?.?.?.0 routed to ?.?.?.0:30 which is correct
Where are you seeing that exactly? Those IPs are unusual (0 last octet) and the destination as port 30 is also...odd.
Try to connect to the server from something external and then check the state table in pfSense. You should see the incoming connection create states on WAN and LAN.
Steve
-
The WAN IP's are what was assigned by my ISP. Including the /30
The ESXi VM works but is only visible on the LAN. All of the other Port Forward rules I've created are working... so I must not be a total idiot !@stephenw10
ssh -vvv root@192.168.15.159 <== the web-server ipThe authenticity of host '192.168.15.159 (192.168.15.159)' can't be established.
ECDSA key fingerprint is SHA256:97vKVNNToueigzZMO7dYK8AGTWdVcontohO1B7kszZo.
I've tried ESXi on a DL-380eGen8 and it does the same thing. Beginning to think the issue is VMware related... But how does pfSense know that the IP should not be forwarded... or is the IP really empty unless viewed on the LAN?
I'm in beyond my depth...
-
Ah, OK. /30 subnets for routing is fine. :30 looks like a port which would be odd.
Ok, so let's see your port forward rules.
You are testing access using ssh on the default port (22) I assume from another host in the LAN?
Are you expecting that to work externally?
Steve
-
@stephenw10
Just completed "yet another test"... and now know the issue lies with FreeBSD.
I have a DL380g7 server that uses ESXi 6.7 and it has a Linux Mint system set up for LAMP stack. It serves 8 of my private use websites flawlessly. But I suspected FreeBSD without the GUI and all would likely be a better choice for the purpose. So have built a FAMP stack on it, and it works quite nicely. But for the issue of being blocked by pfSense...I've run through several versions of ESXi in an attempt to deal with a problem that is unrelated. Some people have said this and that doesn't work. I have two Blue Iris camera systems that serve via the same public IP as the Linux webserver. All have worked flawlessly for 5 or more years. Using the 380g7 box.
Go figure !
I'm in the DUH ! mode right now... totally beyond me... -
This is how I've done all of my Port Forwarding rules.. this one is ported to 83 so as not to interfere with the production server.
@dhenzler
-
Using port forwards that pass that traffic like that rather than using an associated firewall rule is quite unusual. It could cause problems if you have multiple WANs.
It sounds like the FreeBSD server is blocking access from outside it's subnet. Like you have enabled pf or ipfw there.
Steve
-
@stephenw10
I suppose so... but the likelyhood of me ever getting two WAN addresses is NOT happening.I spent a half hour trying to find any signs of a BSD firewall to no avail. Perhaps iptables... but haven't learned how to check that. I did attempt to set the gateway address, and it came back that it had already been set.
I am truely dumfounded by this...
std-famp.txt
The file above is a .sh script that I used to create the server. I can't find anything in it that would kill the server.
Tried setting Apache on port 8080, and reconfigured the forwarding to use that. No Dice!
I am pleased that I've discovered the ESXi is not at fault, and now just need to figure out the mode of failure...Thanks for your continued interest... Maybe you'll figure something out.
I'm heading to bed.
-
Well I would still do the same test: try to connect externally then check the states created in pfSense. Those have to look good before digging any deeper.
If it is some source address restriction in the server (or something in front of it) then adding an outbound NAT rule on LAN in pfSense will work around that and prove it.
Steve
-
is this the rule method...?
-
No, a firewall matching that would be destination 192.168.15.162, since it applies to traffic after NAT has been applied. And it would have a source set as that could be any external IP.
But if you just edit the port forward and set 'Filter rule association' to 'Add associated filter rule', which is the default setting, then it will add the correct firewall rule for you.
Steve
-
Ok did that... as well played with ESXi's passthrough feature. Thought I'd set two of the 4 NIC's up as real hardware. Unfortunately they are still NOT considered as external. Been looking to see if there's a path to make them actually external.
This is SO frustrating. I'm Network litterate enough to know enough to get by, but things like this require a LOT of reading to configure and benefit from.
SR-IOV doesn't work with this hardware... perhaps if I looked for a more recent driver. What's strange to me is that the server NIC's are the path to the LAN for everything else. But blocked for BSD... Strange !
-
Are you at least seeing the correct state created on WAN and LAN in pfSense when you try to connect?
-
I'm not sure I'd know the correct state if it fell on me !
Wish the ESXi NIC thing would have worked. I can by the way ping out from BSD, and see the response... so that says something. I guess I could try firing up WireShark. -
@stephenw10
New info may shed some light on a solution...
https://www.intel.com/content/www/us/en/support/articles/000005722/ethernet-products.html
Frequently Asked Questions for SR-IOV on Intel
Ethernet Server...
Intel Network Adapter FreeBSD* Virtual Function Driver for Intel Ethernet Controller 700 and E810 Series; ... Windows Server 2012* R2, Windows Server 2016*, and Windows Server 2019* include support for SR-IOV-capable network devices. An SR-IOV virtual function of a physical network adapter can assign directly to a virtual machine. Earlier ...I'm going to follow up on this path for a bit...
-
If you have a port forward like this:
And you try to connect to it from the WAN side you should see states like this:
There are no replies shown there because there is no server at 192.168.22.87 listening on port 5555. But the traffic is still NAT'd and routed as expected with states on both WAN and LAN.
Steve
-
@stephenw10
This is what I got...
-
You are testing from inside the network, the source IP is 192.168.15.39.
You have to test from outside the network to hit a port forward on WAN. If you need that to work from the LAN side you have to enable NAT reflection. And it looks like you must have done that since it is redirecting but you have not set 'Enable automatic outbound NAT for Reflection' in Sys > Adv > Firewall&NAT. And that means the server is replying directly to the client creating an asymmetric route.
Enable that or test from an external IP address.
Steve
-
@stephenw10
my server isn't on 240...? 240 is the ring doorbell... -
Ok, then the wrong port forward rule is catching it. Possibly a 1:1 rule but port forwards override those if they match.
Let's see you port forwards.
Steve
