Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Port Forwarding ESXi VM no joy !

    Scheduled Pinned Locked Moved General pfSense Questions
    27 Posts 2 Posters 2.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      dhenzler
      last edited by

      I have successfully done eight or so Port Forwards... the concept is clear in my mind. However in this case I am unable to uderstand what is wrong. Let me give some history:

      Existing DL380g7 running ESXi 6.7 I can forward a Linux Mint VM used as a web server to my PUBLIC IP address.

      Wishing to upgrade my equipment to later version. I configured a DL360p Gen8 server running ESXi HPE version. I successfully created a BSD server and it works on the LAN IP.

      However when I create a Port Forward of that LAN IP and PORT to the PUBLIC IP, it fails to work.

      • Obviously there is nothing wrong with the BSD server
      • ESXi is working properly as it puts the web content on the LAN
      • pfSense is working properly as it has performed on eight other Port Forward configurations.

      I am at a loss for the answer. I have even loaded ESXi ver 6.0 which was a disaster... my datastore was not found, and indicated "depreciated".
      So... hopefully someone will know what this is all about. I have burned 40+ hours on this and still "no joy"

      There are NO firewalls set in ESXi
      There are NO firwalls set in BSD
      The gateway address is correct
      The routing is correct pfSense reports the routing as ?.?.?.0 routed to ?.?.?.0:30 which is correct. As well everything else works flawlessly.

      Have given some thought to creating a new "port group" in ESXi. But that is my last possible answer.

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        pfSense is not running in ESXi here? It's upstream of that?

        The server in ESXi is using an IP in the pfSense LAN subnet directly? Is it statically configured?
        Do you see it in the ARP table in pfSense?

        @dhenzler said in Port Forwarding ESXi VM no joy !:

        The routing is correct pfSense reports the routing as ?.?.?.0 routed to ?.?.?.0:30 which is correct

        Where are you seeing that exactly? Those IPs are unusual (0 last octet) and the destination as port 30 is also...odd.

        Try to connect to the server from something external and then check the state table in pfSense. You should see the incoming connection create states on WAN and LAN.

        Steve

        D 1 Reply Last reply Reply Quote 0
        • D
          dhenzler @stephenw10
          last edited by dhenzler

          The WAN IP's are what was assigned by my ISP. Including the /30
          The ESXi VM works but is only visible on the LAN. All of the other Port Forward rules I've created are working... so I must not be a total idiot !

          @stephenw10 ipv4_routes.png
          ESXi_Network.png
          ssh -vvv root@192.168.15.159 <== the web-server ip

          The authenticity of host '192.168.15.159 (192.168.15.159)' can't be established.

          ECDSA key fingerprint is SHA256:97vKVNNToueigzZMO7dYK8AGTWdVcontohO1B7kszZo.

          I've tried ESXi on a DL-380eGen8 and it does the same thing. Beginning to think the issue is VMware related... But how does pfSense know that the IP should not be forwarded... or is the IP really empty unless viewed on the LAN?

          I'm in beyond my depth...

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Ah, OK. /30 subnets for routing is fine. :30 looks like a port which would be odd.

            Ok, so let's see your port forward rules.

            You are testing access using ssh on the default port (22) I assume from another host in the LAN?

            Are you expecting that to work externally?

            Steve

            D 1 Reply Last reply Reply Quote 0
            • D
              dhenzler @stephenw10
              last edited by

              @stephenw10
              Just completed "yet another test"... and now know the issue lies with FreeBSD.
              I have a DL380g7 server that uses ESXi 6.7 and it has a Linux Mint system set up for LAMP stack. It serves 8 of my private use websites flawlessly. But I suspected FreeBSD without the GUI and all would likely be a better choice for the purpose. So have built a FAMP stack on it, and it works quite nicely. But for the issue of being blocked by pfSense...

              I've run through several versions of ESXi in an attempt to deal with a problem that is unrelated. Some people have said this and that doesn't work. I have two Blue Iris camera systems that serve via the same public IP as the Linux webserver. All have worked flawlessly for 5 or more years. Using the 380g7 box.

              Go figure !
              I'm in the DUH ! mode right now... totally beyond me...

              D 1 Reply Last reply Reply Quote 0
              • D
                dhenzler @dhenzler
                last edited by dhenzler

                This is how I've done all of my Port Forwarding rules.. this one is ported to 83 so as not to interfere with the production server.
                @dhenzler 192.168.15.142.png

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Using port forwards that pass that traffic like that rather than using an associated firewall rule is quite unusual. It could cause problems if you have multiple WANs.

                  It sounds like the FreeBSD server is blocking access from outside it's subnet. Like you have enabled pf or ipfw there.

                  Steve

                  1 Reply Last reply Reply Quote 0
                  • D
                    dhenzler
                    last edited by dhenzler

                    @stephenw10
                    I suppose so... but the likelyhood of me ever getting two WAN addresses is NOT happening.

                    I spent a half hour trying to find any signs of a BSD firewall to no avail. Perhaps iptables... but haven't learned how to check that. I did attempt to set the gateway address, and it came back that it had already been set.
                    I am truely dumfounded by this...
                    std-famp.txt
                    The file above is a .sh script that I used to create the server. I can't find anything in it that would kill the server.
                    Tried setting Apache on port 8080, and reconfigured the forwarding to use that. No Dice!
                    I am pleased that I've discovered the ESXi is not at fault, and now just need to figure out the mode of failure...

                    Thanks for your continued interest... Maybe you'll figure something out.

                    I'm heading to bed.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Well I would still do the same test: try to connect externally then check the states created in pfSense. Those have to look good before digging any deeper.

                      If it is some source address restriction in the server (or something in front of it) then adding an outbound NAT rule on LAN in pfSense will work around that and prove it.

                      Steve

                      D 1 Reply Last reply Reply Quote 0
                      • D
                        dhenzler @stephenw10
                        last edited by

                        @stephenw10

                        is this the rule method...?Screenshot from 2022-05-18 12-38-07.png

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          No, a firewall matching that would be destination 192.168.15.162, since it applies to traffic after NAT has been applied. And it would have a source set as that could be any external IP.

                          But if you just edit the port forward and set 'Filter rule association' to 'Add associated filter rule', which is the default setting, then it will add the correct firewall rule for you.

                          Steve

                          D 1 Reply Last reply Reply Quote 0
                          • D
                            dhenzler @stephenw10
                            last edited by dhenzler

                            @stephenw10

                            Ok did that... as well played with ESXi's passthrough feature. Thought I'd set two of the 4 NIC's up as real hardware. Unfortunately they are still NOT considered as external. Been looking to see if there's a path to make them actually external.

                            This is SO frustrating. I'm Network litterate enough to know enough to get by, but things like this require a LOT of reading to configure and benefit from.

                            SR-IOV doesn't work with this hardware... perhaps if I looked for a more recent driver. What's strange to me is that the server NIC's are the path to the LAN for everything else. But blocked for BSD... Strange !

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Are you at least seeing the correct state created on WAN and LAN in pfSense when you try to connect?

                              D 2 Replies Last reply Reply Quote 0
                              • D
                                dhenzler @stephenw10
                                last edited by

                                @stephenw10

                                I'm not sure I'd know the correct state if it fell on me !
                                Wish the ESXi NIC thing would have worked. I can by the way ping out from BSD, and see the response... so that says something. I guess I could try firing up WireShark.

                                1 Reply Last reply Reply Quote 0
                                • D
                                  dhenzler @stephenw10
                                  last edited by dhenzler

                                  @stephenw10
                                  New info may shed some light on a solution...Screenshot from 2022-05-19 00-45-14.png

                                  https://www.intel.com/content/www/us/en/support/articles/000005722/ethernet-products.html

                                  Frequently Asked Questions for SR-IOV on Intel® Ethernet Server...
                                  Intel® Network Adapter FreeBSD* Virtual Function Driver for Intel® Ethernet Controller 700 and E810 Series; ... Windows Server 2012* R2, Windows Server 2016*, and Windows Server 2019* include support for SR-IOV-capable network devices. An SR-IOV virtual function of a physical network adapter can assign directly to a virtual machine. Earlier ...

                                  I'm going to follow up on this path for a bit...

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    If you have a port forward like this:
                                    Screenshot from 2022-05-19 12-50-56.png

                                    And you try to connect to it from the WAN side you should see states like this:
                                    Screenshot from 2022-05-19 12-52-08.png

                                    There are no replies shown there because there is no server at 192.168.22.87 listening on port 5555. But the traffic is still NAT'd and routed as expected with states on both WAN and LAN.

                                    Steve

                                    D 1 Reply Last reply Reply Quote 0
                                    • D
                                      dhenzler @stephenw10
                                      last edited by

                                      @stephenw10
                                      This is what I got...
                                      states_test.png

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        You are testing from inside the network, the source IP is 192.168.15.39.

                                        You have to test from outside the network to hit a port forward on WAN. If you need that to work from the LAN side you have to enable NAT reflection. And it looks like you must have done that since it is redirecting but you have not set 'Enable automatic outbound NAT for Reflection' in Sys > Adv > Firewall&NAT. And that means the server is replying directly to the client creating an asymmetric route.

                                        Enable that or test from an external IP address.

                                        Steve

                                        D 1 Reply Last reply Reply Quote 0
                                        • D
                                          dhenzler @stephenw10
                                          last edited by dhenzler

                                          @stephenw10
                                          states_test.png
                                          my server isn't on 240...? 240 is the ring doorbell...

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Ok, then the wrong port forward rule is catching it. Possibly a 1:1 rule but port forwards override those if they match.

                                            Let's see you port forwards.

                                            Steve

                                            D 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.