Unable to access Internet from virtual network
-
I'm currently configured with (to my knowledge) the firewall component turned off.
I have a virtual network configured, one of several, for my server devices. That network is configured for 10.1.10.0/24
The WAN port for PFSense is 192.168.1.175
Devices on the 10.1.10.0/24 network are configured with static IPs. 3 servers, each can ping the other two and can ping the 10.1.10.1 gateway that was configured for that virtual network.
The devices can't access internet at all, though I have a global WAN gateway configured with my internet router's address (192.168.1.1)
On the individual network interfaces configuration area the IPv4 Upstream gateway is "None". If I attempt to add a new gateway I get messages that my default gateway (192.168.1.1 as far as I know) is not in the right subnet.
What am I doing wrong and how do I add access to the internet to this virtual network (which I would then need to repeat on other virtual networks that I had created within the ESXi virtual network as well as within the pfsense configuration)
Thanks for any assistance!!!
-
Point of confusion - what should my LAN address be for the pfsense device? (Not the WAN address, but the LAN address)
-
Not clear what role pfSense plays in your network.
You have only mentioned the WAN address of pfSense.
Maybe a drawing could help for clarify. -
Are you volunteering to make the drawing? (Not to give you a hard time, but drawings aren't necessarily that easy to achieve for all of us).
Here's a general picture of things:
Internet / World <---> ISP Router <---> Home network
That's the before picture, simple arrangement.
I'm now adding an ESXi host that is also on the Home network side of the ISP Router, so it would be something like this:
ISP Router <---> Home Network
^----> ESXi hostWithin ESXi host, I have multiple virtual networks carved up.
ESXi Host and Virtual networks:
VMX0 - WAN
VMX1 - LAN / Virtual only (though I want to allow access through to Internet)VMX1 includes multiple virtual networks:
Server1 - 10.1.10.0/24
Server2 - 10.1.15.0/24
Workstation1 - 10.1.20.0/24
Workstation2 - 10.1.30.0/24
DMZ1 - 172.xxxxxx/24
DMZ2 - 172.xxxxxx/24
External1 - 192.168.10.0/24
External2 - etc.Home network (or WAN for the pfsense firewall) is 192.168.1.xxx
PFSense is configured with two interfaces, one to VMX0, the other to VMX1
Trunk Port (4095) on the VMX1 virtual-only network -
This should help make more sense. It only shows part of the network design, but enough to help make sense of things if above text wasn't clear enough.
-
To add here, the role for pfsense is to serve as a virtual router, for the devices within the "DOM" mentioned above. (The ESXi host actually). It is also supposed to serve as a firewall to keep things isolated between networks unless I turn off the firewall or add rules to allow specific traffic through.
In working through things as simply as possible, I have rules that should be allowing all traffic to flow between the virtual ports but that doesn't seem to be working, and even turning off the firewall entirely isn't allowing the traffic to flow outside of either virtual port on through to the internet.
-
I don't see the need of VLANs here, since alle those networks exists only in ESXi. Why don't you just add virtual network (vSwitches) in ESXi.
However, if you connect pfSense to a trunk port like that, you have to define each VLAN in pfSense. Interfaces > VLANs
Then go to Interfaces > Interface Assignments and add an interface for each VLAN and configure it.
The gateway 10.1.10.1 you mentioned above should also be one of these VLAN interfaces, I think.In the WAN interface settings add the 192.168.1.1 as upstream gateway.
-
@viragomann said in Unable to access Internet from virtual network:
I don't see the need of VLANs here, since alle those networks exists only in ESXi. Why don't you just add virtual network (vSwitches) in ESXi.
However, if you connect pfSense to a trunk port like that, you have to define each VLAN in pfSense. Interfaces > VLANs
Then go to Interfaces > Interface Assignments and add an interface for each VLAN and configure it.
The gateway 10.1.10.1 you mentioned above should also be one of these VLAN interfaces, I think.In the WAN interface settings add the 192.168.1.1 as upstream gateway.
There's more to it in terms of why the need, but the virtual networks are defined in ESXi and in pfsense, with each linked as expected.
The default gateway has been configured for each of the virtual lan interfaces.
For the WAN, the 192.168.1.1 address was added as the gateway. For the virtual networks, the configuration looks something the one that is shown here:
Wan Interface looks like this:
The virtual networks are mapped/identical between ESXi and pfsense.
-
More pictures:
-
so you what did you change the lan to on pfsense which defaults to 192.168.1.1 even..
You can not have 192.168.1 on your wan and your lan of of pfsense and expect anything to work.
And again WTF you doing vlans for in pfsense if all of those networks are only virtual and on your esxi host?
-
The LAN on the pfsense side should be 10.1.1.0/24
And again, the WTF is complicated, but expect it to work out to allowing for more firewall control over what passes between the virtual lans.
-
Its NOT going to work how you have it setup because your vswitches will not pass any tags to pfsense!!
You have zero use of tagging anything if all its going to be is virtual vswitches. Just create another vnic for pfsense and attached it to your vswitch you connect those vms too..
All of the networks in pfsense will be native with what pfsense thinks is a real interface - you have just as much control between your networks this way.
-
@terpfan1980 said in Unable to access Internet from virtual network:
The LAN on the pfsense side should be 10.1.1.0/24
And again, the WTF is complicated, but expect it to work out to allowing for more firewall control over what passes between the virtual lans.
@johnpoz said in Unable to access Internet from virtual network:
Its NOT going to work how you have it setup because your vswitches will not pass any tags to pfsense!!
You have zero use of tagging anything if all its going to be is virtual vswitches. Just create another vnic for pfsense and attached it to your vswitch you connect those vms too..
All of the networks in pfsense will be native with what pfsense thinks is a real interface - you have just as much control between your networks this way.
More info please? I'm following along with instructions that were written up by someone else and have been used by others. On the ESXi side, the same VLan ID's exist (is that the tagging you are referring to??) as are used on the pfsense side of things.
So I'm lost a bit on what you are saying about things not being tagged here.
Thanks for helping to explain it more clearly.
-
Another image from the instructions that I've been working from:
-
-
You would use the tags with those switches if going to physical network... I have no idea who wrote up instructions be makes zero sense like that - ZERO!! And they don't seem to understand basic networking in esxi..
When you set a vlan ID on a vswitch it does not pass the tag to the vnics connected to this vswitch... So if your not passing tags, you woudn't be setting up vlans in pfsense.. You only setup vlans in pfsense when pfsense has to deal with the tags...
If you want pfsense to deal with the tags, then it would just be 1 vnic and 1 vswitch and you would use 4095 on the vswitch so you would pass the tags to the vnic so that pfsense could deal with the tags.
Which you set a vlan id on a vswitch its like an access port on a cisco switch.. The vm device connected to it will not see any tags..
if your going to tag your vswithes like that it would be for them to connect to a physical network and to mark traffic towards the real network with the tags so that the real switch can handle the traffic correctly. But any vms connected to those port groups would not be seeing any tags.
His instructions say to create vlans on pfsense??? Or your thinking you need too? You would create another vnic for each vswitch/port group you create for whatever vlans.. But since nothing is going to to the real network just leave the vlan id as 0 on all of them and just name the vswitch/port group for whatever you want to use that network for.. Its like just using a different dumb switch for each network and real physical interface in pfsense connected to the different dumb switches.
-
@johnpoz said in Unable to access Internet from virtual network:
You would use the tags with those switches if going to physical network... I have no idea who wrote up instructions be makes zero sense like that - ZERO!! And they don't seem to understand basic networking in esxi..
When you set a vlan ID on a vswitch it does not pass the tag to the vnics connected to this vswitch... So if your not passing tags, you woudn't be setting up vlans in pfsense.. You only setup vlans in pfsense when pfsense has to deal with the tags...
If you want pfsense to deal with the tags, then it would just be 1 vnic and 1 vswitch and you would use 4095 on the vswitch so you would pass the tags to the vnic so that pfsense could deal with the tags.
Which you set a vlan id on a vswitch its like an access port on a cisco switch.. The vm device connected to it will not see any tags..
if your going to tag your vswithes like that it would be for them to connect to a physical network and to mark traffic towards the real network with the tags so that the real switch can handle the traffic correctly. But any vms connected to those port groups would not be seeing any tags.
His instructions say to create vlans on pfsense??? Or your thinking you need too? You would create another vnic for each vswitch/port group you create for whatever vlans.. But since nothing is going to to the real network just leave the vlan id as 0 on all of them and just name the vswitch/port group for whatever you want to use that network for.. Its like just using a different dumb switch for each network and real physical interface in pfsense connected to the different dumb switches.
The instructions say to create the same VLAN's on pfsense as were created on the ESXi side. The id's have to be the same in both places. On the ESXi side there is one additional VLAN ID (4095) for the Trunk port for pfSense LAN interface.
In my case (as in the case in the instructions), I've got one physical port on the Dell Optiplex Mini box. That physical port is used for the Management network for the ESXi host. (See vSwitch0 in the image sample in earlier reply)
pfsense has connection to that vSwitch as well as the Management Network for ESXi.
So, my understanding of things is that pfSense should be working as a router to allow connections through the various vlans back and forth and on to the outside world if desired. That's where I'm stuck. The communication between networks is happening, but I'm not getting communication out to the internet and beyond from any of the vlans that pfSense knows about (or that were created in the virtual-only switch on the ESXi side)
-
@terpfan1980 said in Unable to access Internet from virtual network:
The instructions say to create the same VLAN's on pfsense as were created on the ESXi side.
Then they are WRONG, or your not understanding them - what is the link to these instructions?.Or who wrote them doesn't understand how it works... Just plain freaking FACT!!! I have run pfsense on esxi since before there was esxi and ran it on vserver 1...
Yeah it can route just fine - has zero to do with creating vlans on pfsense if its never going to see the tags!!!
You came here for help, I am telling you how it works... You can listen or you can dick around with instructions that are just plain freaking WRONG!!! Please just read the esxi docs for EST or VST..
How is a vlan on pfsense going to work when it doesn't see any tags?
You only create vlans on pfsense with it will actually see a tag.. Be it physical or virtual.. That is not the case in such a setup. You need a different vnic for each vswitch you create..
-
@johnpoz said in Unable to access Internet from virtual network:
@terpfan1980 said in Unable to access Internet from virtual network:
The instructions say to create the same VLAN's on pfsense as were created on the ESXi side.
Then they are WRONG, or your not understanding them - what is the link to these instructions?.Or who wrote them doesn't understand how it works... Just plain freaking FACT!!! I have run pfsense on esxi since before there was esxi and ran it on vserver 1...
Yeah it can route just fine - has zero to do with creating vlans on pfsense if its never going to see the tags!!!
You came here for help, I am telling you how it works... You can listen or you can dick around with instructions that are just plain freaking WRONG!!! Please just read the esxi docs for EST or VST..
How is a vlan on pfsense going to work when it doesn't see any tags?
You only create vlans on pfsense with it will actually see a tag.. Be it physical or virtual.. That is not the case in such a setup. You need a different vnic for each vswitch you create..
Company proprietary Wiki site for the instructions, though I can clip some of material there (as included above)
Instructions for ProxMox have the same flow and also include pfSense in the same fashion.
I've not used pfSense that much (I've been a user without knowing it, first time working through the configuration myself). Similarly, my use of ESXi has been more limited, but I understand the basics of creating virtual switches, port groups, etc.
-
ESXi Networking
