Routing using a managed HPE Switch
-
Hi everybody!
I've been spending my nites trying to solve this without any success. Hope someone could help me understanding what's wrong with my config.LAN Structure:
PFSENSE (VIRTUAL ON VMWARE) WITH 3 INTERFACES:
-
WAN (192.168.60.200), DSL CONNECTION, GATEWAY 192.168.60.1
-
3G CONNECTION (10.10.10.10), 3G WAN CONNECTION, GATEWAY 10.10.10.1
-
LAN 172.20.0.200
-
HPE 1920 48G Switch, 172.20.0.254/24
My Pfsense box is connected to a vswitch in vmware that provides it:
VLAN 1, the default, provides LAN
VLAN 2000 provides for WAN DSL connection
VLAN 3000 provides for 3G connection.and here all works correctly.
Now what I did is this:
-
I created a new VLAN, called VLAN 30, class 192.168.4.0/24 under which i have 2 machines:
- FREENAS, 192.168.4.3 (not virtual, but we don't care)
- NAKIVO Backup, 192.168.4.2 (virtual and already connected to the new virtual port in vmware). -
I created a new VLAN interface in my 1920 HPE Switch, with 192.168.4.1/24 IP address
-
I added a gateway in my pfbox called "HPE Switch" with 172.20.0.254 IP address
-
i created a new rule under "LAN" in Firewall - Rules to allow traffic "any - any" to the subnet "192.168.4.0/24" via the new created Gateway.
So, from my workstation (172.20.0.80/24) i can access the webGUI of both machines,and i can ping them as well. BUT:
- both of the machines have no access to the internet
- Nakivo needs to have access to vmware vcenter on 172.20.0.203/24 but pfsense blocks all of the ports needed.
I didn't touch any NAT rules or option.
Could you PLEASEEEEE help me to find the clue?!! This is just a test that i am making because i will be in need shortly to allow WAN and LAN traffic on another LAN link i will be about to make with a couple of antennas… so i need to know how to make it.
Thanks in advance to all of you!
Matteo -
-
If your going to use L3, ie downstream router you need to connect to this via transit network.. Or your going to run into asymmetrical routing problems.
How exactly did you hang this 192.168 network off your PFsense VM? Are they only VM?? Or they are hung off your L3?
It would really help to draw this up.. I have to run to work or would draw up what you have stated and we could verify that everyone is on the same page for how your setup.
But in a nutshell to use downstream router (L3 switch) you could connect this to pfsense via a transit network (vlan or native).. You would then create a gateway on pfsense to downstream IP in the transit network. You would then create a route in in pfsense using this gateway.
Once you create the route, pfsense when in automatic outbound nat would add the downstream networks you created in the routes into your outbound nat.
You then just need to allow the downstream networks via your transit interface firewall rules.
On any other networks you have you would allow them access to these networks, before you force any traffic out some other gateways or gateway groups you have for failover. which seems you might have done since you mention 2 wan connections.
Drawing and rules you have on pfsense interfaces and your new transit network would allow us to figure out whats not correct. You didn't alter your outbound nat to manual did you? Posting of your outbound nat rules wouldn't hurt either.
-
Thanks a lot for your interest!
I guessed it could have been an asymmetrical routing… but it's under your eyes that i am not familiar to L3 routing so :-))I attach the scheme of the LAN a bit simplified and all LAN rules and NAT. In NAT you'll find a manual mapping I created just for accessing via IPSec from outside.
I understood anyway all you said... just.. don't know how to build it. I mean, ok for the transit network which I need to route traffic between the two, but how should I make it in pfSense?
How exactly did you hang this 192.168 network off your PFsense VM? Are they only VM?? Or they are hung off your L3?
right now, PfSense box is a VM, Nakivo is a VM and FreeNAS (these 2 both lying on 192.168.4.x) is a real machine.
They are connected to two ports on the HPE Switch this way: Nakivo via a trunk port because the VM is on the same host as Pfsense; FreeNAS via an access port untagged for VLAN30.
So.. right now I just have that GW 172.20.0.254 (same IP as HPE) used as you'll see to route LAN traffic to 192.168.4.x…If something in the I've drawn scheme is not clear tell me!
-
In your rules you forcing traffic out a gateway to the switch… Doing that is going to lead to problems.. You didn't setup any routes.. Since in your outbound nat where is this 192.168.4.0 network?
Why are you wanting to do layer 3? Where are the downstream neteworks from 192.168.4.1?
The way your drawing looks is you have this vlan 30 connected to pfsense riding on your Nic 1 interface?
"but how should I make it in pfSense?"
What a transit.. its just any network could be native or tagged..
How do you have your vswitches in esxi? Just differen switches tied 3 nics in host tied to three vnics in pfsense using 3 different vswitches?
-
In your rules you forcing traffic out a gateway to the switch… Doing that is going to lead to problems.. You didn't setup any routes.. Since in your outbound nat where is this 192.168.4.0 network?
S…t you're right! Should I do it manually since Pfsense hasn't done it automatically? Or should I create a static route with 192.168.4.1 as destination via Switch Gateway?Why are you wanting to do layer 3? Where are the downstream neteworks from 192.168.4.1?
I wanted to do layer 3 to use the switch basically..and to have much experience in this. I must tell you i am actually having trouble also with another vlan i tried to set up with "printers",but this is another story.
This is what I actually need: the downstream network FROM .4.1. How should I do that without any interface in Pfsense that i could manage?The way your drawing looks is you have this vlan 30 connected to pfsense riding on your Nic 1 interface?
Yes it is, Vlan30 goes straight to pfsense via a trunk port and a vswitch port group accepting all of the Vlans (4095 in vmware)"but how should I make it in pfSense?"
What a transit.. its just any network could be native or tagged..
How do you have your vswitches in esxi? Just differen switches tied 3 nics in host tied to three vnics in pfsense using 3 different vswitches?
No, i have 1 vSwitch in ESXi with different port groups each tagging its vlan, and each one of this is tied to a vnic in pfsense… -
"No, i have 1 vSwitch in ESXi with different port groups each tagging its vlan"
And you have this tied to the physical nics how?
"This is what I actually need: the downstream network FROM .4.1"
What downstream network? The only reason for downstream networks via layer 3 is that you want these multiple layer 3 networks downstream to be able to talk to each other at wire speed because the L3 switch (router) can route the traffic between the networks faster than letting pfsense route/firewall the traffic.
Unless you have such need, other than lab there is zero need for downstream layer 3 network. Just use your switch as layer 2 and let pfsense do all the routing and firewall between your networks.
Once you create a route in pfsense using the gateway (not set to default is it?) you created pfsense would automatic added the outbound nat network.
Another problem I see in your setup is if your going to use this 192.168.4 network to get to other downstream networks 192.168.5, .6, .7 whatever that you have on the switch.. Then you would not put devices on the 192.168.4 – which you show some VM and NAS? It is not a transit network if their are node type devices.. Only thing that would be on a transit network are routers ;)
Think of a transit network as a road to other roads.. Kind of like highway between cities... You don't build your house on the road - you build your houses in the city.. If do place your house on the highway between cities. When you want to go to City A, you don't drive to city B (default gateway) just to turn around and drive back the other direction on the highway to get to city A.
This is why any house you build on the highway (transit network) needs to have host routing to know which path to take to get where. What city do I go to first A or B to get to city C that is past either A or B, etc.
-
Thanks for all of your help!!
I went out of that mess a couple of days ago. What I got wrong was:- LAN interfaces should have no default gateways (!!!!!)
- I let PFbox to do all the routing, adding 2 new port groups to VmWare Vswitch (both tagged) and the related rules.
Now everything is working, i added manually outbound NAT because i wanted to access all of the other subnets from this new one i created, too.
Anyway, my HPE switch has 2 new VLAN interfaces, which i need to make all of this work! But I think this is not a layer 3, in the end… isnt it?
-
"i added manually outbound NAT because i wanted to access all of the other subnets from this new one i created, too."
What? Does that have to do with anything? Outbound nat has zero to do with access to other segments.. Pfsense doesn't even nat between networks on the lan side. It only would nat between a lan side interface and a wan (one with gateway set on it directly)..
You cold have 100's of vlans on your switch.. .Doesn't make it layer 3 routing… Did you set a SVI (Switched Virtual Interface) on these vlans? Ie set an IP address on these vlans?
-
What? Does that have to do with anything? Outbound nat has zero to do with access to other segments.. Pfsense doesn't even nat between networks on the lan side. It only would nat between a lan side interface and a wan (one with gateway set on it directly)..
I attach my outbound nat rules so that you can figure it out.. even if comments are in italian :-)
192.168.10.0/24 is a subnet leading via an Ubiquiti Antenna to my house. To let this have access to the OpenVPN via the pfbox i had to create that rule.. otherwise… no result...
192.168.4.0/24 is another subnet under which i have a couple of machines that need access to the VPN as well.. so i natted it...You cold have 100's of vlans on your switch.. .Doesn't make it layer 3 routing… Did you set a SVI (Switched Virtual Interface) on these vlans? Ie set an IP address on these vlans?
I set 2 different virtual interfaces on the respective Vlans and gave them IP address, ending .1 for each subnet.I imagined that natting was not the top, but creating the firewall rules for each interface was not enough to allow traffic, for example, from "madhouse" to "openvpn".
And actually, from the other end of the vpn i cant access "madhouse"…
the vpn tunnels in 192.168.30.0/24, and the subnet on the other hand is 192.168.0.0/24, so not conflicting with any other of the interfaces...
