Best Network Topology with Current Hardware
-
Hi,
Having had slow inter-VLAN speeds for a while, it has become clear that I'll need to do this at the switch level and use Pfsense purely for the firewall.
To that end my current hardware is:-
- Single internet connection (a pathetic 60Mbps down & 12Mbps up), will be upgraded to 1G up&down in next 12 months once available)
- Dell Optiplex 5050 (i5, 8MB RAM) running Pfsense CE (6 1GB ports plus the motherboard 1G port)
- Draytek P2280X 24 1GB ports & 4 SFP+ 10GB ports (128Gbps switching) (2 of)
- Draytek P1280 24 1GB ports & 4 SFP 1GB ports (56Gbps switching) (2 of)
- Draytek P2100 8 1GB ports & 2 SFP 1GB ports (20Gbps switching) (1 of)
- Netgear GS724T 24 1GB ports & SFP 1GB ports (1 of) - this is in a remote rack away from the main rack and won't be heavily loaded)
- Synology DS920+ NAS (2GB ports), currently connected using a LAG
Given the above hardware, what would be the best topology here?
Was planning to use the 2 P2280X as core / distribution switches linked via LAG using 10GB DAC cables & then the 2 P1280 switches and the Netgear switch as the access switches.
Currently the NAS and Pfsense are on LAG and connected to the two P1280 switches with the 2 P2280X not used.
Any thoughts on how to wire all this up would be greatly appreciated. Main priority is to get wire inter-VLAN speed (via the P2280X's).
Steve
-
@stevencavanagh
I think you plan will give you the best inter-VLAN routing performance with that hardware. I would also connect all performance sensitive devices to the core switches if possible. Do not route access to the storage if you do so. I have two servers connected directly to my core and they are on the same subnet as the main workstations. If you have many devices, using LAGs improves performance, but you may consider using some available links for redundancy instead. Introduce some software, say Zabbix, for traffic monitoring to get good info onto the data volumes. -
@kjk54
Looked at a topology to give redundancy but it was suggesting the 2 core switches are stacked and then uses LAGs to key components such as as servers, routers etc.
However, it seems that the core switches I have (P2280s) cannot be stacked, only being connected via a LAG.
Consequently, I am assuming I would remove all VLANS / DHCP servers from Pfsense and use 1 of the cores switches as a true core (with DHCP sever) and the rest as effectively edge servers but try and get all the performance sensitive devices on the core if possible and if not then on the second P2280X. The rest can go on the other switches.
-
True, your P2280s cannot be stacked, but you still can have redundancy. You just can't have those redundant links active at the same time or put them into a single LAG. You can use STP/RSTP to manage that. You have many ports on your switches so you may be able to afford that.
My core is actually a stack of two switches so each of my access switch/server LAG can connect to both of them. My file and backup servers are Synology NASes, too, and I use their DHCP servers instead of those offered by pfSense. It is not obvious, but those Synology DHCP servers can provide addresses for multiple subnets. Of course, your routing switch needs to be able to forward DHCP requests. I also use their DNS servers in conjunction with the pfSense's resolver. My local network can be fully functional, except for Internet access, even if I need to disconnect or shutdown pfSense for some reason.
-
@kjk54
As you say I should be able to implement STP/RSTP given the number of ports and I'll look into that one!Wasn't aware you could do DHCP on the Synology NAS for multiple subnets but was planning to put the DHCP server on the core L3 switch, which brings me to the next question.
I have read through the Draytek documentation regarding setting up the L3 switch for VLANs (done) and DHCP server (done) but in terms of getting the static route to Pfsense...struggling.
The Pfsense has IP 192.168.0.1
I have set up 6 VLANs on the switch, each with a DHCP server :-
192.168.20.1
192.168.30.1
192.168.40.1
192.168.50.1
192.168.60.1
192.168.70.1Should I add another VLAN for all traffic going to the internet, and if so what do I put in the static route?
I have the option to add a route, with a destination IP, subnet and gateway.
-
Yes, you can use a so-called transit VLAN to link your L3 switch with pfSense, say VLAN99. However, I recommend using in it IP addresses from a different address space than 192.168.0.0/16, say 10.10.99.1 on the switch and 10.10.99.2 on pfSense. This will make it possible to have a summary static route on pfSense. You would need two static routes, one on the switch and the other on pfSense. On the switch, it needs to be a default route with pfSense as the gateway, so it would be something like Dest 0.0.0.0, Prefix 0, Next Hop 10.10.99.2, Interface VLAN99. On pfSense, you would need to create first a gateway with the IP address 10.10.99.1. To avoid some potential problems, disable the gateway monitoring for it and make sure the pfSense WAN is the default gateway. On this new LAN gateway, you would need to create a summary static route to 192.168.0.0/16. You will no longer have VLANs on pfSense so the port used to connect the switch with pfSense would need to be an access port to VLAN99, not a trunk.
That's the easy part of the whole exercise. The difficult part will be to replace the firewall rules you might've created for your VLANs on pfSense with some ACL rules on the switch. ACL rules are not stateful and may be limited on a particular switch. That's the price to pay for faster inter-VLAN routing. Sorry to say that, but I stay away from help with it.
-
@kjk54 said in Best Network Topology with Current Hardware:
Yes, you can use a so-called transit VLAN to link your L3 switch with pfSense, say VLAN99. However, I recommend using in it IP addresses from a different address space than 192.168.0.0/16, say 10.10.99.1 on the switch and 10.10.99.2 on pfSense. This will make it possible to have a summary static route on pfSense. You would need two static routes, one on the switch and the other on pfSense. On the switch, it needs to be a default route with pfSense as the gateway, so it would be something like Dest 0.0.0.0, Prefix 0, Next Hop 10.10.99.2, Interface VLAN99. On pfSense, you would need to create first a gateway with the IP address 10.10.99.1. To avoid some potential problems, disable the gateway monitoring for it and make sure the pfSense WAN is the default gateway. On this new LAN gateway, you would need to create a summary static route to 192.168.0.0/16. You will no longer have VLANs on pfSense so the port used to connect the switch with pfSense would need to be an access port to VLAN99, not a trunk.
That's the easy part?? Will have a go and see where I get.
Cheers
Steve -
@kjk54
So, had a go at the L3 switch config, and.......I created a new VLAN99 (transit).......
Then created a route.........
If I understand it correctly, then this is all I need on the switch side for now and I now need to sort out the Pfsense end??
-
The images are "Not Found."
-
@kjk54
Try these pics...........
-
V viragomann referenced this topic on
-
I'm not familiar with that switch, but the route looks suspicious to me. There is no interface in it. It should indicate that the VLAN99 interface is to be used for it.
Also, have you already tested IPv4 routing on the switch? Have you defined an access port for VLAN99?
-
@kjk54
I followed a Draytek article that created a route for all traffic that wasn't in the VLANs by ticking the 'default' box above.
When I try and add it using the interface i.e. VLAN 99 I get an error.
I have not tested IPV4 routing on the switch as at the moment due to WFH, I simply have the two P2280Xs connected on the desk via the uplinks and a laptop so I don't kill off the main home network. Yes, I defined an access port for VLAN99.
-
"I followed a Draytek article that created a route for all traffic that wasn't in the VLANs by ticking the 'default' box above."
I don't have any idea what it is.
Have you enabled the IPv4 routing on your L3 switch? You need to get IPv4 routing up and running on the switch first. You can test it with two devices connected to the same switch. For the test to be valid you need to disconnect the uplinks. Note that in this configuration, the switch interfaces need be the gateways on your endpoint devices, not the pfSense interfaces.
-
@kjk54
The default box was in the pic above and the article was...
https://www.draytek.co.uk/support/guides/kb-vigorswitch-vlanrouting
However, I have enabled IPv4 routing on the L3 switch. I can disconnect the uplinks but will struggle to test at the moment as I don't have 2 PCs near the switches to test with unfortunately.
-
It may be okay. Possibly they just do not bother to show the outgoing interface since it can be determined from the subnet of the next hop.
You do not necessarily need another PC. Almost anything with a RJ45 jack can be used.
-
In normal you can use all switches as Layer2 switches and the pfSense
is routing the entire traffic and also the vlans or you may be set up a
transfer net between the pfSense and the or one Layer3 switch and the
Layer3 switch is routing then the vlans perhaps a bit faster, but you may
be setting up then ACLs on the switch for the vlan traffic (to secure or allow/deny) -
@kjk54
Okay, I'm sure I can find something with a RJ45 connector!
-
@Dobby_ said in Best Network Topology with Current Hardware:
In normal you can use all switches as Layer2 switches and the pfSense
is routing the entire traffic and also the vlans or you may be set up a
transfer net between the pfSense and the or one Layer3 switch and the
Layer3 switch is routing then the vlans perhaps a bit faster, but you may
be setting up then ACLs on the switch for the vlan traffic (to secure or allow/deny)I am currently using the switches as Layer 2 and routing the VLANs via Pfsense but the inter VLAN speed is pathetic, much lower than it should be but I can't seem to get to the bottom of it. For example Iperf to NAS on 1G network.....
Could put the NAS on the main LAN and get wire speed but still need to access it from the IOT network (firestick)! Hence the reason for going down the L3 route.
Same LAN speed is fine.......Between 2 PCs on same VLAN........
-
The NAS speed can be also based on the HDD/SSD´s speed!
Perhaps there is also not the max. load on the line, so you could try out iPerf
with multiple streams to gain the load for getting better "numbers". -
@Dobby_ said in Best Network Topology with Current Hardware:
The NAS speed can be also based on the HDD/SSD´s speed!
Perhaps there is also not the max. load on the line, so you could try out iPerf
with multiple streams to gain the load for getting better "numbers".How do I run Iperf with multiple streams?
Totally get that the NAS speed could be affected by HDD speed but they are ironwolfs in a Synology DS920+ and reading around people seem to be getting much better results than me!
