Switch+routing is impossible, except in netgate?
-
Running around breaking my head on pfsense and my usecase now for weeks and finally decided to stop reading more posts and ask the question.
I'm trying to understand what to do. As many others I bought a barebone with more then 2 ports and like to use one port for WAN and (in my case) the other three as switch. As I understand from this forum this is not the way to go. Buy another switch, but that reduces my bandwidth on the clients.
Still puzzled I find the manual of Netgate (https://docs.netgate.com/pfsense/en/latest/solutions/sg-1100/configuring-the-switch-ports.html) telling me that it is possible in the netgate hardware. Therefor Netgate has already implemented it, just not for homelab users I presume.
If I look at my barebone it reaches an utilization of 20% when I use a single LAN+WAN configuration and full gaming. My full bandwidth is in use. Still room for running a virtual switch on the same box. Has anyone a clue how Netgate solved this where as the major responses are that it is not feasible???
Or has anyone experience in putting pfsense with an open source switch on one barebone? -
@talldragon said in Switch+routing is impossible, except in netgate?:
Buy another switch, but that reduces my bandwidth on the clients.
And where would you have gotten that idea from?
just not for homelab users I presume.
Some netgate appliances have switch ports, netgate appliances come with FE or the now called + version of pfsense that can manage these switch ports. What 3rd party box has discrete interface ports and switch ports, or even just switch ports that would run pfsense CE?
I have not seen any 3rd party box that has switch ports, nor have I see any DIY sort of whitebox that has built in switch ports.. Sure your typical wifi router has switch ports, but pfsense would not run on such hardware - atleast I am not aware of any common wifi router that would run pfsense.
I am curious where you got the idea that running switches on your network would somehow reduce your bandwidth?
-
@johnpoz thank you for the response.
Where did I get the idea? I have a switch with 8 1GB ports and I use one for trunk with the router, then the other 7 ports (also 1 GB) share the same uplink. If all ports are used at full utilization my stupid assumption seems to be that data of 7GB doesn't fit in 1 1GB trunk without delay. I got that wrong as you explained. And what is also new to me, thanks for that insight, is that NIC's in switches and in ordinary hardware are something completely different. I'm pretty new in the details of switches and thought of them as regular hardware. Did not investigate where they differ from normal NIC's.
In the mean time I found an article of proxmox + pfsense + openvswitch that might solve the use case. Or even Linux bounds/bridges on Proxmox could help out creating a switch of multiple NIC's. Then is just telling pfsense to use the virtual linux bridge as one lan port. Let's find out. It's home lab anyway.
There seems to be a possibility to have virtual NICs that even don't exist in hardware but are part of a switch. Would be nice. -
@talldragon said in Switch+routing is impossible, except in netgate?:
@johnpoz thank you for the response.
Where did I get the idea? I have a switch with 8 1GB ports and I use one for trunk with the router, then the other 7 ports (also 1 GB) share the same uplink. If all ports are used at full utilization my stupid assumption seems to be that data of 7GB doesn't fit in 1 1GB trunk without delay. I got that wrong as you explained. And what is also new to me, thanks for that insight, is that NIC's in switches and in ordinary hardware are something completely different. I'm pretty new in the details of switches and thought of them as regular hardware. Did not investigate where they differ from normal NIC's.
In the mean time I found an article of proxmox + pfsense + openvswitch that might solve the use case. Or even Linux bounds/bridges on Proxmox could help out creating a switch of multiple NIC's. Then is just telling pfsense to use the virtual linux bridge as one lan port. Let's find out. It's home lab anyway.
There seems to be a possibility to have virtual NICs that even don't exist in hardware but are part of a switch. Would be nice.You need to ask yourself the following:-
How many different subnets do I require?
Where does the traffic from each subnet go ?
-
@talldragon said in Switch+routing is impossible, except in netgate?:
stupid assumption seems to be that data of 7GB doesn't fit in 1 1GB trunk without delay
So the only traffic that would go up this trunk would be traffic that needs to be routed by pfsense.. Is your internet faster than 1gig? If not then how would the uplink be slowing down your traffic.
Sure yes you could run into a bottle neck with hairpinning traffic.. If multiple clients on the switch wanted to talk to other multiple clients on the switch that needed to be routed by pfsense and they were all sharing the uplink then yeah you could run into a bottle neck.
You can run into the same problem though on an appliance with switch ports - since the connection from the switch ports to the routing is limited as well.. SG1100 only has 3 1g switch ports.. So not a very good example.
But say the 2100 has 4 ports on its switch, but the soc is only 2.5ge
https://docs.netgate.com/pfsense/en/latest/solutions/netgate-2100/switch-overview.html
The internal uplink port operates at 2.5 Gbps and connects the switch to the SoC.That should be enough to cover any traffic you route, but its still a limitation.. If it had 6 or 8 switch ports it could be a bottleneck..
Hairpinning traffic over the same physical interface, or sharing the a single uplink from switch to another switch, etc. Or an internal connection from switch ports to the routing which could be the case with router and switch ports on it. You still can run into a problem with an uplink be it a shared physical interface, or a soc sort of connection from switch ports to what is routing
So don't hairpin if your concerned.. I have multiple uplinks for different vlans from my switch to the router.. So network X talking to network Y has full gig between, no hairpin through the router.. The only vlans I hairpin are wireless clients through their AP.. These clients are never going to be talking gig anyway, and they don't actually talk to each other either. So really the only traffic running through that uplink is the the internet, my internet is 500, clients don't talk faster than anyway.. So that is not a bottleneck. Or to some other network that is routed say my plex servers network. But again the wireless client(s) can not push gig anyway.. But yeah any physical interface is going to be limited by its bandwidth.
Run a faster uplink than your clients connection, run multiple uplinks.. Or route at the layer3 switch for intervlan traffic..
So what is this 3rd party box you have that has switch ports on it - what is the the uplink bandwidth from the switch ports to the routing engine on this 3rd party box?
-
I need 4.
1 for my PC, homelab and mgt of devices
1 for IoT (Internet of Things)
1 for NoT (Network of Things - do not need a internet connection)
1 GuestI have one incoming ISP modem and three cables leading from the closet of the ISP modem towards three different rooms. So I like to have a virtual network available in the three rooms and put a managed switch in every room for the devices. I have no space for more cables through the apartment.
-
@talldragon And where is this bottleneck going to be? Is your internet over 1 gig? So your devices on 1 cable are going to be talking to other things routed over the same cable via a hairpin at more than gig? So like 2 devices on some downstream switch talking to 2 other things on the same switch moving more 1gig each?
Use 2.5/5/10 for your uplink would be one solution.. I don't see how proxmox is going to fix that problem.. Sure its internal switch you create in software might be say 10gig, and maybe its hardware can do that via software for more than 1gig of bandwidth. How does that solve a problem when you go to physical, ie your 1 cable from your devices to your router?
Also when it comes to "routing" just because you connect two ends at say 10ge - doesn't mean the router can push that much traffic even if the interfaces are not a bottleneck.
-
HUNSN RJ03k with 4x Intel 2.5GbE I226-V LAN - https://www.amazon.nl/dp/B0B6J12LGJ?ref=ppx_yo2ov_dt_b_product_details&th=1
-
@johnpoz
I'm not in this forum to argue choices. My opinion why I started my question: home labs and networking should be fun and exiting. Not hindered by opinions but learned through experimenting based on knowledge and curiosity. IT constantly changes, what was impossible yesterday is the standard of today.And I understand that one can get tired of all those fools that are trying to find solutions to experiment with. But I didn't ask for my use case to be solved, just to explore ways that would work and could create more fun.
So yes, I would like to experiment with my idea and find out where I'm wrong. I would like to play around. And I would like to do that with curious people who also love making fun with networking. Why would I otherwise get pfsense, could have gone for refurbished cisco as well (but that is less fun).
And a good starting point to me is: join the community and build a group of people that share the curiosity and perhaps are steps ahead.Does that make sense or should I explain myself better?
