Issue Bridging OPT1 & LAN for 10GB link into VM Environment
-
Background:
I'm doing something unconventional in order to get a 10GB link from my Workstation into my VM environment without having to buy an expensive 10GBe switch. I have internet on the workstation, but I cannot get the Workstation to talk to the VMs. All the NICs pfsense is accessing are 10gbe. Please see the network diagram below:
Main Question:
I would like "Workstation" to be assigned the IP 192.168.1.201 (preferable via the same dhcp server everything else is using on LAN) and be able to talk to the VMs at 192.168.1.5, 192.168.1.6, 192.168.1.7Right now I can assign the static IP 192.168.1.201 and get internet, but no communication from 192.168.1.201 to 192.168.1.5 or vice versa.
How can I get the machines to talk to each other? I have bridged OPT1 & LAN, and configured Firewall rules that I think should allow them to speak.
See my configurations below:
Thank you for any response on this. I know I'm missing something obvious, or perhaps going about it the wrong way. But if I can avoid purchasing a $1200 10gb switch I will be very happy.
-
if you think your going to get wirespeed off a bridge – yeah good luck. Why does our pc have to be on the same network? Just let pfsense route it/firewall it. What does this have to be a bridge?
-
if you think your going to get wirespeed off a bridge – yeah good luck. Why does our pc have to be on the same network? Just let pfsense route it/firewall it. What does this have to be a bridge?
Thanks for responding, I've noticed your username around a lot on the forum helping people out so thank you sir.
In response,
It doesn't have to be on the same network if that is going to cause performance issues. I just thought since a conventional router normally has a WAN port and usually LAN ports 1-4 that are all on the same network that pfsense could be configured much the same.Would not being on the same network affect the ability for this workstation to recognize for instance (Media services on the network / Other Steam Installs / Other network discovery services)?
That said, I am open to placing the workstation on a different network. When you say "let pfsense route it/firewall it" what steps should I take to do that? (Sorry I haven't worked with multiple networks at once before)
Would it be:A) remove the bridge between OPT1 & LAN
B) Configure OPT1 with its own DHCP server? Something on 192.168.2.x ?
C) Add firewall rules to OPT1? (How would they be different than what I already posted a screenshot of?) -
most network cards aren't capable of being a switch (yes some are). No software-router on the planet is currently capable of forwarding 10Gbe wirespeed … this might change at some point.
pfsense maxes out at 2.5Gbe wirespeed, no matter what ultra-core cpu you throw at it.
-
most network cards aren't capable of being a switch (yes some are). No software-router on the planet is currently capable of forwarding 10Gbe wirespeed … this might change at some point.
pfsense maxes out at 2.5Gbe wirespeed, no matter what ultra-core cpu you throw at it.
I'll be happy with 2.5Gbe speeds over 1gbps speeds 312 MB/s will be much better than the 125 I get now during large file copies. For reference, the NICs in the pfsense box are PCIe 10GB Intel NICs
-
At airport on phone but yes just create new network your rules could be any any if u want but pfsense is not your typical soho router that comes with switch ports but i hear future appliance might have some switch ports. Can anwser in more detail when get home
-
If your physical host has at least 3 network cards (and pfSense runs as a virtual machine), do it like this:
- create a virtual switch for WAN, add one physical network card to it (connect it to your internet), and add pfSense's WAN port here
- create a separate one for LAN, add the other two 10GBe network cards to it, add pfSense's LAN port here, and your other virtual machines ports also
- connect your PC to one of the free 10GBe ports.
-
If your physical host has at least 3 network cards (and pfSense runs as a virtual machine), do it like this:
- create a virtual switch for WAN, add one physical network card to it (connect it to your internet), and add pfSense's WAN port here
- create a separate one for LAN, add the other two 10GBe network cards to it, add pfSense's LAN port here, and your other virtual machines ports also
- connect your PC to one of the free 10GBe ports.
That's exactly what I'm doing. Its the configuration in pfsense after the VM stuff has all been configured that is giving me problems.
I haven't been able to test the suggestions on the thread yet, but I will in a few hours once I get off work.
-
If you do what I wrote above, you don't need any bridge in pfSense. "Bridging" is done by the virtual switch, and not by pfSense.
-
I was able to take the advice given on this thread.
My workstation is on a different subnet, but can still talk to both internet and also devices on other networks. There are still a few little issues with getting things to talk correctly, but I'm able to copy files to the server at ~480 MB/s which is pretty dang good. (still need to add a few vdevs to the ZFS pool to see if its pfsense maxing out or if it is my drives.) Much faster than gigabit and I'm happy with it.Edit: Anybody know why when I send to devices From LAN2 -> LAN1 I get 480 MB/s, but when I receive files from from LAN1 -> LAN2 It goes 180 MB/s? Anything I can try to tune to help?
-
I'm sorry if this post is going on too long. I'm just really excited….
As I show in my initial diagram, I'm using PFSense virtualized in ESXi on the 192.168.1.0 network, passing a 10gbe NIC to ESXI and linking up a physical workstation directly into the 10gbe plug on the 192.168.2.0 network in order to avoid having to purchase a costly 10gb switch.
So I have pfsense working as the (layer 3) link between the two networks bridging the gap between the virtualized network and the physical network.
Initially I was getting 480 MB/s sending to the VM network and ~180MB/s downloading from the VM Network.
I just got jumbo frames working across all networks and I'm BLOWN away...
I'm peaking at over 1.0GB/s during file transfers (yes that's 8gbps+!) both up and down with jumbo frames enabled and averaging ~650 MB/s on large file transfers. (And it's probably my 950 PRO or ZFS Array that is capping out, I'll have to load up a RAMDISK and see what I can really push across the link. :)PFSense really is very capable!
edit: For extra LOLs, the 10gb connection is running over CAT5e 350mhz and seems to be doing fine. (because that is all I have in the wall.)
-
This post is deleted! -
@robi said in Issue Bridging OPT1 & LAN for 10GB link into VM Environment:
- create a separate one for LAN, add the other two 10GBe network cards to it, add pfSense's LAN port here, and your other virtual machines ports also
Hi! Sorry for retrieving this thread after few years, but I'm trying now to accomplish this virtual switch on Xen (XCP-NG). However, I can not create any network with more than one physical interface. Is it possible to make this kind of virtual switch on XCP-NG? Thanks!
