[Solved] With all the Server NICs active, I lose the connection in LAN
-
I don't know if the problem is pfSense, but now I'm trying everything.
I have a server with 3 NICs, connected to 3 switches. There are no VLANs.
On these 3 switches there are connected the 3 network cards of pfSense that provides 3 DHCP.
To the server I connect via SSH with a PC in LAN to the NIC of the server corresponding to DMZ #3.
After about a minute I lose the connection.If I disable the server NIC on network # 1, the problem is not there.
I tried to replace the switches and also to swap the cables, but the problem returns. So it is not related to server NICs .If instead I connect to the server with a GUI, I lose the connection, but then it is restored and then I lose it again after a short time. So, it is therefore an intermittent problem.
As I said I don't know if the problem is of pfSense, but I don't find errors neither in the Syslog of the server, nor in the switches (which I have replaced anyway).
After several tests I try to imagine that the problem is due to an incorrect configuration of pfSense. On the other hand the problem is intermittent and so it shouldn't be a wrong rule.Thanks in advance for any help.
-
You mention pfSense - and a server.
pfSense typically uses a NIC that acts as a WAN interface.
A second NIC that acts as a LAN - this one has the DCP-server activated by default.
A third NIC, initially called OPT1, has no firewall rule, and no DHCP server. You have to put them in place yourself.@whitetiger-it said in With all the Server NICs active, I lose the connection in LAN:
On these 3 switches there are connected the 3 network cards of pfSense that provides 3 DHCP
This is strange.
The WAN uses DHCP-client (and not a DHCP-server) - if you use that WAN connection method.Btw : pfSense doesn't 'serve' and isn't a server. It's a router/firewall.
-
@whitetiger-it said in With all the Server NICs active, I lose the connection in LAN:
I have a server with 3 NICs, connected to 3 switches. There are no VLANs.
After about a minute I lose the connection.
I try to imagine that the problem is due to an incorrect configuration of pfSenseWhat screams out to me with a device connected to multiple networks is Asymmetrical traffic..
Multi-homing a device almost always leads to asymmetrical traffic..
So you have some client in network A, and wanting to talk to IP in network B.. This is routed through pfsense. State is created when pfsense see the syn. But device in B says - oh I am on that network, I will just send the syn,ack directly to that IP.
At some point pfsense is going to say.. I see no answers back from this state I created - going to close it via timeout. Now your connection drops.
Multi-homing devices almost always a bad idea, and unless you fully understand the complexity and security issues this brings.. You shouldn't be doing it.. The funny thing is - if you do understand them, you would know how bad of an idea it is - and not do it ;)
If you are going to start a conversation with a multi-homed device. When you want to talk to it from a device that is in one of its multiple networks, then use the devices IP in that same network to talk to it.
Vs using network B ip to talk to the device, use its network A address from device in network A - now you never run into asymmetrical problem with a stateful firewall/router
-
pfSense has 4 NICs: 1 WAN, 1 LAN, 2 DMZ.
I was referring to these last 3 NICs. -
@whitetiger-it you could have 100 nics and networks or just 2 networks.. Makes no difference.. You have stated you have a server with connection in multiple networks.. If half the traffic flows through pfsense and the reply does not - then your going to have problems with any stateful firewall be it pfsense or any other..
-
I get it, and maybe in going forward in the tests I inserted one more problem.
So I take a step back.The server is with Debian + Proxmox to create and manage Virtual Machine.
The starting point was this:source /etc/network/interfaces.d/* auto lo iface lo inet loopback iface eno1 inet manual iface eno2 inet manual iface eno3 inet manual iface eno4 inet manual iface ens1f0 inet manual iface ens1f1 inet manual auto vmbr1 iface vmbr1 inet static address 192.168.201.31/24 bridge-ports eno1 bridge-stp off bridge-fd 0 #Bridge LAN auto vmbr3 iface vmbr3 inet static address 192.168.203.31/24 gateway 192.168.203.1 bridge-ports eno3 bridge-stp off bridge-fd 0 #Bridge DMZ3Here there is no longer the multi network, but the intermittent blocks were there anyway.
-
This is the present configuration now, with the interruptions still.
source /etc/network/interfaces.d/* auto lo iface lo inet loopback iface eno1 inet manual iface eno2 inet manual iface eno3 inet manual iface eno4 inet manual iface ens1f0 inet manual iface ens1f1 inet manual auto vmbr1 iface vmbr1 inet static address 192.168.201.31/24 gateway 192.168.201.1 bridge-ports eno1 bridge-stp off bridge-fd 0 #Bridge LAN auto vmbr2 iface vmbr2 inet static address 192.168.202.31/24 gateway 192.168.202.1 bridge-ports eno2 bridge-stp off bridge-fd 0 #Bridge DMZ2 auto vmbr3 iface vmbr3 inet static address 192.168.203.31/24 gateway 192.168.203.1 bridge-ports eno3 bridge-stp off bridge-fd 0 #Bridge DMZ3 -
@whitetiger-it how do you expect to bridge multiple networks ranges to the same network interface?
-
@johnpoz
Why the same interface? I do not understand.vmbr1 is on eno1
vmbr2 is on eno2
vmbr3 is on eno3The addresses are also different
192.168.201.31/24
192.168.202.31/24
192.168.203.31/24 -
This is the simplified representation of the network I am implementing.
A physical PC in the LAN must be able to connect to a Host Proxmox server in DMZ3.
Inside this server there are 3 virtual networks corresponding to the LAN, DMZ2 and DMZ3.
Obviously, the PCs in the LAN (whether physical or virtual) must be able to connect to the PCs and Servers in the DMZ2 and DMZ3.
Instead:- the virtual PCs present in the DMZ2 must be able to connect only to the servers of the DMZ2.
- The virtual PCs present in the DMZ3 must be able to connect only to the servers of the DMZ3.
source /etc/network/interfaces.d/* auto lo iface lo inet loopback iface eno1 inet manual iface eno2 inet manual iface eno3 inet manual iface eno4 inet manual iface ens1f0 inet manual iface ens1f1 inet manual auto vmbr1 iface vmbr1 inet static address 192.168.201.31/24 gateway 192.168.201.1 bridge-ports eno1 bridge-stp off bridge-fd 0 #Bridge LAN auto vmbr2 iface vmbr2 inet static address 192.168.202.31/24 gateway 192.168.202.1 bridge-ports eno2 bridge-stp off bridge-fd 0 #Bridge DMZ2 auto vmbr3 iface vmbr3 inet static address 192.168.203.31/24 gateway 192.168.203.1 bridge-ports eno3 bridge-stp off bridge-fd 0 #Bridge DMZ3 -
I solved by removing both the gateways and the IP addresses of vmbr1 and vmbr2
Now I finally have no more interruptions and the VMs are still able to access the Internet.
