Proxmox virtualization: multiple subnets, cisco switch, ssh sessions closed
-
Hi,
I'm trying to reach the following installation (proxmox 7.3).
I'll have more networks/vlans (for proxmox clustering, storage, ...) but keep it simple for the moment.Switch: Cisco Nexus 3172tq, all access ports on vlan10, vlan20 or vlan100. Just L2 vlans.
WAN comes into proxmox eno0, bridged to pfsense WAN.
Other three ports on proxmox (bridged to pfsense as well) are connected to vlan10, vlan20 and vlan100 ports on the switch. No vlans in proxmox/pfsense at the moment.
pfSense rules permit from vlan10 to vlan100, blocking everything else (vlan20 will be isolated).Figure:
Problem 1:
Is it ok not to use vlans in pfsense/proxmox? I do not use trunks and should not need them, I just need port segmentation on the switch. Or should I enable vlans in pfSense? How to manage them without doing port trunk with vlan1 and another single vlan on the switch?Problem 2:
If I put a PC on vlan10 and another in vlan100 on the switch, I can ping and ssh one another, freely, as per pfSense rules.
If I try to ssh into proxomox from vlan10 to vlan10, everything works great as well.
But if I try to ssh into proxmox from vlan10 to vlan100, I can ping, but the ssh session gets closed after just some seconds. vlan20 is properly isolated.
What could be happening for inter-vlan ssh to proxmox getting screwed while inter-vlan ssh outside works, and in-vlan ssh to proxmox works as well?Thank you very much.
-
@urbaman75 said in Proxmox virtualization: multiple subnets, cisco switch, ssh sessions closed:
I do not use trunks and should not need them, I just need port segmentation on the switch.
Checkout the Chapter: Configuring Private VLANs of the switch docs.
-
@viragomann I know pvlans, and could be a viable way to properly segment and isolate vlans.
I was going with vlans just to be able to manage inter-vlan traffic from pfSense firewall (some of them should be intercommunicating, or be accessed from say a few cross-vlan IPs, or even different rules)Still can't understand the second problem, where connecting to proxmox from a different vlan (10 to 100 for example) stumble in the connections being closed in seconds, but only for connections to the host and from a different vlan
-
@urbaman75 said in Proxmox virtualization: multiple subnets, cisco switch, ssh sessions closed:
I know pvlans, and could be a viable way to properly segment and isolate vlans.
I was going with vlans just to be able to manage inter-vlan traffic from pfSense firewall (some of them should be intercommunicating, or be accessed from say a few cross-vlan IPs, or even different rules)This does exactly what you try to achieve. As you said, you have connect each of your "vlans" to a separate interface on the Proxmox host.
So if your switch provides to divide it into multiple virtual switches, why do you want to tag the vlans? Just assign, say port 1 to 4 to virtual switch 100 for you management lan and connect one port of them to pfSense and the other to the other devices in the management subnet. And so on.Still can't understand the second problem, where connecting to proxmox from a different vlan (10 to 100 for example) stumble in the connections being closed in seconds, but only for connections to the host and from a different vlan
As I got your setup, Proxmox is multi-homed. It has an IP in vlan10 and also one in vlan100.
This will result into asymmetric routing issues. The request goes from vlan10 though pfSense out to vlan100 to the Proxmox IP. But Proxmox sends its response back to the source IP, which is vlan10 and which it has an interface IP within. So the response goes back the "short way", out on the vlan10 interface and hence doesn't pass pfSense.This may work for ICMP packets (ping), since they not statefull, but TCP traffic fails.
So you should remove the vlan10 IP from Proxmox.
-
@viragomann said in Proxmox virtualization: multiple subnets, cisco switch, ssh sessions closed:
As I got your setup, Proxmox is multi-homed. It has an IP in vlan10 and also one in vlan100.
This will result into asymmetric routing issues. The request goes from vlan10 though pfSense out to vlan100 to the Proxmox IP. But Proxmox sends its response back to the source IP, which is vlan10 and which it has an interface IP within. So the response goes back the "short way", out on the vlan10 interface and hence doesn't pass pfSense.
This may work for ICMP packets (ping), since they not statefull, but TCP traffic fails.
So you should remove the vlan10 IP from Proxmox.Ok, so this is a perfect explanation. I'll give Proxmox only the management IP (on 100) and the single IPs it needs on the "internal" vlans (proxmox cluster, ceph cluster...) that do not need inter-vlan communication.
Or, I will not try to do cross-vlan to proxmox anyway, only in-vlan.Thank you very much.
-
@viragomann I can confirm your diagnosys, removing the "10" IP, now the sessions from external "10" to "100" work and are stable.
Thank you very much again.
