Inter vLAN comm doesn't work
-
@itestandroid can they ping the the pfsense IP on their own vlan? Can they ping the IP of pfsense IP of different vlan?
Sure not some odd thing with the masks? Where they think this other vlan IP is on their own network? Do a traceroute from them to say IP of pfsense other vlan your wanting to talk to.. You should see it sent to the pfsense IP in their vlan.
-
@johnpoz They can ping the gateway, yes. They can ping other VMs in same vLAN.
So basically vLAN #1 in netplan:
10.10.50.1/24and vLAN #2 is 10.10.60.1/24
10.10.50.X VMs talk to each other and so does vLAN #2 VMs among each other.
Even Windows guest on vLAN #2 can talk to machines on vLAN #1.
But Ubuntu guests can't. Something is off I think in netplan and routes.
The thing is I see the packets in pfSense. But it stays at SYN_SENT state.
-
@itestandroid that makes no sense..
Any client pointing to pfsense as its gateway trying to ping IP in another In a different vlan, pfsense doesn't care what the client is.. Nor would it even know..
Just so we are clear 10.10.50.1/24 is not a network, that is host address - I take it that is pfsense IP on vlan 1. The network would be 10.10.50.0/24
So saying windows client lets say
so windows box 10.10.60.100/24 can ping 10.10.50.1 (pfsense IP on vlan 1) but unbuntu box at say 10.10.60.101/24 can not ping 10.10.50.1
Where exactly are you seeing this state??
These are both vms? How exactly are these VMS running, what host esxi? What sort of vm networking is setup? You sure ones not on a nat in VM, or different port group on the vswitch. And where is pfsense exactly? Is it another VM, is it the physical word.. Is there any tagging going on with this vlan on the clients, on pfsense, some switch? etc..
-
@johnpoz "that makes no sense"... Tell me about it.
Here are facts:
VM#1 10.10.50.15 Ubuntu, static IP, via netplan, vlan 100
VM#3 10.10.50.20 Ubuntu, static IP, via netplan, vlan 100
VM#5 10.10.50.70 Win10, static IP, via network settings, vlan 100VM#2 10.10.60.10 Ubuntu, static IP, via netplan, vlan 101
VM#4 10.10.60.30 Ubuntu, static IP, via netplan, vlan 101
VM#6 10.10.60.70 Win10, static IP, via network settings, vlan 101So...
VM#1 <-> VM#2 can't communicate
I sent ping or wget/curl command between the two, go to pfSense system logs, states, I can see SYN_SENT state for this ping or wget/curl to 443 port, but doesn't go through.When I ping 10.10.60.1 from VM1 (which points to pfsense even though its gateway4 IP in netplan is 10.10.50.1) ping works, it pings pfsense itself.
But I ping 10.10.60.10 from VM1 which is another VM on another vLAN it doesn't work.
But from Windows VMs on both vLAN everything works. In Ubuntu VMs they can only ping other vLAN's gateway IP/interface, but not VMs under those vLANs
-
@itestandroid said in Inter vLAN comm doesn't work:
But I ping 10.10.60.10 from VM1 which is another VM on another vLAN it doesn't work.
If windows machines in other vlans ping each other.. But a unbuntu box can not ping windows machine? That is different than ubuntu can not ping ubuntu, you could have firewall on ubuntu.
All clients can ping the IP of pfsense in the other vlan - right??
When you ping another device in the other vlan - that host firewall could be blocking the traffic.
If you see syn sent traffic in pfsense state.. But no response that screams firewall, or wrong gateway on device trying to ping..
Simple test.. Sniff on pfsense interface A when client in A pings IP in vlan B.. Do you see the traffic hit pfsense on interface A? Now sniff on interface B on pfsense - do you see that traffic sent on - but no response? If sent on - and no response not pfsense causing you your issues..
-
@johnpoz Yes, Windows VMs can ping every VM from any VLAN.
Sniffing / tcpdump will be a bit challenging since this is a "production" pfsense and so much going on there.
But, here's my netplan example
network:
ethernets:
ens1:
dhcp4: false
addresses:
- 10.10.50.15/24
gateway4: 10.10.50.1
nameservers:
addresses: [10.10.50.1]
routes:
- to: 10.10.60.0/24
via: 10.10.50.1
version: 2It was without the "routes" section, I just recently added that, still same. What else I can check?
All VMs can ping pfSense on their own /24 range IP, meaning
VM with 10.10.50.10 IP, can ping 10.10.50.1, without "routes" above, it couldn't ping 10.10.60.1 (still pfsense), with route, I can ping pfsense on other vlan, other IP range, but not VMs under that vLAN -
@johnpoz
I think I found something super weirdroot@vm1:~# ping 10.50.0.1
PING 10.50.0.1 (10.50.0.1) 56(84) bytes of data.
64 bytes from 10.50.0.1: icmp_seq=1 ttl=64 time=0.461 ms
64 bytes from 10.50.0.1: icmp_seq=2 ttl=64 time=0.294 ms
^C
--- 10.50.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1026ms
root@vm1:~# ping 10.50.0.10
PING 10.50.0.10 (10.50.0.10) 56(84) bytes of data.
From WAN_IP_HERE icmp_seq=1 Destination Net Unreachable
^C
--- 10.50.0.10 ping statistics ---
2 packets transmitted, 0 received, +1 errors, 100% packet loss, time 1002msYou see "WAN_IP_HERE"
It seems like its related to 1:1 NAT for WAN_IP bindings to these VMs and funnily enough Windows VMs don't have IP bindings....
Does this give you any clues?
P.S. Actually... The WAN IP is invalid...
My WAN IP is /25 range. But it's my main first WAN IP minus 1... so instead of 130, its pointing to 129 which I don't own, thats my WAN gateway IP
-
@itestandroid said in Inter vLAN comm doesn't work:
From WAN_IP_HERE icmp_seq=1 Destination Net Unreachable
What is your outbound nat look like in pfsense? You doing a port forward with 1:1 natting should have nothing to do with clients talking to another local IP on some other vlan..
Again I ask if you have any sort of policy route on your rules.. Do you have a gateway set?
These clients that show that wan IP here - do they have more than their single 10.50.0.X IP?
-
@johnpoz No, they have 1 single vLAN internal NAT IP and in pfSense I do 1:1 to give them WAN IP as well so I can SSH into them remotely.
But I shouldn't see my WAN gateway IP erroring out in PING from inside that VM, that's really weird and strange
As for route policy, as far as I know I only have 1:1 NAT, that's it. Anything else I should check?
-
@johnpoz
As for Outbount NAT,I have a bunch of auto generated stuff that I can't delete/edit.
Then I have nothing manually added there in Mappings. Should I have some stuff in there?
-
@itestandroid said in Inter vLAN comm doesn't work:
see my WAN gateway IP erroring out in PING from inside that VM
Yeah trying to wrap my head around that myself.. hmmmmm? Maybe there is something with the 1:1 nat I am not understanding.. Have to play around with it on a VM and see if can duplicate your problem.
But in general - doing a 1:1 nat from public IP to rfc1918 IP should have zero to do with that rfc1918 address talking to some other local rfc1918 on a different vlan of yours..
-
@johnpoz I can even give you TeamViewer access if that's possible instead of you setting things up. It's just weirdest thing to me. No matter what NAT reflection method I chose, no matter what I did with NAT rules, I can't get VMs to ping internal IPs on other vLANs. It goes to gateway.... and errors out
-
@itestandroid maybe we can set that up tmrw if that is ok with you... I want to watch the new dave chappelle special on netflix.. And then should be about time for the MLB playoff game to start.. Lets go yanks - have money on them! Love that sports betting is legal - makes baseball exciting ;)
-
@johnpoz Sure, I'm around. NY Yankees FTW!
Just one question till tomorrow, I have an automatically generated rule:
127.0.0.0/8 ::1/128 192.168.1.0/24 10.10.50.0/24 10.10.20.0/24
in source and source port, destination and destination port is * and NAT Address is pointing to "WAN address" and its for all ports, not static port.Is this normal/OK?
-
@itestandroid that is just normal auto outbound nat, like this..
Those should be all your networks/vlans and tunnel networks for say openvpn, etc.
or do you mean somewhere else?
I have an automatically generated rule:
Picture always worth 10k words if you ask me.. What gets me thinking your not talking about outbound nat is ""WAN address"" ?? That would be like a port forward?
