[SOLVED] Cannot ping devices in other VLAN

CodeNinja · May 19, 2020, 1:20 PM

I (currently) have 2 networks. Each network has its own VLAN. These networks need to be able to cummunicate with eachother as
one of the networks contain only servers and the other one contains only end user devices (notebooks, desktops and smartphones).

In future we will ad more networks like a DMZ. There will also come a OpenVPN access server. Users connected via VPN will have access to one or more networks depending on their permissions.

The VLANS i defined for this example is

VLAN 10 - 192.168.10.0/24
VLAN 20 - 192.168.20.0/24

Some findings:

Devices from both VLANs are able to access the internet with the "automaticly generated" NAT rules.
Devices are able to ping devices in the same network
Devices cannot ping devices in the other network (VLAN)
Both PC's are able to ping the trunk port of both VLAN's

This diagram sketches the findings above:

I have only defined 2 firewall allow * from * to * rules, 1 for each VLAN.
For VLAN10:

And for VLAN20:

I have no "floating rules" configured.

How can i achieve that both computers (PC1 and PC2) can communicate with each other. Means what setting/configuration do i miss to be able to access SSH and web interfaces in the other VLAN/network.

DaddyGo · May 19, 2020, 2:30 PM

Help me understand exactly what you want (inter-VLAN routing or ???)

The point of this is that VLAN..., we are talking about VLAN, so we separate it.....
why you create VLANs, if you want them to be full interoperable?
A VLAN is a way to segment our network using, say with manageable switch as, if they were physically separate

more recently, but a great option for a grand network design

more simply: why would you want to ping a unit in another VLAN

the rule which is you posted is completely incomprehensible? what did you mean then...

CodeNinja · May 19, 2020, 2:49 PM

@DaddyGo
Thanks for your reply.

It not about pinging, its more about communication between those 2 VLANs at all and i testing the connection with ping. In real situation i need to access SSH and web interfaces on the other VLAN. I think inter-VLAN routing may be the correct term. (sorry i'm not a specialist in networking, i'm actually a software engineer)

There are multiple reasons why i think we should use multiple VLANs. Some of them should be able to exchange information or services with other VLAN's.

One of the networks contains only end user devices and the other one only servers. There will me more VLAN's in future which are not allowed to "communicate" with devices in all VLANS. E.g. end user devices are allowed to access devices in the server VLAN but not in the test VLAN.
There will be an OpenVPN access server. Depending on the user permissions, a user is able to "access" some networks (subnets). I read that its bad practice to have multiple subnets inside 1 physical lan.
We will also have a DMZ which will become a separated VLAN. In the DMZ we will have simple web servers which have no connection with servers in the "server VLAN" but also servers which should be able to access some "backend" services (only specific ports) in the server VLAN (e.g. a DB server).

johnpoz · May 19, 2020, 2:52 PM

@CodeNinja said in Cannot ping devices in other VLAN:

but i suppose i missed something?

Prob the most often missed thing is host firewall/security software that blocks traffic from the other network..

I would say this is number 1 issue, or client not using pfsense as gateway..

Why is your switch in your drawing screaming layer 3, yet from your description your just using it as layer 2.. Ie its not routing..

Can you ping pfsense IP in the other vlan from client?

Example can client in vlan 10, ping pfsense IP in vlan 20, I would guess 192.168.20.1 ?

If so then sniff on vlan 20 interface.. Now ping something in the 20 vlan from client in vlan 10, say 20.42 or whatever an active machine IP is in that vlan.. Now does your sniff show that pfsense set the traffic - if so then its something on that devices "host firewall" or it never got there after it left pfsense. But pfsense did its part..

CodeNinja · May 19, 2020, 2:54 PM

@johnpoz
Thanks for your reply.

Maybe its not 100% clear for me whats the difference between a layer 2 and 3 switch. I thought the only difference was that one of both supports VLANs and the other doesn't. Sorry when it should be a layer 2 switch in the diagram. I will change/update it tomorrow.

I already found in the internet that in some cases the host blocks trafic from another network. For that reason i already tried to ping the pfsense port in the other network as you asked before i posted the question and NO its not possible. So i think it has nothing to do with the host firewall(s).

johnpoz · May 19, 2020, 3:03 PM

Well if you can not ping pfsense IP in the other vlan.. Then yeah you have something wrong.

Post your rules for the vlan your pinging from - you sure you didn't leave it on tcp.. .This would block ping..

Layer 3 switch can route, vlans are layer 2..

If you wanted to call out that your switch was doing vlans, you could label say smart ;) Or managed..

I take it your clients in each vlan are getting ips from pfsense, and can use the internet through pfsense? They can not just ping each others network? Your rule you showed was not doing any policy routing - ie forcing traffic out out a gateway, which would prevent vlans from talking to each other.. But you only showed 1 vlan.. Do you have any floating rules?

All that is required for device to route through pfsense.. Is for the networks to be setup in pfsense, and the firewall rules to allow the traffic...

DaddyGo · May 19, 2020, 4:09 PM

@CodeNinja
There are plenty of differences between purely Layer2 and Layer3 switches, for example there is one such terminology including L2+ or L3+.
Basically, L2 + or L3/L3+ is also capable of some basic routing. (static route, the L2+ switch adds some L3 features. For example: DHCP snooping, ACL rule according to IP address.
The L3+ switch adds some L4 features. For example, you can classify traffic according to socket number.

It is necessary to understand these concepts first and then to deal with VLANs,
although you described perfectly well, what you would like to apply in your example

CodeNinja · May 20, 2020, 7:30 AM

@johnpoz said in Cannot ping devices in other VLAN:
Thanks for the patience and support.

I updated/re-wrote my question and included my most recent findings - as i'm able to ping the pfsense interfaces also from the other VLAN since this morning (i have no idea why this didn't work yesterday). I did not change anything to the firewall rules (except for the description). I had (and still have) 2 firewall rules allow * from * to * on both VLANs.

I hope my re-written question is more clear for those who want to help me but also for people who run into a similar issue i have.

Layer 3 switch can route, vlans are layer 2..
If you wanted to call out that your switch was doing vlans, you could label say smart ;) Or managed..

Thanks will keep this in mind for future questions

I take it your clients in each vlan are getting ips from pfsense, and can use the internet through pfsense? They can not just ping each others network? Your rule you showed was not doing any policy routing - ie forcing traffic out out a gateway, which would prevent vlans from talking to each other.. But you only showed 1 vlan.. Do you have any floating rules?

Ok, this sounds like it could be my problem. I only have the 2 mentioned rules and if i understand well, these rules do no "policy routing" which is required for communication over VLANs or different networks ? How do i configure a rule with "policy routing", simply by give an explicit source and destination?

All that is required for device to route through pfsense.. Is for the networks to be setup in pfsense, and the firewall rules to allow the traffic...

Thats also what i found on the internet. Both networks are setup in pfsense for 100% sure so that shouldn't be the problem so the rules are the only thing left... I thought a "wild card" rule should always work but it doesn't ? (at least not in this case?)

CodeNinja · May 20, 2020, 7:45 AM

I can ping devices in the other VLAN now with the settings from the question!

As i was able to ping the the Pfsense interface in VLAN 20 from the host in VLAN10 this morning i tried to ping the host in VLAN20 as well. As this failed i started checked what @johnpoz said:

Prob the most often missed thing is host firewall/security software that blocks traffic from the other network..
I would say this is number 1 issue, or client not using pfsense as gateway..

After disabling the firewall on the host, it was possible to ping the device in the other network.

I'm currently not able to SSH to the device in VLAN20 so i will continue digging and post updates here.

johnpoz · May 20, 2020, 9:04 AM

@CodeNinja said in Cannot ping devices in other VLAN:

I thought a "wild card" rule should always work but it doesn't ?

Those rules would work! They are full any any rules...

With those rules - you would be able to access ssh or http if the dest devices allowed it... Again simple enough to just sniff..

If you can ping the device, there is zero reason with those rules listed why you wouldn't be able to ssh.

CodeNinja · May 20, 2020, 9:56 AM

@johnpoz said in Cannot ping devices in other VLAN:

If you can ping the device, there is zero reason with those rules listed why you wouldn't be able to ssh.

I thought that also but its not possible to SSH.

I setup a new machine (PC3) in VLAN20. I can SSH from PC3 to PC2 but i can't from PC1 (VLAN10).
In my experience this is wierd as i can ping from PC1 to PC2. All PC's are clean ubuntu 20.04 installations with a disabled firewall see:

root@pc2:# ufw status
Status: inactive

This let me think that it is still a pfSense configuration thing but i'm not sure.. How can i find this out?
Do i need something like NAT configuration to be able to

johnpoz · May 20, 2020, 10:07 AM

@CodeNinja said in Cannot ping devices in other VLAN:

How can i find this out?

As already stated.. .sniff on your dest vlan interface in pfsense.. Now generate traffic from vlan 10.. Do you see pfsense send that traffic?

Here.. sniffing on my dmz interface 192.168.3, I sshed to a box from my 192.168.9 network (lan) you see the traffic..

do you see that first syn get sent on by pfsense?

If you see pfsense send the syn on to your dest IP.. Pfsense did what it was suppose to do, if you don not see an answer that is on the dest device not answering, or not getting what pfsense sent on.

Unless your running some IPS on pfsense, that could be blocking the traffic.. All that is required is the firewall rule to allow the traffic you want. Which your any any rule would do.

Do i need something like NAT configuration to be able to

NO!!! you would not be natting traffic between local vlans.. Did you dick with outbound nat?

CodeNinja · May 20, 2020, 10:15 AM

@johnpoz
After a reboot of PC2 it works fine now. I can ping and SSH.

THANKS A LOT for the support. Is there a way i can do something back?

p.s. the sniffing example/explanation is helpful other situations i'm working on so also thanks for that!.

johnpoz · May 20, 2020, 10:18 AM

@CodeNinja said in [SOLVED] Cannot ping devices in other VLAN:

Is there a way i can do something back?

Pay it forward ;) If you can help someone else here - that is always good for the community.. Or help someone else in some other way if you can.

Glad you got it sorted.