How to set the same VLANs between the switch and PfSense

SteveITS

@jt40 said in How to set the same VLANs between the switch and PfSense:

with a mask of 255.255.255.0 I can still set up 192.168.5.1 and 192.168.9.1 networks

255.255.255.0 means "the first three numbers must match." So:

192.168.0.1 / 255.255.255.0
is in a different network than
192.168.1.1 / 255.255.255.0

but
192.168.0.1 / 255.255.0.0
is in the same network as
192.168.1.1 / 255.255.0.0

So the mask matters but they matter together.

So if you tell pfSense WAN to use 192.168.0.1 / 255.255.0.0 then it knows all those addresses are WAN. You can't also set up LAN as 192.168.5.1 / 255.255.255.0 unless you change the mask on WAN.

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

https://avinetworks.com/glossary/subnet-mask/

from that
"The “255” address is always assigned to a broadcast address, and the “0” address is always assigned to a network address"

Wow!! that is just plain wrong!

If I had a network 192.168.0.0/23 192.168.0.255/23 would be just a host address, and 192.168.1.0/23 would also just be a host address.

Why do they bring up Classes - network classes haven't been a thing since early 1990s..

If I used a mask of say /25 on 192.168.0.0 the broadcast would be 192.168.0.127 not .255, with that same mask the next network 192.168.0.128/25 would be the network..

Such a statement would only be valid for for specific mask that use up the whole octet, like /8 or /16 or /24 - again back to the class days, which hasn't been a thing for going on 30 years.

or the mask doesn't define the network private/public type?

The mask has nothing to do with the network being public or private. What makes it public or private is the address range used..

rfc1918 or private space is spelled out here
https://datatracker.ietf.org/doc/html/rfc1918

10.anything is a private space 10.0.0.0/8 which is the range 10.0.0.0 - 10.255.255.255, where yes .0 is the network or wire and 10.255.255.255 would be the broadcast. but say 10.90.90.0/8 would be just a host, and 10.0.0.255/8 also just a host..

This would prob be a better place to start
https://docs.netgate.com/pfsense/en/latest/network/addresses.html

I remembered the classes as the new cool thing :D , you make me feel old now ahaahah.

I didn't understand this, was it a typo in the end?
How both can be hosts?? Is it an exception of that address range defined in some RFC?

10.anything is a private space 10.0.0.0/8 which is the range 10.0.0.0 - 10.255.255.255, where yes .0 is the network or wire and 10.255.255.255 would be the broadcast. but say 10.90.90.0/8 would be just a host, and 10.0.0.255/8 also just a host..

Thank you again :)

JT40

@steveits said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

with a mask of 255.255.255.0 I can still set up 192.168.5.1 and 192.168.9.1 networks

255.255.255.0 means "the first three numbers must match." So:

192.168.0.1 / 255.255.255.0
is in a different network than
192.168.1.1 / 255.255.255.0

but
192.168.0.1 / 255.255.0.0
is in the same network as
192.168.1.1 / 255.255.0.0

So the mask matters but they matter together.

So if you tell pfSense WAN to use 192.168.0.1 / 255.255.0.0 then it knows all those addresses are WAN. You can't also set up LAN as 192.168.5.1 / 255.255.255.0 unless you change the mask on WAN.

Thank you.
So this means that every interface on PfSense needs to be on the same address range and configured with the same mask, right?
Regardless of uplink or other LANs.

At that point, how do I route packets from/to 192.168.0.1 to 10.90.90.130 for example?
Shall I create a gateway? That would be a virtual gateway in PfSense.

What would be its IP then?? I ask because the interface asks me for an IP address, but I don't want to create only 192.168.0.x networks/interfaces, I actually need to route down the switch on a network with range 10.90.90.x (I think I can even change that, but let's make it work as by default for now, just to keep things easy)

SteveITS

@jt40 said in How to set the same VLANs between the switch and PfSense:

So this means that every interface on PfSense needs to be on the same address range and configured with the same mask, right?

No. They just can’t overlap.

At that point, how do I route packets from/to 192.168.0.1 to 10.90.90.130 for example?

PfSense will know where it’s own interfaces/networks are. You just need to create firewall rules to allow traffic. Only LAN has an allow to any rule.

Re:.0/.255 …networks can have masks like 255.255.255.128 or other masks.

JT40

@steveits said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

So this means that every interface on PfSense needs to be on the same address range and configured with the same mask, right?

No. They just can’t overlap.

At that point, how do I route packets from/to 192.168.0.1 to 10.90.90.130 for example?

PfSense will know where it’s own interfaces/networks are. You just need to create firewall rules to allow traffic. Only LAN has an allow to any rule.

Re:.0/.255 …networks can have masks like 255.255.255.128 or other masks.

No. They just can’t overlap.

Do you mean they can't overlap by IP address?

PfSense will know where it’s own interfaces/networks are. You just need to create firewall rules to allow traffic. Only LAN has an allow to any rule.

So you mean that nothing will stop the traffic by default between LANs on the same PfSense device? And no virtual gateway is needed?

So If I assign the following IPs to different interfaces like:

INET 1 190.168.0.4 ( Web GUI management interface, even though I'd like that IP on another range if possible)
INET 2 192.168.0.5 (random usage)
INET 3 UPLINK 192.168.0.6 (how does it know that it's UPLINK?)
INET 4 DOWNLINK to the switch 10.90.90.120 (the default IP of my switch is 10.90.90.90, but that should be reserved to the Web GUI)

Is this supposed to work?

I'll try this evening.

SteveITS

@jt40 By "can't overlap" I mean you can't use 192.168.0.1/255.255.0.0 on WAN, and 192.168.5.1/255.255.255.0 on LAN because that LAN is inside that WAN network. Each interface has to be a unique network/subnet.

Your example INET1-4 will not work because 1, 2, and 3 are not separate networks, assuming they each have a 255.255.255.0 (/24) mask.

JT40

I did many tests, this is what I noticed:

The auto rollback function seems working, the process exits and I also rebooted the machine to be sure, but I can't login anymore even if I rolled back prior the firewall changes. The result is that I can't connect anymore to the box with the WebGUI... The connection suddenly dropped, I think it's the firewall but I strongly don't think it's related to my lack of knowledge or my mistake, I made a rollback before that change and it didn't change anything... I also check my laptop config to be sure, IP and Gateway were correct...
The firewall is slow time to time to make simple changes, like to enable an interface, or modify an existing interface IP, sometimes it takes 1 second or 20 seconds... This wide range of time doesn't make sense, plus the HW that I have is overkill...

Are you aware of the issues above? Or it is just me? I can also blame myself but all this story seems weird to me...

I still can't make it work :D

The last test was done with the current config.

Modem/router 192.168.0.1 - mask 255.255.255.0
PFSense - Uplink interface (unique IP and different network) 192.168.0.220 - mask 255.255.255.0 (I was not able to ping the modem/router with an IP of 192.168.3.220 for example, so I different network can't ping the next network node for some reason, which is the modem/router, the one that is connected to the real WAN...)
No other devices in the downstream network, unless my laptop but that's on a different interface with a different range, something like 192.168.2.1

I was able to ping the modem/router from the pfsense box, but not any public IP address...
I don't have firewall rules as such on the modem/router, so I don't understand why I don't succeed with it...
I only succeeded randomnly in the past days, but I don't really know the details of old tests as of now, it was just luck I guess...

The first part of my setup is to be able to ping a public IP at least, so I know that at least the pfsense box can reach internet, then I start to think about other devices.

SteveITS

@jt40 said in How to set the same VLANs between the switch and PfSense:

I was able to ping the modem/router from the pfsense box, but not any public IP address...

Is the gateway correct on the pfSense WAN? (should be 192.168.0.1 if that's the upstream router) What does a traceroute show? (which is also on the diagnostics menu)

I think I would start with one WAN and one LAN on the pfSense and add other interfaces after you get it working. Just disable the other interfaces for now.

johnpoz

@steveits said in How to set the same VLANs between the switch and PfSense:

I think I would start with one WAN and one LAN on the pfSense and add other interfaces after you get it working

exactly - I suggested this way back when in this thread ;) And gave a basic diagram with IPs and masks, etc.

JT40

@steveits said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

I was able to ping the modem/router from the pfsense box, but not any public IP address...

Is the gateway correct on the pfSense WAN? (should be 192.168.0.1 if that's the upstream router) What does a traceroute show? (which is also on the diagnostics menu)

I think I would start with one WAN and one LAN on the pfSense and add other interfaces after you get it working. Just disable the other interfaces for now.

This is how it was after the restart, I kept the maintenance LAN plus the uplink (WAN, but it's always a LAN port because it's not the first entry point).

That's how I tested it, I can do it again with traceroute but I need to restore it again, give me some time...

Is the gateway correct on the pfSense WAN? (should be 192.168.0.1 if that's the upstream router)

What do you mean? There is no wateway assigned, as the menu mentions as well I should not assign a gateway if that's a LAN to LAN communication. PfSense is not connected to the ISP, the modem/router that I have it is...
We discussed here previously about it and I was told do not use a gateway on the UPLINK port, the one you called WAN is actually a LAN UPLINK port. It's also mentioned in the description in PfSense.
@johnpoz did I understand correctly? I think it was you to have made me notice it :D

johnpoz

@jt40 said in How to set the same VLANs between the switch and PfSense:

What do you mean? There is no wateway assigned, as the menu mentions as well I should not assign a gateway if that's a LAN to LAN communication

What?? Where pfsense connection to your isp device is its WAN!!! That is how it gets to the internet.. No shit it would never work without a gateway..

Again going to state this with emphasis - you need to research the basic concepts of how a router works, what a network is..

I clearly went over this..

Without a gateway - where would pfsense send traffic that was destined for some network its not directly attached too??

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

What do you mean? There is no wateway assigned, as the menu mentions as well I should not assign a gateway if that's a LAN to LAN communication

What?? Where pfsense connection to your isp device is its WAN!!! That is how it gets to the internet.. No shit it would never work without a gateway..

Again going to state this with emphasis - you need to research the basic concepts of how a router works, what a network is..

I clearly went over this..

Without a gateway - where would pfsense send traffic that was destined for some network its not directly attached too??

192.168.0.1 and 192.168.0.220 are in the same network for what I know with a mask of 255.255.255.0, the communication should happen automatically, it's like 2 hosts in the same VLAN, don't tell me what I'm wrong even here :D ...
Obviosuly I consider this case correct without firewall rules blocking the traffic.

What it may not be obvious for PfSense is how to take the traffic from other interfaces and route it to the UPLINK, in that case I think a gateway is needed, and it should be the PfSense Gateway, like 192.168.0.220 (current UPLINK port), or any custom IP if it is possible, I need to check. I strongly don't think is the same IP of the WebGUI.

Regarding the graph, maybe I confused your previous graph, where you mentioned NO GATEWAY between the PfSense box and the downstream switch, that's another story.

johnpoz

@jt40 said in How to set the same VLANs between the switch and PfSense:

the communication should happen automatically, it's like 2 hosts in the same VLAN, don't tell me what I'm wrong even here :D ...

OMG dude.. Really? Yes they would be able to talk to each other - but why would pfsense send traffic for say 8.8.8.8 to 192.168.0.1 unless it was set as the gateway, ie the default route for pfsense.

Where do you have pfsense sending traffic this is not local? How would anything get to the internet if not sent to your ISP device? What is pfsense default gateway set too?

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

the communication should happen automatically, it's like 2 hosts in the same VLAN, don't tell me what I'm wrong even here :D ...

OMG dude.. Really? Yes they would be able to talk to each other - but why would pfsense send traffic for say 8.8.8.8 to 192.168.0.1 unless it was set as the gateway, ie the default route for pfsense.

Where do you have pfsense sending traffic this is not local? How would anything get to the internet if not sent to your ISP device? What is pfsense default gateway set too?

That was my initial guess :D
I think I just got confused about the previous sentence, where you said a gateway it was not needed, but actually you were referring to the pfsense --> <-- switch communication, where I also have truncate ports with different VLANs on the same port.
I'll test it and let you know

JT40

Ok the gateway was all about it, what a bs...... :D

Now I have another problem, which makes tiredesome every troubleshooting or setup I do...

Real world example:

Management interface:

IP set as 192.168.2.1
mask 255.255.255.0 or /24
no DHCP whatsoever
The connection from my laptop works

Management interface change

I changed the interface IP to 192.168.20.1, mask 255.255.255.0 or /24 (I can't connect anymore, I'm using the same port and I reconfigured my laptop in this way):

laptop IP 192.168.20.2
gateway 192.168.20.1 (nothing else than the interface IP)
no DHCP whatsoever

Revert the change

I reverted the change directly from the machine, basically assigning the IP 192.168.2.1 to the same interface, no DHCP again, nothing specific.
Yes, I tried to hit 192.168.2.1 :D from the WebGUI

I can't connect anymore, normally I would expect my laptop to be the problem, specifically in this case.
With "ifconfig" I see that the IP is correctly assigned, but I can't see the gateway and it's normal so far...
I checked the respective network config file under /etc/sysconfig/network/scripts/ , well, it's not there... Actually many of them are missing there.... The only thing I noticed is that there are few wireless and wired connections, so not only one of this type, but that's it, what I need is not there.

Even after I re-created the network config in the laptop UI, the config is not in that directory...
Also, this file is empty: /etc/sysconfig/network , and it shouldn't

Looking at this doc, these 2 things mentioned above should be there... https://docs.fedoraproject.org/en-US/Fedora/14/html/Deployment_Guide/ch-Network_Interfaces.html
The doc is a bit old but I don't think they made changes on how this component works...

In the end, I found the config file under /etc/NetworkManager/system-connections/ , it contains all the correct details, even though I found strange this line:
[ipv4]
address1=192.168.2.1/24,192.168.2.1

As you can see, the gateway is on the same line of the machine IP address... I think it's weird but looking at other files it seems correct.

PfSense system reboot it didn't help...

I'll configure a new interface, but this looks a bit crazy.

I exausted my ideas, let me know what you think :D

johnpoz

@jt40 what rules did you put on pfsense "management" interface you created?

If you had set the source to 192.168.20/24 or something sim at first and then changed the IP, this rule would not have changed.

The rule on the management interface your creating should prob be source "management net". This way when you change the management IP on pfsense, the alias for "management net" would change to reflect the network network.

When troubleshooting local connectivity like laptop plugged directly into pfsense port, or even via a switch with devices on the same network.

First thing to validate is you can arp for the IP your trying to talk to on the same network.

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 what rules did you put on pfsense "management" interface you created?

If you had set the source to 192.168.20/24 or something sim at first and then changed the IP, this rule would not have changed.

The rule on the management interface your creating should prob be source "management net". This way when you change the management IP on pfsense, the alias for "management net" would change to reflect the network network.

When troubleshooting local connectivity like laptop plugged directly into pfsense port, or even via a switch with devices on the same network.

First thing to validate is you can arp for the IP your trying to talk to on the same network.

@jt40 what rules did you put on pfsense "management" interface you created?

I didn't set any rule, if what you mean is a firewall rule, there is no FW rule at the moment.

If you had set the source to 192.168.20/24 or something sim at first and then changed the IP, this rule would not have changed.

Due to my previous answer, I think that there is nothing to worry about here.

The rule on the management interface your creating should prob be source "management net". This way when you change the management IP on pfsense, the alias for "management net" would change to reflect the network network.

Thanks for the tip, but I still didn't create any rule, so my case it's much simpler.

When troubleshooting local connectivity like laptop plugged directly into pfsense port, or even via a switch with devices on the same network.

First thing to validate is you can arp for the IP your trying to talk to on the same network.

I can see the ARP table from the laptop, it's something like this:

<FIRST_PART_MISSING_OF_THE_NETWORK_INTERFACE_NAME>_gateway 192.168.2.1 at <MAC that is correct> [ether] on enp2s0

The beginning means that I miss whatever is before _gateway. In any case it seems strange, that is actually the gateway name, the one that in PfSense is 192.168.0.1 (my modem/router).

Btw, I'm still able to ping google from the shell, so the UPLINK interface is working...
It doesn't show what interface did it use, but I guess that it used the UPLINK, not the MANAGEMENT interface, I don't have a gateway assigned on the MANAGEMENT interface.

I'll reboot also my laptop, that's the last thing that remains for my knowledge...

Anyway, thank a lot so far to everyone :)

JT40

I also did the following:

Rebooted the laptop
Assigned a new IPV4 IP to a new interface, never used with this setup. I did the same simple setup and assigned 192.168.5.1/24, no gateway assigned
Reconfigured the laptop network and assigned the IP 192.168.5.200 or 192.168.5.2
ARP table looks good
Rebooted the WebConfigurator in PfSense

After all that, I can't connect to the WebGUI of PfSense... Thepage keeps loading until it doesn't timeout, but it's not a gateway timeout or something very well defined, it's just a timeout due to the browser config...

I also did:

I checked the interface config in PfSense, it looks ok
The traffic between UPLINK and modem/router (gateway) is all good, but it was already visible from a simple ping test.
No other error messages, unless there is some sort of dmesg for such scenarios in BSD, something easy to read all in one place, I have experience with Linux but not BSD, so I'll need to dig almost from scratch.

What an experience, a bit speechless :D , fun but never had so many obstacles :D

JT40

I listed all the possible bugs here: https://forum.netgate.com/topic/168438/bunch-of-weird-things-happening-here/2

I also met this situation:
Configured the interface with HTTP from the backend, now it doesn't ask me anymore to set it up with HTTPS... So it remains in HTTP, this is definitely a bug.
Anyway, it doesn't make a difference for my case.

JT40

I also tried enabling DHCP, I successfully received the IP address in the range I specified, but I can't connect...

Interface IP: 192.168.2.1
Mask /24
DHCP range 192.168.2.2 <--> 192.168.2.254
IP received: 192.168.2.2

Looks good but I can't connect to the WebUI, HTTP or HTTPS same thing, but currently it's stuck with HTTP.
As mentioned previously, I can't change the protocol anymore...

I'm done for today, see you next week, LOL :D
Thanks everyone.