How to set the same VLANs between the switch and PfSense

SteveITS

@jt40 By "can't overlap" I mean you can't use 192.168.0.1/255.255.0.0 on WAN, and 192.168.5.1/255.255.255.0 on LAN because that LAN is inside that WAN network. Each interface has to be a unique network/subnet.

Your example INET1-4 will not work because 1, 2, and 3 are not separate networks, assuming they each have a 255.255.255.0 (/24) mask.

JT40

I did many tests, this is what I noticed:

The auto rollback function seems working, the process exits and I also rebooted the machine to be sure, but I can't login anymore even if I rolled back prior the firewall changes. The result is that I can't connect anymore to the box with the WebGUI... The connection suddenly dropped, I think it's the firewall but I strongly don't think it's related to my lack of knowledge or my mistake, I made a rollback before that change and it didn't change anything... I also check my laptop config to be sure, IP and Gateway were correct...
The firewall is slow time to time to make simple changes, like to enable an interface, or modify an existing interface IP, sometimes it takes 1 second or 20 seconds... This wide range of time doesn't make sense, plus the HW that I have is overkill...

Are you aware of the issues above? Or it is just me? I can also blame myself but all this story seems weird to me...

I still can't make it work :D

The last test was done with the current config.

Modem/router 192.168.0.1 - mask 255.255.255.0
PFSense - Uplink interface (unique IP and different network) 192.168.0.220 - mask 255.255.255.0 (I was not able to ping the modem/router with an IP of 192.168.3.220 for example, so I different network can't ping the next network node for some reason, which is the modem/router, the one that is connected to the real WAN...)
No other devices in the downstream network, unless my laptop but that's on a different interface with a different range, something like 192.168.2.1

I was able to ping the modem/router from the pfsense box, but not any public IP address...
I don't have firewall rules as such on the modem/router, so I don't understand why I don't succeed with it...
I only succeeded randomnly in the past days, but I don't really know the details of old tests as of now, it was just luck I guess...

The first part of my setup is to be able to ping a public IP at least, so I know that at least the pfsense box can reach internet, then I start to think about other devices.

SteveITS

@jt40 said in How to set the same VLANs between the switch and PfSense:

I was able to ping the modem/router from the pfsense box, but not any public IP address...

Is the gateway correct on the pfSense WAN? (should be 192.168.0.1 if that's the upstream router) What does a traceroute show? (which is also on the diagnostics menu)

I think I would start with one WAN and one LAN on the pfSense and add other interfaces after you get it working. Just disable the other interfaces for now.

johnpoz

@steveits said in How to set the same VLANs between the switch and PfSense:

I think I would start with one WAN and one LAN on the pfSense and add other interfaces after you get it working

exactly - I suggested this way back when in this thread ;) And gave a basic diagram with IPs and masks, etc.

JT40

@steveits said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

I was able to ping the modem/router from the pfsense box, but not any public IP address...

Is the gateway correct on the pfSense WAN? (should be 192.168.0.1 if that's the upstream router) What does a traceroute show? (which is also on the diagnostics menu)

I think I would start with one WAN and one LAN on the pfSense and add other interfaces after you get it working. Just disable the other interfaces for now.

This is how it was after the restart, I kept the maintenance LAN plus the uplink (WAN, but it's always a LAN port because it's not the first entry point).

That's how I tested it, I can do it again with traceroute but I need to restore it again, give me some time...

Is the gateway correct on the pfSense WAN? (should be 192.168.0.1 if that's the upstream router)

What do you mean? There is no wateway assigned, as the menu mentions as well I should not assign a gateway if that's a LAN to LAN communication. PfSense is not connected to the ISP, the modem/router that I have it is...
We discussed here previously about it and I was told do not use a gateway on the UPLINK port, the one you called WAN is actually a LAN UPLINK port. It's also mentioned in the description in PfSense.
@johnpoz did I understand correctly? I think it was you to have made me notice it :D

johnpoz

@jt40 said in How to set the same VLANs between the switch and PfSense:

What do you mean? There is no wateway assigned, as the menu mentions as well I should not assign a gateway if that's a LAN to LAN communication

What?? Where pfsense connection to your isp device is its WAN!!! That is how it gets to the internet.. No shit it would never work without a gateway..

Again going to state this with emphasis - you need to research the basic concepts of how a router works, what a network is..

I clearly went over this..

Without a gateway - where would pfsense send traffic that was destined for some network its not directly attached too??

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

What do you mean? There is no wateway assigned, as the menu mentions as well I should not assign a gateway if that's a LAN to LAN communication

What?? Where pfsense connection to your isp device is its WAN!!! That is how it gets to the internet.. No shit it would never work without a gateway..

Again going to state this with emphasis - you need to research the basic concepts of how a router works, what a network is..

I clearly went over this..

Without a gateway - where would pfsense send traffic that was destined for some network its not directly attached too??

192.168.0.1 and 192.168.0.220 are in the same network for what I know with a mask of 255.255.255.0, the communication should happen automatically, it's like 2 hosts in the same VLAN, don't tell me what I'm wrong even here :D ...
Obviosuly I consider this case correct without firewall rules blocking the traffic.

What it may not be obvious for PfSense is how to take the traffic from other interfaces and route it to the UPLINK, in that case I think a gateway is needed, and it should be the PfSense Gateway, like 192.168.0.220 (current UPLINK port), or any custom IP if it is possible, I need to check. I strongly don't think is the same IP of the WebGUI.

Regarding the graph, maybe I confused your previous graph, where you mentioned NO GATEWAY between the PfSense box and the downstream switch, that's another story.

johnpoz

@jt40 said in How to set the same VLANs between the switch and PfSense:

the communication should happen automatically, it's like 2 hosts in the same VLAN, don't tell me what I'm wrong even here :D ...

OMG dude.. Really? Yes they would be able to talk to each other - but why would pfsense send traffic for say 8.8.8.8 to 192.168.0.1 unless it was set as the gateway, ie the default route for pfsense.

Where do you have pfsense sending traffic this is not local? How would anything get to the internet if not sent to your ISP device? What is pfsense default gateway set too?

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

the communication should happen automatically, it's like 2 hosts in the same VLAN, don't tell me what I'm wrong even here :D ...

OMG dude.. Really? Yes they would be able to talk to each other - but why would pfsense send traffic for say 8.8.8.8 to 192.168.0.1 unless it was set as the gateway, ie the default route for pfsense.

Where do you have pfsense sending traffic this is not local? How would anything get to the internet if not sent to your ISP device? What is pfsense default gateway set too?

That was my initial guess :D
I think I just got confused about the previous sentence, where you said a gateway it was not needed, but actually you were referring to the pfsense --> <-- switch communication, where I also have truncate ports with different VLANs on the same port.
I'll test it and let you know

JT40

Ok the gateway was all about it, what a bs...... :D

Now I have another problem, which makes tiredesome every troubleshooting or setup I do...

Real world example:

Management interface:

IP set as 192.168.2.1
mask 255.255.255.0 or /24
no DHCP whatsoever
The connection from my laptop works

Management interface change

I changed the interface IP to 192.168.20.1, mask 255.255.255.0 or /24 (I can't connect anymore, I'm using the same port and I reconfigured my laptop in this way):

laptop IP 192.168.20.2
gateway 192.168.20.1 (nothing else than the interface IP)
no DHCP whatsoever

Revert the change

I reverted the change directly from the machine, basically assigning the IP 192.168.2.1 to the same interface, no DHCP again, nothing specific.
Yes, I tried to hit 192.168.2.1 :D from the WebGUI

I can't connect anymore, normally I would expect my laptop to be the problem, specifically in this case.
With "ifconfig" I see that the IP is correctly assigned, but I can't see the gateway and it's normal so far...
I checked the respective network config file under /etc/sysconfig/network/scripts/ , well, it's not there... Actually many of them are missing there.... The only thing I noticed is that there are few wireless and wired connections, so not only one of this type, but that's it, what I need is not there.

Even after I re-created the network config in the laptop UI, the config is not in that directory...
Also, this file is empty: /etc/sysconfig/network , and it shouldn't

Looking at this doc, these 2 things mentioned above should be there... https://docs.fedoraproject.org/en-US/Fedora/14/html/Deployment_Guide/ch-Network_Interfaces.html
The doc is a bit old but I don't think they made changes on how this component works...

In the end, I found the config file under /etc/NetworkManager/system-connections/ , it contains all the correct details, even though I found strange this line:
[ipv4]
address1=192.168.2.1/24,192.168.2.1

As you can see, the gateway is on the same line of the machine IP address... I think it's weird but looking at other files it seems correct.

PfSense system reboot it didn't help...

I'll configure a new interface, but this looks a bit crazy.

I exausted my ideas, let me know what you think :D

johnpoz

@jt40 what rules did you put on pfsense "management" interface you created?

If you had set the source to 192.168.20/24 or something sim at first and then changed the IP, this rule would not have changed.

The rule on the management interface your creating should prob be source "management net". This way when you change the management IP on pfsense, the alias for "management net" would change to reflect the network network.

When troubleshooting local connectivity like laptop plugged directly into pfsense port, or even via a switch with devices on the same network.

First thing to validate is you can arp for the IP your trying to talk to on the same network.

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 what rules did you put on pfsense "management" interface you created?

If you had set the source to 192.168.20/24 or something sim at first and then changed the IP, this rule would not have changed.

The rule on the management interface your creating should prob be source "management net". This way when you change the management IP on pfsense, the alias for "management net" would change to reflect the network network.

When troubleshooting local connectivity like laptop plugged directly into pfsense port, or even via a switch with devices on the same network.

First thing to validate is you can arp for the IP your trying to talk to on the same network.

@jt40 what rules did you put on pfsense "management" interface you created?

I didn't set any rule, if what you mean is a firewall rule, there is no FW rule at the moment.

If you had set the source to 192.168.20/24 or something sim at first and then changed the IP, this rule would not have changed.

Due to my previous answer, I think that there is nothing to worry about here.

The rule on the management interface your creating should prob be source "management net". This way when you change the management IP on pfsense, the alias for "management net" would change to reflect the network network.

Thanks for the tip, but I still didn't create any rule, so my case it's much simpler.

When troubleshooting local connectivity like laptop plugged directly into pfsense port, or even via a switch with devices on the same network.

First thing to validate is you can arp for the IP your trying to talk to on the same network.

I can see the ARP table from the laptop, it's something like this:

<FIRST_PART_MISSING_OF_THE_NETWORK_INTERFACE_NAME>_gateway 192.168.2.1 at <MAC that is correct> [ether] on enp2s0

The beginning means that I miss whatever is before _gateway. In any case it seems strange, that is actually the gateway name, the one that in PfSense is 192.168.0.1 (my modem/router).

Btw, I'm still able to ping google from the shell, so the UPLINK interface is working...
It doesn't show what interface did it use, but I guess that it used the UPLINK, not the MANAGEMENT interface, I don't have a gateway assigned on the MANAGEMENT interface.

I'll reboot also my laptop, that's the last thing that remains for my knowledge...

Anyway, thank a lot so far to everyone :)

JT40

I also did the following:

Rebooted the laptop
Assigned a new IPV4 IP to a new interface, never used with this setup. I did the same simple setup and assigned 192.168.5.1/24, no gateway assigned
Reconfigured the laptop network and assigned the IP 192.168.5.200 or 192.168.5.2
ARP table looks good
Rebooted the WebConfigurator in PfSense

After all that, I can't connect to the WebGUI of PfSense... Thepage keeps loading until it doesn't timeout, but it's not a gateway timeout or something very well defined, it's just a timeout due to the browser config...

I also did:

I checked the interface config in PfSense, it looks ok
The traffic between UPLINK and modem/router (gateway) is all good, but it was already visible from a simple ping test.
No other error messages, unless there is some sort of dmesg for such scenarios in BSD, something easy to read all in one place, I have experience with Linux but not BSD, so I'll need to dig almost from scratch.

What an experience, a bit speechless :D , fun but never had so many obstacles :D

JT40

I listed all the possible bugs here: https://forum.netgate.com/topic/168438/bunch-of-weird-things-happening-here/2

I also met this situation:
Configured the interface with HTTP from the backend, now it doesn't ask me anymore to set it up with HTTPS... So it remains in HTTP, this is definitely a bug.
Anyway, it doesn't make a difference for my case.

JT40

I also tried enabling DHCP, I successfully received the IP address in the range I specified, but I can't connect...

Interface IP: 192.168.2.1
Mask /24
DHCP range 192.168.2.2 <--> 192.168.2.254
IP received: 192.168.2.2

Looks good but I can't connect to the WebUI, HTTP or HTTPS same thing, but currently it's stuck with HTTP.
As mentioned previously, I can't change the protocol anymore...

I'm done for today, see you next week, LOL :D
Thanks everyone.

johnpoz

@jt40 said in How to set the same VLANs between the switch and PfSense:

Looks good but I can't connect to the WebUI,

And what are the rules you put on the interface?

There are not rules on a new interface - you have to create them.

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

Looks good but I can't connect to the WebUI,

And what are the rules you put on the interface?

There are not rules on a new interface - you have to create them.

No rules, very simple setup.

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

Looks good but I can't connect to the WebUI,

And what are the rules you put on the interface?

There are not rules on a new interface - you have to create them.

To be honest I can't specify any rule there, if for "rules" you mean firewall rules.
The shell is quite limited in terms of setup, I recall interface IPV4, IPV6, gateway (if any), DHCP, HTTP or HTTPS, and that's it.

johnpoz

@jt40 said in How to set the same VLANs between the switch and PfSense:

No rules, very simple setup.

How is it going to work with NO rules? You have to add the rules in the gui! It will not work without rules! Because the default rule is DENY! so no rules - nothing works! If you locked yourself out of the gui. Disable the firewall for a minute, access the gui and adjust the rules on this interface.

This is why if your going to do an admin interface, its should be the default LAN, which has the antilock out rule on it. And create other interfaces for your other networks.

JT40

@johnpoz said in How to set the same VLANs between the switch and PfSense:

@jt40 said in How to set the same VLANs between the switch and PfSense:

No rules, very simple setup.

How is it going to work with NO rules? You have to add the rules in the gui! It will not work without rules! Because the default rule is DENY! so no rules - nothing works! If you locked yourself out of the gui. Disable the firewall for a minute, access the gui and adjust the rules on this interface.

This is why if your going to do an admin interface, its should be the default LAN, which has the antilock out rule on it. And create other interfaces for your other networks.

I'll test it, but how did I login before???
In the backend, it asked me the same info as in the WebGui...
So every time I change the IP of the interface, it denies the WebGUI from every network interface?
What aboutt the rest of the communications? I'd be quite scared if every time I need to reconfigure everything... Especially because the rollback didn't allow me to connect, or it was not enough, say it as you wish.

Where is located this rule?? Do you mean firewall rule?

On the other side, it's possible that I won't change the interface IPs, or at least I should not change it if my assumptions are correct :D