None of my firewall rules are working on VLANs
-
@Jarhead said in None of my firewall rules are working on VLANs:
Do you realize that setting a static IP doesn't put you on a different network?
Yeah I brought that up right away - his wording of how he was changing "vlans" was off - just saying he changed his ip?
Running a router as VM can have some advantages - but you better be fairly knowledgeable on what has to be done to do that for vlans or other networks, and fully understand networking to be honest. I ran mine as vm on esxi for a while, and did setup one on proxmox to ship to another location for work.
Proxmox was a bit of learning curve from esxi.
As a vm there are some advantages - I could easy switch from one instance of pfsense to another, so allowed for easy testing of some new version or even other flavors of router distros. Very easy backup with just a snapshot I could roll back too. But they do have some draw backs. To do anything with the vm host my whole network was down. There is a learning curve on how the vm host software handles vlan tags, etc. Which can be different than just your typical vlan switch.
Much better setup when I moved pfsense to hardware for my edge router that is for sure. I still play with different versions of pfsense as VM but they are just on my nas and for testing. If I have to do something with my nas (vm host) my whole network isn't down ;) I wouldn't go back to on a VM as my main router.
-
@johnpoz Completely agree and I do the same with VM's for testing. Can't beat them for that, too easy to just restore a snapshot when something goes wrong.
But to trust your whole network to a VM, to me that's not very smart. It goes back to the "Just because you can, doesn't mean you should" logic. -
@Jarhead agree.. There are scenarios where sure it makes sense.. And I can see doing it, there are pro's and cons for sure. Which need to be weighed out..
Personally I wouldn't go back to running my edge on vm ever.. The con's outweigh the pros imho.
And if you are going to run your production edge router/firewall on VM, you should have a really good handle on what that entails.
And if you do run into something where you have a question - first line of your post should be I am running this on VM, here are the details of the setup.
While in this post they do mention proxmox - they do not clearly state that pfsense is on proxmox, etc. For all we know from the first post is their proxmox could just be some other client on the network.
-
@johnpoz said in None of my firewall rules are working on VLANs:
@ErniePantuso said in None of my firewall rules are working on VLANs:
it does not pull an IP address.
This is the big clue that you have something at L2 borked..
I understand (I think) that “L2” means “layer 2” but I don’t have a very good understanding of the OSI 7-layer burrito. Are you referring to my switch configuration? That’s entirely possible. I’ll post a screenshot of my switch configuration later.
If you have a dhcp running and you don't get an IP.. Look to the dhcp server - did it see the discover, and not an IP to hand out is that why? Or did it never get the discover, so your L2 is borked.
If dhcp is not working, setting a static isn't going to work either. Unless it was a failure of the actual dhcp server. Again looking to see if you saw discover in the logs.
This may be my biggest problem. When I look at my logs, it’s not clear to me how to read them and/or how to even find the specific entries that are applicable. Can you recommend a good article or YouTube video that could educate me on this?
As to your dns error - well yeah if you can't talk to the dns, your never going to get an answer.. Simple check for this is, can you ping your gateway for checking connectivity, can you ping the dns IP if different than your gateway.. Can you do a dns query with nslookup, dig, there are many tools you can use to validate dns is working while at the same time checking the other aspects of your connectivity..
Thanks, I’ll check on that when I’m back onsite.
@Bob-Dig said in None of my firewall rules are working on VLANs:
Learning pfSense and routing and proxmox and virtualization at the same time might be to much for you.
I’ve been using virtualization since the early days of VMware. I still use VirtualBox regularly. I’m no expert on Docker but I use it frequently and without any major issues. Proxmox is new to me - and while it does seem to be significantly more complex than Docker/VirtualBox/VMware, it doesn’t feel “over my head”. (So far, anyway - if and when I get more deeply into any of the virtual networking stuff, the same gap in my knowledge that’s causing me problems here are likely to trip me up there, too, I’m sure.) All that said, learning pfSense and routing might be too much for me! Hahaha…
@Jarhead said in None of my firewall rules are working on VLANs:
@ErniePantuso See the 0/0 B on the left of the rules? That means that rule has never been hit, unless you just rebooted before that pic was taken.
You definitely have a layer 2 problem as John said.Not trying to say you're an idiot here. Don't take it that way, but you clearly are confused.
I am indeed confused and no worries - I wasn’t taking offense; I really appreciate all the help. I just wanted to convey that I do have a modicum of knowledge on this stuff.
Do you realize that setting a static IP doesn't put you on a different network? You have to physically connect to it still.
No, I did not realize that, but OTOH, I do have a physical connection. The port that my laptop is connected to is tagged (in my switch configuration) for VLAN 10 (Biz_VLAN). As a further troubleshooting step, I took a port that I wasn’t previously using, tagged it for VLAN 10, moved my cable over, and switched to DHCP. As stated previously, it didn’t pull an IP so I guess it’s looking more and more like a switch problem.
I never understand why anyone would want a router to run as a VM, it's your main entrance to your network, it should be a separate machine. But if that's what you want, you will have to learn about virtual switches too then.
I’m not sure where this came from. I’m not running my router as a VM. Nor is my switch virtualized. Again, I’m not onsite at the moment but when I am, I’ll post a diagram of what I’m working with so that everything is a little more clear.
I want to also note that I’m very committed to learning this and making it work. So much so that I’ve actually purchased the same switch and router for my homelab (I’ll be setting up a Proxmox server at home as well) so that I can work on all this from home as well — but I don’t yet have everything setup at home.
Start small. Just put an any/any rule on both interface, turn on the DHCP server on both, then connect a device to each and go from there. Once you have both networks working you can then lock them down as needed.
Sounds like great advice. At some point, I think I’ve done that (I’m actually very creative and quite good at troubleshooting; I’ve tried dozens of different things and I only turn to help via forums/reddit/etc. when I can’t think of anything else to try) but I think it’s time to go back to square one.
-
@johnpoz said in None of my firewall rules are working on VLANs:
While in this post they do mention proxmox - they do not clearly state that pfsense is on proxmox, etc. For all we know from the first post is their proxmox could just be some other client on the network.
Ah, now I understand the misconception. That last statement is correct; my Proxmox server is just a client on Biz_VLAN. Once network communication is working, I’ll move on to setting up a Windows file server VM and Active Directory domain controller along with a handful of other containers including Dokuwiki (for documenting all of the IT infrastructure), Kanboard (for project management), a LEMP stack for a couple of internal web apps, a SIEM tool, a backup server, and probably a honeypot in a DMZ (to prove to my 87 year old boss how significant the cybersecurity threats are! “We’ve never had a problem before…..” SMH)
-
@ErniePantuso said in None of my firewall rules are working on VLANs:
No, I did not realize that, but OTOH, I do have a physical connection. The port that my laptop is connected to is tagged (in my switch configuration) for VLAN 10 (Biz_VLAN). As a further troubleshooting step, I took a port that I wasn’t previously using, tagged it for VLAN 10, moved my cable over, and switched to DHCP. As stated previously, it didn’t pull an IP so I guess it’s looking more and more like a switch problem.
Well, that's probably one of the problems right there. Is the laptop also tagged?
If not, the port you plug it into needs to be untagged.
So if you are only testing with one port, you would need to untag vlan 10 on it for bizvlan, then if you want to go back to your LAN, you would need to untag it with that vlan.
Easier to use 2 ports if you can, leave one as LAN and untag the other with 10. -
@Jarhead said in None of my firewall rules are working on VLANs:
Well, that's probably one of the problems right there. Is the laptop also tagged?
Ok, now we're really getting to the bottom of the problem - the old chair : keyboard interface error! I'm obviously 100% clueless about VLAN configuration. How in the heck do I tag the laptop? Somewhere in pfSense, I'm guessing because the switch configuration is pretty simple.
If not, the port you plug it into needs to be untagged.
So if you are only testing with one port, you would need to untag vlan 10 on it for bizvlan, then if you want to go back to your LAN, you would need to untag it with that vlan.
Easier to use 2 ports if you can, leave one as LAN and untag the other with 10.Now I'm really confused! Let me post my network diagram and switch configuration.
-
-
@ErniePantuso
You don't want to tag the laptop, you want to untag the switchport.
What port goes to the pfSense LAN? It needs to be a "trunk", meaning it carries multiple vlans.
This is done in pfSense when you create a vlan and use the LAN as it's parent.
This is done in the switch by tagging which vlan(s) you want on it.
You have 4 vlans there (technically 5 but 4 that need to be tagged), doesn't look like you're using any of them on that switch.
You would have to untag ports to be able to access them from the switch.
So let's say you have port 1 as the trunk. It would need vlan 1 untagged, then all other vlans tagged on it.
This brings all vlans into the switch where they can all now be accessed.
Then let's say you want port 2 to be vlan 10.
You need to untag port 2 with vlan 10. Then go to the PVID Settings page and set port 2 to PVID 10.
That's it. Do that with any other ports and whatever vlan you want to use it for.If you want to carry all those vlans to another switch, you would then trunk another port on both switches.
That will make all vlans accessible on the other switch as well. Then untag access ports with the vlan id you want on that port.Tagged ports need to be connected to tagged ports. A typical laptop nic will be untagged, but some drivers do allow you to tag it. Not usually what you want though.
Just noticed the numbers on your switch diagram, assuming those are ports.
1 should be = vlan 1 untagged, all others tagged
2 = vlan 10 untagged
3 = vlan 20 untagged
4 = vlan 30 untagged
7 & 8 = vlan 1 untagged and 10 and 50 untagged (or all if any need for wireless on others.)
Set PVID's accordingly. 7 and 8 will be PVID 1.
AP's will need to be vlan capable. -
@Jarhead said in None of my firewall rules are working on VLANs:
2 = vlan 10 untagged
I believe he wants to use more than 1 vlan on his proxmox server which looks like its connected to port 2 of his switch. So he would want to tag the vlans he wants to have his other machines on.. Loots like he has one VM there on vlan 10.1.1 whatever ID that is, and then other is on 10.1.10..
-
Thank you both so much for all the help!
I've made some major changes...
I decided that I didn't really have very much invested in my current setup (at least not much that was working properly) - and since there was a significant change I had been wanting to make (using a trunk port, which I had actually created - but only after I'd already created all my VLANs - but it wasn't in use and I was worried about totally borking everything by putting it in use so......) I reset pfSense to factory defaults, reset all the VLAN cfg on my switch, and started over.
Now I have my MGMT (basically the LAN interface - not a VLAN - but renamed and with only management devices on it), and my 4 VLANS (BIZ, PHONES, POS, and GUEST_WIFI) on the trunk port. I also have PFSENSE_DIRECT which is on igb3 of the Sophpos box to give me a place to connect my laptop directly in case I really screw everything up.
Here's what's working so far:
- I can access the pfSense GUI on PFSENSE_DIRECT
- I can pull an IP address on MGMT and access the web gui when connected to any port on the switch.
I have zero firewall rules for trunk or any of the VLANs but then I don't have anything cfg'd on the switch, either.
I'm going to try to post a bunch of screenshots of my new configuration so everything is clear. I haven't updated my network diagram but I do now have a cable running from igb2 on the router (trunk) to port 3 on the switch.
@johnpoz said in None of my firewall rules are working on VLANs:
I believe he wants to use more than 1 vlan on his proxmox server which looks like its connected to port 2 of his switch. So he would want to tag the vlans he wants to have his other machines on.. Loots like he has one VM there on vlan 10.1.1 whatever ID that is, and then other is on 10.1.10..
Most of the VMs and containers in Proxmox will be on BIZ. Only 1 or 2 will be on MGMT (which isn't a VLAN).
@Jarhead said in None of my firewall rules are working on VLANs:
@ErniePantuso
You don't want to tag the laptop, you want to untag the switchport.
What port goes to the pfSense LAN? It needs to be a "trunk", meaning it carries multiple vlans.Thanks for this! I think a lot has changed so all your advice may not still apply but I'm going to go thru it carefully and try to apply it and learn from it. I think mostly I just need to understand/wrap my head around the difference between tagged and untagged and when to use each.
-
@ErniePantuso You didn't have to create an interface named trunk, the original Lan would have been a trunk when you assigned vlans to it. But if you have the spare router port, it'll do.
Now you need to trunk a port on the switch, and connect it to igb2.
Just follow my previous post. It's still the same, only difference now is you have the vlan 1 on a separate interface.
So the trunk connected to the router won't need vlan 1 untagged but the trunks to the AP's will. -
@Jarhead said in None of my firewall rules are working on VLANs:
So the trunk connected to the router won't need vlan 1 untagged but the trunks to the AP's will.
Exactly you can have more than 1 uplink from the switch that carry different networks/vlans - if you have free ports on your switch and router this actually good idea, now if you have intervlan traffic you don't have to worry about hairpin, as long as you don't put the uplink on the same physical.
So for example.
So you can see vlans are on my igb2 interface, there is also an untagged vlan on this, this is vlan 2 on the switch.
igb actually goes thru my switch as well, this is vlan 99 on the switch untagged and from another port on the switch untagged in vlan 99 goes to my modem.
Lan is untagged goes to vlan 9 on my switch.
All the vlans that are tagged on on the switch igb2 plugs into. These are all wireless networks, and there is no intervlan traffic between them if talk to them, its from say my lan network.
Roku is also a wireless network but this is where all my media players are and no need to share bandwidth the other wireless networks and I had spare ports. Then dmz is another untagged network
if you have the ports on your router and switch, nothing really needs to be tagged, they can all be untagged native networks on pfsense. And then untagged in whatever vlan those networks are on your switch.
The only time you have to tag is when your going to carry more than one network/vlan over the same physical wire.
