None of my firewall rules are working on VLANs
-
@johnpoz said in None of my firewall rules are working on VLANs:
@ErniePantuso Why are blocking bogon on your lan side interfaces?
That was the default so I left it.
There is no point to putting bogon on any of your lan side interfaces.
Got it. Fixed. Thanks!
-
@ErniePantuso said in None of my firewall rules are working on VLANs:
That was the default so I left it.
No it is not that is for sure.. It is default on the WAN, not any other interface.
-
@johnpoz said in None of my firewall rules are working on VLANs:
@ErniePantuso said in None of my firewall rules are working on VLANs:
That was the default so I left it.
No it is not that is for sure.. It is default on the WAN, not any other interface.
OK. I guess I don't remember when or why that got changed. I've really struggled to learn all this stuff; I've asked for and received help from a lot of different sources. Either I got some bad information or (more likely) I misunderstood the info I got.
-
@ErniePantuso It has never been the default, ever - because it make no logical sense at all.. I have been running pfsense pretty much since it has come out.. And never has block bogon been a thing on the lan side.. It only makes sense on the wan. And even then its not very useful or needed..
If you mean when you changed it ok ;) its not a thing anyone would ever have need to do.. And if pfsense didn't pull out rfc1918 from the bogon list you would of broken your network.. Would of remembered then when you changed it ;) hehehe
They enable you to be able to check it - because maybe its a transit network, and you have no idea what the source would be, or maybe its an actual wan and you would want to enable it.
-
@johnpoz said in None of my firewall rules are working on VLANs:
@ErniePantuso . . . And if pfsense didn't pull out rfc1918 from the bogon list you would of broken your network.. Would of remembered then when you changed it ;) hehehe
Not necessarily. In the first place, I've never really had the network setup. It's been in an alpha state for months as I slog my way through learning this stuff. And within that framework, I've broken my network and restored pfSense from backed up configurations numerous times - so it could have been any of those and I don't know which! hahaha
-
@Jarhead said in None of my firewall rules are working on VLANs:
@ErniePantuso You need to read up on rules. They need to be on the interface the traffic originates on. You have a rule on bizvlan with a destination of bizvlan. Not gonna work.
You have an any/any rule with other pass rules below it. Not gonna work.
If you want to block (or allow) the bizvlan access to LAN, the rule goes on bizvlan.You have no traffic shown on the bizvlan rules. Are you sure you're even connecting to it? Does it get a DHCP address if you connect to it?
I'm not a complete idiot. ;D I do have a basic understanding of rules. I understand that any pass rules after the any/any rule are superfluous. I put the any/any rule there as a troubleshooting measure. And I understand that rules for BizVLAN have to go on BizVLAN. My rule on BizVLAN with a destination of BizVLAN subnets was intended to allow nodes on that VLAN to be able to talk to each other. I thought it was blocked otherwise. But I now know that's not correct because I tried disabling ALL the rules on BizVLAN and (with a static IP in that subnet), I can still access my Proxmox server. That confuses the heck out of me!
How/where do you see that I have no traffic shown on the bizvlan rules? Am I connected to it? Well, again, with a static IP in that subnet, I can access my Proxmox server - and when I go back to a NetMgmt IP, I can't. That would seem to indicate that I'm connected, wouldn't it? I wasn't worrying yet about DHCP but now that you asked, when I switch my laptop to DHCP, it does not pull an IP address.
So what does all that tell me? I keep trying to download an LXC container template from my Proxmox server and it continues to give me a DNS error - even when the only rule on BizVLAN is the pass any/any. If it's allowed to pass any/any, shouldn't that include DNS server(s)?
-
@ErniePantuso said in None of my firewall rules are working on VLANs:
was intended to allow nodes on that VLAN to be able to talk to each other.
Doesn't work like that, but it comes up quite often here to be honest..
it does not pull an IP address.
This is the big clue that you have something at L2 borked.. If you have a dhcp running and you don't get an IP.. Look to the dhcp server - did it see the discover, and not an IP to hand out is that why? Or did it never get the discover, so your L2 is borked.
If dhcp is not working, setting a static isn't going to work either. Unless it was a failure of the actual dhcp server. Again looking to see if you saw discover in the logs.
As to your dns error - well yeah if you can't talk to the dns, your never going to get an answer.. Simple check for this is, can you ping your gateway for checking connectivity, can you ping the dns IP if different than your gateway.. Can you do a dns query with nslookup, dig, there are many tools you can use to validate dns is working while at the same time checking the other aspects of your connectivity..
-
@ErniePantuso said in None of my firewall rules are working on VLANs:
It's been in an alpha state for months as I slog my way through learning this stuff.
Learning pfSense and routing and proxmox and virtualization at the same time might be to much for you.
-
@ErniePantuso See the 0/0 B on the left of the rules? That means that rule has never been hit, unless you just rebooted before that pic was taken.
You definitely have a layer 2 problem as John said.Not trying to say you're an idiot here. Don't take it that way, but you clearly are confused.
Do you realize that setting a static IP doesn't put you on a different network? You have to physically connect to it still.
I never understand why anyone would want a router to run as a VM, it's your main entrance to your network, it should be a separate machine. But if that's what you want, you will have to learn about virtual switches too then. Just guessing you don't have the vSwitch configured correctly which is why you aren't connecting to the vlan.
Start small. Just put an any/any rule on both interface, turn on the DHCP server on both, then connect a device to each and go from there. Once you have both networks working you can then lock them down as needed. -
@Jarhead said in None of my firewall rules are working on VLANs:
Do you realize that setting a static IP doesn't put you on a different network?
Yeah I brought that up right away - his wording of how he was changing "vlans" was off - just saying he changed his ip?
Running a router as VM can have some advantages - but you better be fairly knowledgeable on what has to be done to do that for vlans or other networks, and fully understand networking to be honest. I ran mine as vm on esxi for a while, and did setup one on proxmox to ship to another location for work.
Proxmox was a bit of learning curve from esxi.
As a vm there are some advantages - I could easy switch from one instance of pfsense to another, so allowed for easy testing of some new version or even other flavors of router distros. Very easy backup with just a snapshot I could roll back too. But they do have some draw backs. To do anything with the vm host my whole network was down. There is a learning curve on how the vm host software handles vlan tags, etc. Which can be different than just your typical vlan switch.
Much better setup when I moved pfsense to hardware for my edge router that is for sure. I still play with different versions of pfsense as VM but they are just on my nas and for testing. If I have to do something with my nas (vm host) my whole network isn't down ;) I wouldn't go back to on a VM as my main router.
-
@johnpoz Completely agree and I do the same with VM's for testing. Can't beat them for that, too easy to just restore a snapshot when something goes wrong.
But to trust your whole network to a VM, to me that's not very smart. It goes back to the "Just because you can, doesn't mean you should" logic. -
@Jarhead agree.. There are scenarios where sure it makes sense.. And I can see doing it, there are pro's and cons for sure. Which need to be weighed out..
Personally I wouldn't go back to running my edge on vm ever.. The con's outweigh the pros imho.
And if you are going to run your production edge router/firewall on VM, you should have a really good handle on what that entails.
And if you do run into something where you have a question - first line of your post should be I am running this on VM, here are the details of the setup.
While in this post they do mention proxmox - they do not clearly state that pfsense is on proxmox, etc. For all we know from the first post is their proxmox could just be some other client on the network.
-
@johnpoz said in None of my firewall rules are working on VLANs:
@ErniePantuso said in None of my firewall rules are working on VLANs:
it does not pull an IP address.
This is the big clue that you have something at L2 borked..
I understand (I think) that “L2” means “layer 2” but I don’t have a very good understanding of the OSI 7-layer burrito. Are you referring to my switch configuration? That’s entirely possible. I’ll post a screenshot of my switch configuration later.
If you have a dhcp running and you don't get an IP.. Look to the dhcp server - did it see the discover, and not an IP to hand out is that why? Or did it never get the discover, so your L2 is borked.
If dhcp is not working, setting a static isn't going to work either. Unless it was a failure of the actual dhcp server. Again looking to see if you saw discover in the logs.
This may be my biggest problem. When I look at my logs, it’s not clear to me how to read them and/or how to even find the specific entries that are applicable. Can you recommend a good article or YouTube video that could educate me on this?
As to your dns error - well yeah if you can't talk to the dns, your never going to get an answer.. Simple check for this is, can you ping your gateway for checking connectivity, can you ping the dns IP if different than your gateway.. Can you do a dns query with nslookup, dig, there are many tools you can use to validate dns is working while at the same time checking the other aspects of your connectivity..
Thanks, I’ll check on that when I’m back onsite.
@Bob-Dig said in None of my firewall rules are working on VLANs:
Learning pfSense and routing and proxmox and virtualization at the same time might be to much for you.
I’ve been using virtualization since the early days of VMware. I still use VirtualBox regularly. I’m no expert on Docker but I use it frequently and without any major issues. Proxmox is new to me - and while it does seem to be significantly more complex than Docker/VirtualBox/VMware, it doesn’t feel “over my head”. (So far, anyway - if and when I get more deeply into any of the virtual networking stuff, the same gap in my knowledge that’s causing me problems here are likely to trip me up there, too, I’m sure.) All that said, learning pfSense and routing might be too much for me! Hahaha…
@Jarhead said in None of my firewall rules are working on VLANs:
@ErniePantuso See the 0/0 B on the left of the rules? That means that rule has never been hit, unless you just rebooted before that pic was taken.
You definitely have a layer 2 problem as John said.Not trying to say you're an idiot here. Don't take it that way, but you clearly are confused.
I am indeed confused and no worries - I wasn’t taking offense; I really appreciate all the help. I just wanted to convey that I do have a modicum of knowledge on this stuff.
Do you realize that setting a static IP doesn't put you on a different network? You have to physically connect to it still.
No, I did not realize that, but OTOH, I do have a physical connection. The port that my laptop is connected to is tagged (in my switch configuration) for VLAN 10 (Biz_VLAN). As a further troubleshooting step, I took a port that I wasn’t previously using, tagged it for VLAN 10, moved my cable over, and switched to DHCP. As stated previously, it didn’t pull an IP so I guess it’s looking more and more like a switch problem.
I never understand why anyone would want a router to run as a VM, it's your main entrance to your network, it should be a separate machine. But if that's what you want, you will have to learn about virtual switches too then.
I’m not sure where this came from. I’m not running my router as a VM. Nor is my switch virtualized. Again, I’m not onsite at the moment but when I am, I’ll post a diagram of what I’m working with so that everything is a little more clear.
I want to also note that I’m very committed to learning this and making it work. So much so that I’ve actually purchased the same switch and router for my homelab (I’ll be setting up a Proxmox server at home as well) so that I can work on all this from home as well — but I don’t yet have everything setup at home.
Start small. Just put an any/any rule on both interface, turn on the DHCP server on both, then connect a device to each and go from there. Once you have both networks working you can then lock them down as needed.
Sounds like great advice. At some point, I think I’ve done that (I’m actually very creative and quite good at troubleshooting; I’ve tried dozens of different things and I only turn to help via forums/reddit/etc. when I can’t think of anything else to try) but I think it’s time to go back to square one.
-
@johnpoz said in None of my firewall rules are working on VLANs:
While in this post they do mention proxmox - they do not clearly state that pfsense is on proxmox, etc. For all we know from the first post is their proxmox could just be some other client on the network.
Ah, now I understand the misconception. That last statement is correct; my Proxmox server is just a client on Biz_VLAN. Once network communication is working, I’ll move on to setting up a Windows file server VM and Active Directory domain controller along with a handful of other containers including Dokuwiki (for documenting all of the IT infrastructure), Kanboard (for project management), a LEMP stack for a couple of internal web apps, a SIEM tool, a backup server, and probably a honeypot in a DMZ (to prove to my 87 year old boss how significant the cybersecurity threats are! “We’ve never had a problem before…..” SMH)
-
@ErniePantuso said in None of my firewall rules are working on VLANs:
No, I did not realize that, but OTOH, I do have a physical connection. The port that my laptop is connected to is tagged (in my switch configuration) for VLAN 10 (Biz_VLAN). As a further troubleshooting step, I took a port that I wasn’t previously using, tagged it for VLAN 10, moved my cable over, and switched to DHCP. As stated previously, it didn’t pull an IP so I guess it’s looking more and more like a switch problem.
Well, that's probably one of the problems right there. Is the laptop also tagged?
If not, the port you plug it into needs to be untagged.
So if you are only testing with one port, you would need to untag vlan 10 on it for bizvlan, then if you want to go back to your LAN, you would need to untag it with that vlan.
Easier to use 2 ports if you can, leave one as LAN and untag the other with 10. -
@Jarhead said in None of my firewall rules are working on VLANs:
Well, that's probably one of the problems right there. Is the laptop also tagged?
Ok, now we're really getting to the bottom of the problem - the old chair : keyboard interface error! I'm obviously 100% clueless about VLAN configuration. How in the heck do I tag the laptop? Somewhere in pfSense, I'm guessing because the switch configuration is pretty simple.
If not, the port you plug it into needs to be untagged.
So if you are only testing with one port, you would need to untag vlan 10 on it for bizvlan, then if you want to go back to your LAN, you would need to untag it with that vlan.
Easier to use 2 ports if you can, leave one as LAN and untag the other with 10.Now I'm really confused! Let me post my network diagram and switch configuration.
-
-
@ErniePantuso
You don't want to tag the laptop, you want to untag the switchport.
What port goes to the pfSense LAN? It needs to be a "trunk", meaning it carries multiple vlans.
This is done in pfSense when you create a vlan and use the LAN as it's parent.
This is done in the switch by tagging which vlan(s) you want on it.
You have 4 vlans there (technically 5 but 4 that need to be tagged), doesn't look like you're using any of them on that switch.
You would have to untag ports to be able to access them from the switch.
So let's say you have port 1 as the trunk. It would need vlan 1 untagged, then all other vlans tagged on it.
This brings all vlans into the switch where they can all now be accessed.
Then let's say you want port 2 to be vlan 10.
You need to untag port 2 with vlan 10. Then go to the PVID Settings page and set port 2 to PVID 10.
That's it. Do that with any other ports and whatever vlan you want to use it for.If you want to carry all those vlans to another switch, you would then trunk another port on both switches.
That will make all vlans accessible on the other switch as well. Then untag access ports with the vlan id you want on that port.Tagged ports need to be connected to tagged ports. A typical laptop nic will be untagged, but some drivers do allow you to tag it. Not usually what you want though.
Just noticed the numbers on your switch diagram, assuming those are ports.
1 should be = vlan 1 untagged, all others tagged
2 = vlan 10 untagged
3 = vlan 20 untagged
4 = vlan 30 untagged
7 & 8 = vlan 1 untagged and 10 and 50 untagged (or all if any need for wireless on others.)
Set PVID's accordingly. 7 and 8 will be PVID 1.
AP's will need to be vlan capable. -
@Jarhead said in None of my firewall rules are working on VLANs:
2 = vlan 10 untagged
I believe he wants to use more than 1 vlan on his proxmox server which looks like its connected to port 2 of his switch. So he would want to tag the vlans he wants to have his other machines on.. Loots like he has one VM there on vlan 10.1.1 whatever ID that is, and then other is on 10.1.10..
-
Thank you both so much for all the help!
I've made some major changes...
I decided that I didn't really have very much invested in my current setup (at least not much that was working properly) - and since there was a significant change I had been wanting to make (using a trunk port, which I had actually created - but only after I'd already created all my VLANs - but it wasn't in use and I was worried about totally borking everything by putting it in use so......) I reset pfSense to factory defaults, reset all the VLAN cfg on my switch, and started over.
Now I have my MGMT (basically the LAN interface - not a VLAN - but renamed and with only management devices on it), and my 4 VLANS (BIZ, PHONES, POS, and GUEST_WIFI) on the trunk port. I also have PFSENSE_DIRECT which is on igb3 of the Sophpos box to give me a place to connect my laptop directly in case I really screw everything up.
Here's what's working so far:
- I can access the pfSense GUI on PFSENSE_DIRECT
- I can pull an IP address on MGMT and access the web gui when connected to any port on the switch.
I have zero firewall rules for trunk or any of the VLANs but then I don't have anything cfg'd on the switch, either.
I'm going to try to post a bunch of screenshots of my new configuration so everything is clear. I haven't updated my network diagram but I do now have a cable running from igb2 on the router (trunk) to port 3 on the switch.
@johnpoz said in None of my firewall rules are working on VLANs:
I believe he wants to use more than 1 vlan on his proxmox server which looks like its connected to port 2 of his switch. So he would want to tag the vlans he wants to have his other machines on.. Loots like he has one VM there on vlan 10.1.1 whatever ID that is, and then other is on 10.1.10..
Most of the VMs and containers in Proxmox will be on BIZ. Only 1 or 2 will be on MGMT (which isn't a VLAN).
@Jarhead said in None of my firewall rules are working on VLANs:
@ErniePantuso
You don't want to tag the laptop, you want to untag the switchport.
What port goes to the pfSense LAN? It needs to be a "trunk", meaning it carries multiple vlans.Thanks for this! I think a lot has changed so all your advice may not still apply but I'm going to go thru it carefully and try to apply it and learn from it. I think mostly I just need to understand/wrap my head around the difference between tagged and untagged and when to use each.
