I can't get VLANs to work / No DHCP
-
@johnpoz These are Unifi switches, so what would normally be called a trunk is called "ALL" in the Unifi world.
This is passing all VLANs to the physical NIC in the host.
I have also tried this configuration without VLANs in pfSense, using instead the port group that is assigned to VLAN 108, and that does not work either.
As to why I'm running such an old ESXi - It's a home server. I've only upgraded it once in its lifetime. It is in a secure environment, and it's only my wife and I at home. Yes, I run stuff on it that I need to be running, but for the most part I do this so that I'm not completely hands-off working in production networks, as I moved into InfoSec and then InfoSec Audit several years ago.
-
@robh-0 said in I can't get VLANs to work / No DHCP:
as I moved into InfoSec
And you don't understand how vlans work? On the unifi switch you still have to create the vlan.. Did you do that for 108? 4095 as an ID is not a valid vlan for esxi.. Did you connect this device in your other port where vlan 108 was set?
You created an interface in pfsense vm that it sees as em2... If you put em2 in the port group you set with vlan 108.. As tagged traffic comes into esxi through its physical port it would send that tagged traffic stripped of its tag to this vlan 108 port group.
What makes no sense if you have not assigned vlan 108.. So this could not be setup. Did you actually setup whatever network you want to run on em2? Native - since if that virtual nic you created for pfsense is in port group 108.. That traffic would be untagged.
As to upgrading - I don't care if sitting in a closet not connected to anything other than a laptop all airgaped.. How exactly are you in infosec and just continue a FREE product that is EOL? Its not like its 3k and you don't have the money to upgrade - esxi is FREE.. Just upgrade to current. Also I highly doubt it supports the OS your trying to run - so you have no idea what sort of issues trying to run freebsd 12.3 OS on such old version of esxi..
I would suggest you upgrade to current esxi, and then also correctly setup your vswitches and port groups to do what you want, be the traffic tagged or untagged from pfsense perspective.
Your in infosec audit - so pretty much you yell at people all day about how they need to stay current with versions and patches.. But then you have a specific EOL version that is 5 some years old running at home ;)
-
@johnpoz Yes, I understand how VLANs work.
If you scroll up, you'll see a screenshot that shows VLAN108 and pfsenseLAN tied to the same physical port in my vSwitch topology.
Yes, I created the VLANs on the switch. I have multiple VLANs, all working fine, and have for years. I created two new VLANs just for this project, but the other ones have been there a long time and working great.
Yes, I created an interface that it sees as EM2. I have tried both sending the stripped traffic to that, and I've tried assigning a VLAN in pfSense. Neither works.I'm working on getting the upgrade to 7.0 right now. I'm logged in to the VMWare site, I just can't get the manual download button to work. I seem to have this problem with their site every time I go out there, yet another reason I probably haven't upgraded, the site is frustrating. I've also moved and other things over the years, it has been a time thing.
No, I do not yell at anyone. :) In fact, at the start of this year I moved out of managing the audit team and now I'm on the policy and standard group. Now I have no employees, and I don't have to talk to anyone about their security practices anymore.
Hey, at least I can just wipe this VM and start over, it only takes a minute or so to reinstall pfSense.
-
@robh-0 your download issue prob related to blocking, pihole or adblocker, etc.
I did see what you were talking about, I then set adblock off on this page - same issue. So then pointed my client to just unfiltered dns (pfsense vs pihole).. And restarted browser to clear its cache and download no problem.
As to your vlan - I don't show your vlan actually assigned to an interface.. And if your going to do vlan 108 untagged to pfsense, then you would need that setup for network and dhcp directly on the em2 interface.
So here you can see I have vlans actually assigned to an interface.
So those pfsense needs to see the tags come in on that igb2, if untagged then its on the native vlan
-
@robh-0 7.0 Update 3 installed. I'm going to do a new install for pfSense just to make sure I don't have any compatibility issues on the last install.
Thank you for those images, they are very helpful.
-
@robh-0 I use to run pfsense on esxi for years.. But since I got my sg4860, and got rid of my esxi box for just a nas that I run a few vms I need, I haven't played with it since prob 2018. I was running esxi 6.7 back then.
I do have my old esxi box still on the shelf - maybe I could fire it up and install the 7 update 3c I just downloaded - and just leave it for reference when others have issues with vlans, etc.
I do have a esxi flex mini switch I could play with - and throw that into the mix as well - just not sure when I could get around to that.. Maybe this weekend.. I had it up and running with some vlans for test... But until I replace the other vlan switch I have behind my tv with it, I pulled it out of the network. Overall I wasn't all that impressed with it, other than how freaking small it was ;)
-
@johnpoz
I hope you have an Intel NIC in the VMvare Box ....
My little "NUC lookalike" (Acer) has a realtek , and i slipstreamed the "old" realtek (linux subsystem) driver in.Guess what subsystem has been deprecated in 7.x ...
On 6.7 forever ....
/Bingo
-
@bingo600 Its an old Gen 8 HP microserver.. I do believe I had a 2 port HP intel nic in it.. low profile ;)
Have to pull it off the shelf and fire it up.. I had pulled all the disks out of it, but think I have 128GB ssd I could use to test it out with.. enough to get a pfsense up and running on it, from looking real quick, looks like freebsd 12.x was added in 6.7 - so its possible something odd with the old 6.5 and freebsd.. You could quite often get OS that were not officially supported to work.. But always better to be an actual supported guest OS.
-
@robh-0 OK here's the new install:
VLAN assigned to VLAN Tag 108, I still can't ping it from 192.168.108.20. Like I was saying, I know I'm missing something simple here, just can't figure out what. I've also tried sending it a trunk with VLAN108 tagged, and I've tried sending it just 108, untagged
@bingo600 Twin, real Intel Quad Port cards. Everything is working great. I did notice though when I did the upgrade that they said my poor old Xeon E3-1246 v3 may not be supported in future releases.
-
Oh, and I did add the firewall rule back, just forgot to share that.
-
@robh-0 ok if your setting it up on pfsense as tagged, then it needs to be connected to a vswitch/port group that has vlan 4095 ID set so it doesn't strip the tags.
Did you assign anything on this em2? If you have your lan interface on a vswitch/port group with vlan ID 4095 set, then put this vlan on that interface on pfsense. looks like em1
-
@johnpoz This is what I have.
I tried it assigned as shown in the first screenshot in my last page, and a trunk port going to that. Doesn't work. So then I tried native 108 to that port and just using EM2, that doesn't work. And I tried every combination of those I could think of, and saw no change. I'm starting to get to the point that I want to build a non-VM machine for this, but that seems like a waste of my closet space and would add more heat, I really want to use the VM host.
-
@robh-0 and what interface of your virtual pfsense is connected to thse port groups. They have the same mac..
So what specific interface is connected to these. If your going to connect em2 to vlan 108, then there should be a different mac. And this em2 should just be native, no vlan setup on it.
If your going to put your vlan on the interface connected to this 4095 port group. Then your vlan should be on that - em1? since I take that is your lan interface and working..
if your sending tagged traffic then the interface that is in the 4095 should see it and the tag..
Example.. If I sniff on my igb2, I can see tagged traffic.
Set to full verbose level
-
@johnpoz I was just thinking that. I'm going to delete EM2 and put it on EM1. I was following the official documentation from pfSense as best I could, and they had me setting up a second interface - a virtual one. So give me a minute and I'll reconfigure.
-
@robh-0 See my edit with showing tags... If your sending stuff tagged that whatever interface is on the 4095 then your vlans are working... If your not seeing any tags, then you have a problem..
You can also see there is untagged traffic being seen as well.
-
@johnpoz I switched it all over, got rid of em2, put the VLAN on em1, ensured that it is on 4095 and I'm not seeing those tags. I have to go to some conference calls (dang work getting in the way of my fun) but once I can look over my configs on my switches and such, I'll get back. I did try setting up a port group on the switches so that all VLANs were tagged, but that made no difference either.
-
@robh-0 well if your on a vlan id 4095 on yoru port group, and your not seeing any tagged traffic, then there is something wrong before pfsense. Pfsense has no control over what gets sent to its interface..
You should be seeing arps in your vlan for example.. You should be seeing pings to pfsense vlan with the tag, etc.. Does your device in the vlan see the mac of the interface in pfsense vlan interface? when you try and ping it? If not then no it would never send traffic - but you should see the arp requests even if pfsense didn't answer, etc.
If pfsense never sees the arp, then it couldn't answer, if can not answer with its mac - then no you couldn't have any conversations. Or other traffic. If your sniffing on full verbose, and not seeing the tagged traffic since 4095 shouldn't strip any tags, then traffic is not going to pfsense to do anything with.
With vlan ID set at 4095 you are in VGT mode
-
@johnpoz I can't figure this out. I fired up Wireshark on my laptop and things are really weird. If I connect to the default LAN, and ping the pfSense box, that works just fine. If I put my laptop on the VLAN and then try to ping the VLAN interface on pfSense, not only do I not see the pings, but in Wireshark I never see the pings go out from the laptop when I try and ping the pfSense VLAN interface! I have no idea why this is happening, you'd think I'd see the attempts. If I plug another device into the VLAN and ping it from my laptop, it works.
I'm digging in even deeper to see if I can get this figured out over on the Unifi forum, but I'm pretty confused how this could happen. All the devices are on the same switch, so it's not like I have an issue between switches.
I'm kind of bummed because I need these VLANs to work to be able to migrate off of the USG.
-
@robh-0 said in I can't get VLANs to work / No DHCP:
never see the pings go out from the laptop when I try and ping the pfSense VLAN interface
As I stated if you do not get a mac back, then no you can not actual send a ping - you need a mac to be able to send traffic.
If you sniff on your pfsense vm parent vlan interface - and you don't even see the arp, then the traffic is not getting to pfsense - so it can not answer even arp, etc.
Look in your arp table on your machine - do you see a mac for this IP...
You have something blocking it, either your switch config is wrong, or something not setup correctly in esxi.. But if pfsense never sees the traffic - then nothing it can do.. If your switch was sending tagged packets to esxi, and you have the vswitch set to 4095 then you should see the tags at pfsense, even if you don't have anything setup for that tag on the virtual nic in pfsense. Take your switch out of the equation, and connect your pc or laptop direct to this this physical interface you have on the esxi box.. Now set your nic on your interface on this PC/laptop to send tags.. Should be able to do that in the advanced settings on the interface..
-
@johnpoz Good point! My brain was fried at the point I posted that, thank you for being the voice of reason. :) I'll dig into it more today if I can get around to it.
