(Solved) pfSense blocks static and DHCP ip requests from unraid when bridging is enabled

johnpoz · Dec 26, 2018, 4:10 AM

Is pfsense running on hardware or a VM?

brandon3055 · Dec 26, 2018, 4:35 AM

Ok so i got pfsense reinstalled and even with factory default settings there is no change.

So for now i am going to forget about DHCP and go back to debugging with a static ip because i assume its the same issue affecting both DHCP and static. I also really only care about static anyway.

Now i dont think i explained exactly what happens in this configuration so since its probably important.
If i set a static ip in unraid (192.168.1.133) unraid has access to LAN and devices on LAN can access unraid. The only thing unraid can not access is pfsense. If i try to ping pfsense from unraid i get
"From 192.168.1.133 icmp_seq=575 Destination Host Unreachable"
Similar thing if i try to ping unraid from pfsense just no response.
I dont see anything in the pfsense logs that looks related to this.

@johnpoz said in pfSense blocks static and DHCP ip requests from unraid when bridging is enabled:

Is pfsense running on hardware or a VM?

Hardware.
Its running on a Pentium 4 dual core @3.20GHz with 1GB of ram. Its a pretty old system but for pfsense its fine... i think..

Edit: Once i get some better hardware for unraid i want to visualise pfsense (or switch to opensense) and run it in a vm under unraid with with a couple nics passed directly through to the pfsense vm. But thats assuming i can get this issue fixed.

johnpoz · Dec 26, 2018, 5:10 AM

Your problem seems to be something wrong with unraid..

So your default rules on pfsense for its lan 192.168.1.0/24 I take it pfsense is say 192.168.1.1? is any any - the default. If you set your unraid to be 192.168.1.133/24 and it can not ping pfsense then you have something wrong with your connectivity. When you ping for pfsense IP say 192.168.1.1 do you see an arp entry for it in unraids arp cache? Can pfsense ping 192.168.1.133 what is in pfsense arp cache for this .133 IP?

Your saying your VMS you create on unraid say 192.168.1.150/24 or something can ping pfsense and they can get to the internet through pfsense?

brandon3055 · Dec 26, 2018, 5:50 AM

And i figured it out.... Thankyou so much for suggesting i check the arp cache that lead me to this

Which is how i discovered that for whatever stupid reason pfsense sets its lan mac address to "00:00:00:00:00:00" by default. I mean why not just use the mac of the lan nic?

Anyway i added a mack address and suddenly EVERYTHING WORKS!!!

Thankyou for your help. Any idea why pfsense does not supply a valid mac on the lan interface by default?

Gertjan · Dec 26, 2018, 6:00 AM

@brandon3055 said in (Solved) pfSense blocks static and DHCP ip requests from unraid when bridging is enabled:

why pfsense does not supply a valid mac on the lan interface by default?

pfSense isn't supplying MAC addresses to interfaces. It's the other way around.
Although users, that is, the 'admins', could change these MAC addresses.
Without being an expert here, I guess, when bridging interfaces, only one MAC from one interface is retained, all the others - members of the bridge, are discarded.

johnpoz · Dec 26, 2018, 11:20 AM

He is not bridging on pfsense... He is bridging on his unraid box for VMs

Why your pfsense box had 0 for mac have no idea.. That sure and the hell is not any sort of default thing.. Had you messed with bridges on pfsense? Had you played with mac cloning or setting the mac on pfsense?

You mean you set a static mac for pfsense on your unraid, or you had to add a mac to the interface on pfsense?

brandon3055 · Dec 26, 2018, 12:16 PM

When i discovered the issue it was a fresh unmodified install.

johnpoz · Dec 26, 2018, 12:35 PM

Have never heard of a nic not having a mac.. So what mac did you give it exactly? You just made one up?

6a:62:b2

And you did that where exactly?

What does your dmesg show for the nic as it boots?

BTW - ip 192.168.1.8 for pfsense would not be a fresh unmodified install.. 192.168.1.1 is the default IP that is on the lan interface.

brandon3055 · Dec 26, 2018, 2:25 PM

It is just am random mac from a random generator i found. I set it on the LAN interface page

If i leave it set to the default blank and then go to the interface assignments page i see this.

also

I rebooted the box a couple times but i could not catch the dmesg. Its nothing but a blur of white text that even my phone could not capture.

@johnpoz said in (Solved) pfSense blocks static and DHCP ip requests from unraid when bridging is enabled:

BTW - ip 192.168.1.8 for pfsense would not be a fresh unmodified install.. 192.168.1.1 is the default IP that is on the lan interface.

Rite the one thing i did do was go through the initial basic setup which involved setting the IP.

Gertjan · Dec 26, 2018, 2:39 PM

@brandon3055 said in (Solved) pfSense blocks static and DHCP ip requests from unraid when bridging is enabled:

Its nothing but a blur of white text that even my phone could not capture.

?

You're an admin, so use the admin tools ^^
Go god mode (console) : option 8 and type the command

dmesg

Still to fast ?

Use this one :

dmesg | grep 'Ethernet'

If you found a " 00:00:00:00:00:00" then yes, things go downhill fast.

johnpoz · Dec 26, 2018, 2:49 PM

Yeah its going to need a mac that is for sure.. msk I assume that is a marvel nic.. What are the specifics of it? You should see that in the dmesg.

As stated from console if going by too fast you could also more the output

dmesg | more

Or grep for msk0

dmesg | grep msk0

Or from the web gui
Diagnostics / Command Prompt

run dmesg there..

brandon3055 · Dec 27, 2018, 6:26 AM

Using: dmesg | grep 'msk' (LAN)

mskc0: <Marvell Yukon 88E8055 Gigabit Ethernet> port 0xce00-0xceff mem 0xfdbfc000-0xfdbfffff irq 17 at device 0.0 on pci3
msk0: <Marvell Technology Group Ltd. Yukon EC Ultra Id 0xb4 Rev 0x02> on mskc0
msk0: Using defaults for TSO: 65518/35/2048
miibus1: <MII bus> on msk0
msk0: link state changed to DOWN
msk0: link state changed to UP
msk0: link state changed to DOWN
msk0: link state changed to UP
msk0: link state changed to DOWN
msk0: link state changed to UP

Using: dmesg | grep 're0' (WAN)

re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0xde00-0xdeff mem 0xfdcff000-0xfdcfffff,0xfdcf8000-0xfdcfbfff irq 16 at device 0.0 on pci2
re0: Using 1 MSI-X message
re0: Chip rev. 0x2c000000
re0: MAC rev. 0x00200000
miibus0: <MII bus> on re0
re0: Using defaults for TSO: 65518/35/2048
re0: Ethernet address: 80:1f:02:45:29:8f
re0: netmap queues/slots: TX 1/256, RX 1/256
firewire0: <IEEE1394(FireWire) bus> on fwohci0
sbp0: <SBP-2/SCSI over FireWire> on firewire0
firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0)  (me)
firewire0: bus manager 0
re0: link state changed to DOWN
re0: link state changed to UP

Gertjan · Dec 27, 2018, 6:40 AM

Ok for the Realteck interface, is has a MAC :
@brandon3055 said in (Solved) pfSense blocks static and DHCP ip requests from unraid when bridging is enabled:

re0: Ethernet address: 80:1f:02:45:29:8f

The msk0 interface : no.

I tend to say : check with "Marvell Yukon 88E8055 Gigabit Ethernet" support how you can set it's original MAC back.

brandon3055 · Dec 27, 2018, 6:57 AM

@gertjan said in (Solved) pfSense blocks static and DHCP ip requests from unraid when bridging is enabled:

I tend to say : check with "Marvell Yukon 88E8055 Gigabit Ethernet" support how you can set it's original MAC back.

honestly i don't really care. Its working now and i have no problem with using a random generated mac for the LAN interface.
If this was caused by an issue with my specific nic then thats fine. The override fixes it so thats all that matters. I just initially assumed this was caused by pfsense.

johnpoz · Dec 27, 2018, 9:52 AM

In my some 30 years in the biz, I can honestly say I do not recall ever seeing such a thing.. How could the nic not have a mac?

Does it get a mac if you boot some other os, so you use a different driver? Has to be some sort of issue with the driver? Or the thing is faulty out of the gate? I would be more concerned to be honest.. I wouldn't trust using it.. Until you figure out what exacty causes such a problem..

I have setup 10's of thousands of machines over the years - never seen such a thing.

Had you flashed the nics firmware at some point? What was on the machine before you installed pfsense on it?

It was working before with pfsense? What version of pfsense was that - the version of freebsd would of been different, etc.

Doing a google for all zeros mac and marvel does have quite a few hits... Not seeing any sort of rhyme or reason to what finding, most people just seem to get a new nic ;)

The only thing close is flashing a dd-wrt firmware on old router set the mac to same thing as everyone else using the firmware, and this would cause problems if someone else with your ISP had also flashed theirs and they got on first.. That took a bit to track down when saw.. Fix was to get the original mac back..

I wold have to say something wrong with the driver or the nic bad firmware flash or something in the factory... What you did by setting a mac is a work around.. But personally I would just get a different nic if you can not track down a reason to why mac went missing.

brandon3055 · Dec 28, 2018, 3:19 AM

The nic is the built in nic on an old acer pc that looks like it originally came out of a school. I have been running the system with pfsense for about 3 years now and the nic has never caused any issues other than this one. This is only running my home network so nothing critical. It it does ever cause issues i will just throw in a different nic ore move it to a different system.