Can't attach interface {} to bpf device /dev/bpf0
-
RESOLVED: The root cause of this issue was with a "feature" of the server motherboard - BMC/IPMI. While live booting into a normal linux distro for testing I noticed "NSCI < common ISP DHCP IP address>" flashed on the screen briefly on boot. After digging in the BIOS I discovered that something called "NSCI" was grabbing the DHCP address. I am clearly not an expert and don't fully understand what these mgmt tools are used for but after changing NSCI from "unspecified" to "static" my problem was resolved!
Hello all,
I spent most of my Friday troubleshooting this issue on IRC and I still am stuck. I'm hoping someone can spot my issue.
Background: I have CenturyLink DSL via PPPoE (modem was in transparent bridging mode) but just signed up/switched to a regional company that is now offering fiber in my area! I am in an apartment so I just have a ethernet jack in my unit with no access to the ONT.
Problem: Two different test laptops (Windows and Linux) directly connected to the jack get a DHCP address from the new fiber ISP. Pfsense does not get a DHCP address. The error in System Logs when I go to Status > Interfaces and renew the DHCP address on WAN is the following (pfsense spins for 2ish minutes when I do):
/status_interfaces.php: The command '/usr/local/sbin/dhclient {$ipv} -d -r -lf '/var/db/dhclient.leases.vtnet0' -cf '/var/etc/dhclient_wan.conf' -sf '/usr/local/sbin/pfSense-dhclient-script'' returned exit code '1', the output was 'Internet Systems Consortium DHCP Client 4.3.6-P1 Copyright 2004-2018 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Listening on BPF/vtnet0/fe:da:1e:fd:1d:91 Sending on BPF/vtnet0/fe:da:1e:fd:1d:91 Can't attach interface {} to bpf device /dev/bpf0: Device not configured If you think you have received this message due to a bug rather than a configuration issue please read the section on submitting bugs on either our web page at www.isc.org or in the README file before submitting a bug. These pages explain the proper process and the information we find helpful for debugging. exiting.'
At one point I also took a packet cable of WAN and got back this: https://gist.github.com/marshallford/e1c7fd2319cdc7e86604e6d7bbfe4882
If you run those IP addresses through ipinfo.io they seem to be owned by the fiber company. Strange right?ISP: I called the ISP early on and they do not do any MAC filtering. The support rep also mentioned they could see my Asrock server mobo NIC in their list of past/current devices along side my laptop's MAC. They also confirmed they only hand out one dynamic IP per customer (which makes sense, this is a normal residential plan).
Setup: Proxmox 6, pfsense VM. See imgur gallery for VM info and bridge setup. Two physical NICS, one cable goes to the fiber ethernet jack (WAN), the other goes to a 24 port unmanaged switch (LAN).
What I have tried:
- Checked cables
- Confirmed link lights when the fiber is plugged into the "WAN" nic
- Proxmox 5.X and Proxmox 6.X
- E1000 for the NIC model vs VirtIO
- Spoofing MAC in both pfsense and in Proxmox
- No WAN entries in the ARP table - that doesn't seem good!
- Restarting the host (Proxmox), restarting pfsense
- Factory resetting pfsense
- Fresh VM of pfsense 2.4
- Lastly I upgraded the fresh pfsense VM to the latest 2.5 dev snapshot.
Acting as an ISP that does DHCP: Because I was using PPPoE before fiber I wondered if my pfsense setup ever worked with ISPs that used DHCP. To test this, I reset my CenturyLink modem/router combo that was in bridging mode back to factory settings and set it up normally. I plugged in a cable from a lan port on the back of that device and into the WAN port of the pfsense hardware. Pfsense grabbed a DHCP address right away when I hit "renew" in the UI. Weird right? Do real ISPs do DHCP differently from how a consumer modem/router combo would hand out DHCP addresses in a home network? What is going on?
Thanks to all who even try to make sense of this,
Marshall
-
EDIT: Full TCP dump of switching cables to fiber ISP and then renewing/releasing DHCP in the UI: https://gist.github.com/marshallford/f6fd85988b2ceaed882cec37038efcfd
EDIT 2: TCP dump of plugging in fiber directly to linux laptop: https://gist.github.com/marshallford/c67afcfb121c13f20df8dc830fc50b13