My Installation Experience
-
Since you said ANYTHING:
I had the same if not similar symptoms. After pulling my hair out for a day it turned out to be the Cat6 cable going from the modem to the router. Replaced it with another cable and it instantly worked. I don't know why as that 'bad' cable works fine elsewhere.
Anyway you asked for 'anything' :)
-
I thank you very much for your replies.
@edmund…no <expletive>way!!!!! that's to easy and I never thought to try it
@AR15USR .. I did purchase a brand new cat6 cable for the box but I can't for the life of me remember if I tried it on pfsense or if I purchased it and swap it out just before the install of the other five UTM;'s I tried....
2 good suggestions, thanks. I'll try to give em a go this weekend and report back here...no promises though, I've put in 50hrs already this week and it doesn't look like it's going to end anytime soon...I might not get a chance to image the drive until next weekend.
Thanks!
Appreciate your efforts.</expletive>
-
"was told to connect the "router" (PFsense) to the cable modem just before bedtime, and leave the modem and the router powered on and connected. "
So you have 25 years exp, and you followed those nonsense instructions??
So did you think to sniff on and see what is happening without getting an IP? Where you seeing dhcp offers comeback from your discover?
If you have an actual "cable" modem then reboot it.. Wait til it gets sync then then boot up your pfsense box. Those nics are not really fav nics with freebsd/pfsense - do you have a intel nic you can use?
-
I would have to agree. When I saw "25 years experience" I said to myself, it all depends on what things you have done, how deep you have gotten and how much learning you do on your own otherwise.
Back to the subject, I have seen this 'sort of issue' time and time again in the forums.
People simply cannot think logically when it comes to certain things. Perhaps because they have never done any (embedded) programming, or don't understand 100% how the network stack works at the physical (L2) and the logical levels (L3), plus the slight 'weird' behavior of certain devices.Here it is:
When a cable modem boots, it will talk to the first ethernet (L2) device that it gets a requests from.
After that, t will NOT talk/respond to ANY other device on the same network (WAN). Usually you only have 1 device on the WAN side, so it's OK, BUT…
The issue comes up when you go and try to connect a DIFFERENT L2 device to the WAN.
The cable modem remembers the MAC Address of that first device it talked to, and again, will NOT respond to anybody else.
Why do they do this? Mostly historical reasons (before NAT routers, to only allow people to connect 1 PC/device).SOLUTION: REBOOT cable modem every time you change WAN devices!!!
I get around this in my redundant CARP WAN setup, where both the Master and Backup pfsense use a 'specific' Mac Address on the WAN.
In the Backup pfsense, I did write/implement a shell script that gets triggered when WAN Carp goes to Master (change WAN MAC to the 'specific' Mac Address) and another script that gets triggered when WAN Carp goes to Backup (change WAN MAC to the built-in Mac Address).This allows me to have 2 pfsense (virtual) machines acting as CARP Master/Backup for redundancy on the WAN WITH a Non-Static IP Address (DHCP from ISP).
"was told to connect the "router" (PFsense) to the cable modem just before bedtime, and leave the modem and the router powered on and connected. "
So you have 25 years exp, and you followed those nonsense instructions??
So did you think to sniff on and see what is happening without getting an IP? Where you seeing dhcp offers comeback from your discover?
If you have an actual "cable" modem then reboot it.. Wait til it gets sync then then boot up your pfsense box. Those nics are not really fav nics with freebsd/pfsense - do you have a intel nic you can use?
-
^ agreed.. The cable modem likes to bind to that mac of the device connected to it. I do a somewhat sim trick when I want to run a different vm of pfsense or another router/firewall distro to test out by just using the same mac on that vm.. Shutdown the old vm, turn on the new vm with the same mac = no reboot of the cable modem.
-
I think that the "how a cable modem works" discussion (while educational and useful) is getting away from the root of the problem here - which is that the auto-negotiate interface speed function is not working well - to put it politely.
I've seen pfSense, on multiple occasions, over the last week completely fail to auto-negotiate a reliable interface speed and lock me out of the system with "Bad Gateway" errors or simply hang up the system. Just what is going on to make a 2GHz, 4 core machine become virtually unresponsive is a mystery but it can be fixed by setting the WAN interface to 100baseTX and ignoring the stern "WARNING: MUST be set to autoselect (automatically negotiate speed) unless the port this interface connects to has its speed and duplex forced." on the interface page.
-
I think that the "how a cable modem works" discussion (while educational and useful) is getting away from the root of the problem here - which is that the auto-negotiate interface speed function is not working well - to put it politely.
I've seen pfSense, on multiple occasions, over the last week completely fail to auto-negotiate a reliable interface speed and lock me out of the system with "Bad Gateway" errors or simply hang up the system. Just what is going on to make a 2GHz, 4 core machine become virtually unresponsive is a mystery but it can be fixed by setting the WAN interface to 100baseTX and ignoring the stern "WARNING: MUST be set to autoselect (automatically negotiate speed) unless the port this interface connects to has its speed and duplex forced." on the interface page.
Since you are using your personal experience as anecdotal evidence, I'll do the same. I have been using monowall first and then pfSense for years on a wide variety of hardware: embedded, run of the mill PCs, expesive servers and all kinds of VMs and I never had to set any ports to anything other than auto-negotiate.
-
Since you are using your personal experience as anecdotal evidence, I'll do the same. I have been using monowall first and then pfSense for years on a wide variety of hardware: embedded, run of the mill PCs, expesive servers and all kinds of VMs and I never had to set any ports to anything other than auto-negotiate.
That's been my experience too until this month, I've never seen this problem before. I started with FreeBSD, moved to M0n0wall, and then pfSense about 10 years ago. To be fair, at that time 100Mb was fast and 1000Mbs was too expensive to worry about and until recently all my firewalls used 10/100 NICs so auto-negotiate was simpler.
I'm open to the possibility that this is could be a hardware issue, that NICs in the current firewall may not actually be real Intel NICs and that it could be a FreeBSD issue. The thing is that if it's a real bug it's only going to show up when both ends of the cable are connected to 1000Mbs capable NICs and one of them refuses to auto-negotiate a 1000Mbs connection. Under almost any other circumstances everything will work correctly.
-
Here it is:
When a cable modem boots, it will talk to the first ethernet (L2) device that it gets a requests from.
After that, t will NOT talk/respond to ANY other device on the same network (WAN). Usually you only have 1 device on the WAN side, so it's OK, BUT…
The issue comes up when you go and try to connect a DIFFERENT L2 device to the WAN.
The cable modem remembers the MAC Address of that first device it talked to, and again, will NOT respond to anybody else.
Why do they do this? Mostly historical reasons (before NAT routers, to only allow people to connect 1 PC/device).SOLUTION: REBOOT cable modem every time you change WAN devices!!!
That's only correct, when you get only 1 dynamic IP from your ISP,
or you have used up all the available public IP's you get from your ISP.Per example, if you get 4 public IP's, and you have connected 4 devices, and you disconnect 1 of these 4 devices,
the new device cannot get a IP, untill the lease for the disconnected device is ended.By rebooting the cable modem, you can end the actives leases instantly.
After connecting a new device, this get a new IP and IP-lease is provided for the MAC-adress of the new device.This all depends on what type subscription you have from your ISP (1 IP or multiple IP's)
Back in the old days (2000), here in Belgium you only get 1 public IP from the ISP, and for more computers/laptops,
you had to pay extra for a subscription with 4 IP's.If today, and i plug in a switch with per example 16 ports right behind my cable modem (modem-only, no wifi), and i connect 16 different devices to this switch,
all 16 devices get instantly a dynamic public IP from my ISP.
With my subcription, i even got unlimited amount of public IP's available from my ISP.Grtz
DeLorean -
"i even got unlimited amount of public IP's available from my ISP."
IPv6 sure why not… But I find it hard to believe they just give you unlimited public ipv4 addresses..
-
Aren't public IPs, statically assigned (i.e. NON DHCP)??
in the US, and in Mexico and South America and UK (that I know), people only get 1 IP Address for residential service.
Even in the US, for Commercial service, you only get 2 IPs and you don't get them with DHCP, they are static, so you set them yourself.
Again, what you are saying is that in Germany, you can get up to 16 DHCP addresses from your ISP as a residential user? (Is this Ip V6?)
Here it is:
When a cable modem boots, it will talk to the first ethernet (L2) device that it gets a requests from.
After that, t will NOT talk/respond to ANY other device on the same network (WAN). Usually you only have 1 device on the WAN side, so it's OK, BUT…
The issue comes up when you go and try to connect a DIFFERENT L2 device to the WAN.
The cable modem remembers the MAC Address of that first device it talked to, and again, will NOT respond to anybody else.
Why do they do this? Mostly historical reasons (before NAT routers, to only allow people to connect 1 PC/device).SOLUTION: REBOOT cable modem every time you change WAN devices!!!
That's only correct, when you get only 1 dynamic IP from your ISP,
or you have used up all the available public IP's you get from your ISP.Per example, if you get 4 public IP's, and you have connected 4 devices, and you disconnect 1 of these 4 devices,
the new device cannot get a IP, untill the lease for the disconnected device is ended.By rebooting the cable modem, you can end the actives leases instantly.
After connecting a new device, this get a new IP and IP-lease is provided for the MAC-adress of the new device.This all depends on what type subscription you have from your ISP (1 IP or multiple IP's)
Back in the old days (2000), here in Belgium you only get 1 public IP from the ISP, and for more computers/laptops,
you had to pay extra for a subscription with 4 IP's.If today, and i plug in a switch with per example 16 ports right behind my cable modem (modem-only, no wifi), and i connect 16 different devices to this switch,
all 16 devices get instantly a dynamic public IP from my ISP.
With my subcription, i even got unlimited amount of public IP's available from my ISP.Grtz
DeLorean -
The ONLY 2 times I have seen an issue with auto negotiation, were hardware related, bad cable, bad connector or bad nic/port.
When you understand that the speed negotiation is very hardware dependent, you go chasing those ghosts somewhere else…
Here, I would say, try a different cable first, then a different NIC (all/most brands work, but Intel do have better quality and performance).
I think that the "how a cable modem works" discussion (while educational and useful) is getting away from the root of the problem here - which is that the auto-negotiate interface speed function is not working well - to put it politely.
I've seen pfSense, on multiple occasions, over the last week completely fail to auto-negotiate a reliable interface speed and lock me out of the system with "Bad Gateway" errors or simply hang up the system. Just what is going on to make a 2GHz, 4 core machine become virtually unresponsive is a mystery but it can be fixed by setting the WAN interface to 100baseTX and ignoring the stern "WARNING: MUST be set to autoselect (automatically negotiate speed) unless the port this interface connects to has its speed and duplex forced." on the interface page.
Since you are using your personal experience as anecdotal evidence, I'll do the same. I have been using monowall first and then pfSense for years on a wide variety of hardware: embedded, run of the mill PCs, expesive servers and all kinds of VMs and I never had to set any ports to anything other than auto-negotiate.
-
I now have to wonder, how/why/when I learned this logical/illogical quirk about cable-modems :-)
I think I noticed that a reboot was needed when changing WAN devices, while playing with virtual servers (esxi), so then I went on chasing the WHY, and figured out the modem remembered the MAC address of the previous device….and I wanted a way to keep connectivity without the need for a lengthily cable-modem reboot...
I have been using my virtual pfsense master/backup setup in 2 machines under ESXi, for the last 5 or 6 years and I could not be happier.
^ agreed.. The cable modem likes to bind to that mac of the device connected to it. I do a somewhat sim trick when I want to run a different vm of pfsense or another router/firewall distro to test out by just using the same mac on that vm.. Shutdown the old vm, turn on the new vm with the same mac = no reboot of the cable modem.
-
Ethernet autoneg is a function of the hardware, no? The most the software does is reinit the phy, perhaps put in a configuration, but the hardware is where it actually happens (unless I'm misrembering my BroadComm specs). Heck a lot of times "forcing" a configuration the software doesn't disable autoneg, it simply limits the configurations. Speed and duplex are both functions of autoneg, not just speed (and that's part of the problem if one end is autoneg the other forced. Duplex often fails to negotiate so what should be a 100/Full winds up at 100/Half on on end).
And yes, different hardware can have problems talking with other hardware. Lots of times it comes down to how the mfg interpreted a spec.
-
From the posts above it's clear that auto-negotiation is a somewhat murky subject dependent on many factors.
What puzzled me with my installation was that pfSense was showing that the auto-negotiated WAN NIC status was up but that there was no data connection or IP address. Yet it would reliably detect that there was an upgrade available, download it and install it. Upon rebooting it reported the NIC was up but pfSense continued to fail to pull an IP address from the cable modem until I manually set the interface to 100Mbs, disabling auto-negotiation.
I would have thought that if pfSense shows the NIC is up at 1000Mbs then it means that auto-negotiation has detected and set the link speed correctly. Apparently this is not the case.
I would have thought that it pfSense shows no IP address for the WAN connection then I don't have a connection - apparently this is wrong too since pfSense detected that an update was available, downloaded it, and upgraded just fine each time even though it was showing that it had not received an IP address from the modem.
Frankly, I found the symptoms and the diagnostic information confusing - and looking at the other message posted in this forum, I wonder if auto-negotiation issues might be a lot more common then people think.
-
"was no data connection or IP address. Yet it would reliably detect that there was an upgrade available, download it and install it. "
Dude what are you on?? That is just not possible now is it…
Kind of hard to talk on the internet without a IP address..
Where were you seeing that it had no IP, did it show the gateway up with NO ip? So your gui showed you what 0.0.0.0?? What did the console menu show that shows you interface IPs, what did ifconfig show??
-
The only relationship that autonegotiation has to an interface getting an IP address is that a link must be physically up for DHCP requests to go out and for responses to come back.
Edmund, was your pfSense WAN connected directly to your cable modem or was there a switch in between? Are you sure your cable modem was also set to autoneg?
Auto neg has 2 parts: speed and duplex. If one end is forced the other not, speed will often be correct, but duplex is wrong. Duplex wrong is one end thinks "full" the other thinks "half" and you wind up with a lot of errors on the interface.
-
"was no data connection or IP address. Yet it would reliably detect that there was an upgrade available, download it and install it. "
Dude what are you on?? That is just not possible now is it…
Kind of hard to talk on the internet without a IP address..
Where were you seeing that it had no IP, did it show the gateway up with NO ip? So your gui showed you what 0.0.0.0?? What did the console menu show that shows you interface IPs, what did ifconfig show??
This was an installation from a USB drive to the system disk so I was working from the default pfSense stats/dashboard "interfaces" display - my feeling is that the dashboard display is not to be 100% trusted since, as you point out, there must have been a connection there for the upgrade to occur. I don't remember the GUI showing any IP address although I had the green up arrow indicating the cable was plugged in. Looking at the WAN interface it show that it was configured as expected but it would not pull an address if I tried manually - the overall performance of pfSense was very slow unless the WAN interface was disabled so clearly something was going on in the background - two of the cores were at 100%
-
@mer:
The only relationship that autonegotiation has to an interface getting an IP address is that a link must be physically up for DHCP requests to go out and for responses to come back.
Edmund, was your pfSense WAN connected directly to your cable modem or was there a switch in between? Are you sure your cable modem was also set to autoneg?
Auto neg has 2 parts: speed and duplex. If one end is forced the other not, speed will often be correct, but duplex is wrong. Duplex wrong is one end thinks "full" the other thinks "half" and you wind up with a lot of errors on the interface.
The modem was directly connected to the WAN interface via a 6 foot length of CAT5 cable, no switch. In the end I got it to work by replacing the 5 foot CAT5 cable with a 25 foot length of CAT6 cable - with the CAT6 cable it connected at 1000Mbs but refused make a connection at any speed with the CAT5 cable.
The cable modem has no user accessible controls that I can find - I'll check on it's spec but I'd assumed that pfSense would auto-negotiate regardless - at least that's always been my experience in the past. I've ordered a bunch of CAT6 cables to try and do a bit more research into this, I have a ton of CAT5 cables in the bin.
-
cat 5 or 5e? How old are these cables?
