Replace 2 port with 4 port nic card

Sergei_Shablovsky

@gertjan said in Replace 2 port with 4 port nic card:

There always might be some trial and error, as it's unknown how freebsd upon boot numbers the devices. If your new cards have another driver name, like "igb", the other driver name order might change.
In my my case, em3 becomes em0 em4 becomes em1 as the orginbal em0 and em1 are now vanished (replaced).

Thank You for detailed answer!

Could You be so please to explain:
On which principles FreeBSD (because pfSense are on top of FreeBSD) giving the NICs ports identity?

From my knowledge, there are involved (from HIGHT IMPACT to LOW):
motherboard schematics,
BIOS subsystem,
NICs mechanical interface (PCI, PCI-e, etc...),
NICs order in motherboard slots,
FreeBSD drivers,
...
?

Thank You for help and opinions!

stephenw10

The only things that use the hardware device name directly are things that can be (or are required to be) added to an unassigned parent interface.
So VLANs, QinQ, LAGG, PPPoE.

Everything else is abstracted to the assigned interface names to make re-assigning much easier.

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

The only things that use the hardware device name directly are things that can be (or are required to be) added to an unassigned parent interface.
So VLANs, QinQ, LAGG, PPPoE.

Everything else is abstracted to the assigned interface names to make re-assigning much easier.

Thank You, Steve!

I need to add useful utility (from a FreeBSD userforum answer:

=====================================
Two options come to mind.

Manually set interface names in /boot/device.hints
Code:

hint.igb.0.at="pci0:6:0:3"
hint.igb.1.at="pci0:6:0:2"
hint.igb.2.at="pci0:6:0:1"
hint.igb.3.at="pci0:6:0:0"
hint.igb.4.at="pci0:9:0:0"

Drawback here is any PCIe card install/removal could change pci bus address of your interfaces and need fixup.

Then Eric A. Borisch's port.
sysutils/ethname

[NEW PORT] sysutils/ethname: boot-time (re)naming of ethernet devices by MAC

An rc-script for pinning an ethernet network name to a MAC address.

--

*This isn't typically needed on PCIe systems, but for systems with multiple
USB ethernet (ue) devices, which seem to like to come up in
non-deterministic order, it is very helpful; doubly so when the system in
question is a router / firewall where the network config and security
concerns vary wildly from one device to the next.

It could also be of use for traditional NICs (PCIe) when adding a new card
to a system, for example, and ensuring that the existing, previously
configured device sticks to the MAC address, and not having to worry about
which ends up /dev/xxxN vs /dev/xxxM.

The script inserts itself before netif, waits an adjustable delay for the
expected devices to appear, and then renames them as requested by the user.
All of the device configuration, pf, etc., can be written with the new
names. It does not attempt to automatically handle devices added after boot.*

--

Usage is described in the script, but effectively:

$ cat /etc/rc.conf
ethname_enable="YES"
ethname_foo0_mac="aa:bb:cc:dd:ee:00"
ethname_bar_mac="aa:bb:cc:dd:ee:01"

# Optionally, otherwise it'll just enumerate all ethname_*_mac vars:
ethname_names="foo0 bar"

stephenw10

Yes, you can use PCI device wiring to set the NICs by their bus location as shown there.

But if you do that and have to re-install ever those values will be lost the interfaces re-ordered. It's better to just configure them in the enumerated order IMO.

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

Yes, you can use PCI device wiring to set the NICs by their bus location as shown there.

But if you do that and have to re-install ever those values will be lost the interfaces re-ordered. It's better to just configure them in the enumerated order IMO.

Steve
Thank You for answering, Steve!

So, If I understand correctly, NetGate have no any step-by-step user guide for most frequent situation:

replace one NIC to another in case increasing port speed (for example from 100Mb/s -> 1Gb/s, with a same number of ports);
replace one NIC to 2(two) NICs in case increasing port speed & numbers of ports (for example 4 x 100Mb/s -> 2 x 1Gb/s + 4 x 1Gb/s);

with minimal configuration editing and minimal risks to losing settings ?

If so, this is VERY strange situation from user care point of view, because of MILLIONS of users of pfSense CE worldwide have the identical difficulties in case when they need upgrade NICs on their installations...

stephenw10

For most users in that situation:

Shutdown
Replace the NIC(s)
Boot back up
Re-assign the interfaces

Done.

It's far more common to replace the hardware entirely but the same applies really if the interfaces are different.

Steve

Gertjan

I did exactly that half a year ago :
I replace my 3 single 'realtek' 100 Mbits NICs for one quad NIC 1 Gbit Intel NIC.

I rebooted after the replacement ones, to see what happens, and assigned my new drivers em0 to em3. replacing the old re0, re1 and re3 with a new em0 em1 and a em2 - the em3 being a non assigned spare.
With this info, the current config file, I edited my previous backup up config, and imported it.
Done.

Btw using the CE version.

Sergei_Shablovsky

@stephenw10
Thank You! :)

Sergei_Shablovsky

@gertjan
Thank You!

Sergei_Shablovsky

@gertjan said in Replace 2 port with 4 port nic card:

I did exactly that half a year ago :
I replace my 3 single 'realtek' 100 Mbits NICs for one quad NIC 1 Gbit Intel NIC.

I rebooted after the replacement ones, to see what happens, and assigned my new drivers em0 to em3. replacing the old re0, re1 and re3 with a new em0 em1 and a em2 - the em3 being a non assigned spare.
With this info, the current config file, I edited my previous backup up config, and imported it.
Done.

Btw using the CE version.

So, because I have several NICs installed on one motherboard, and need to replace only 2(TWO) of them,
At the first step I change one NIC, then correct Interface assigning, then reboot. All working properly on that stage.

At the second stage I replace another NIC and... pfSense stuck on booting on Starting DNS Resolver...

Rolling back on a previous NIC not changing situation: just stuck on the same Starting DNS Resolver....

Please help :)

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

For most users in that situation:

Shutdown

Replace the NIC(s)

Boot back up

Re-assign the interfaces

Done.

It's far more common to replace the hardware entirely but the same applies really if the interfaces are different.

Steve

System stuck on a “Starting DNS Resolver...”

stephenw10

It can wait there some time if the NIC you replaced was the WAN so it now has no valid upstream connection. It should timeout and move past that though.

Try hitting ctl+t what process is it waiting for?

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

It can wait there some time if the NIC you replaced was the WAN so it now has no valid upstream connection. It should timeout and move past that though.

No, NICs belong to internal LANs.

Waiting within 7, 10, 20, 45min,- no changes

Try hitting ctl+t what process is it waiting for?

load: 1.06 cmd: unbound 68727 [running] 462.68r 462.44u 0.07s 100% 98180k

stephenw10

Hmm, hard to imagine how changing a NIC could prevent Unbound starting correctly. Unless maybe it was configured to listen on a set of interfaces that includes one no longer present?

Can you ctl+c past that?

It's probably going to be fastest to reinstall and restore the config. That way you can be sure it's not a hardware compatibility issue.

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

Hmm, hard to imagine how changing a NIC could prevent Unbound starting correctly. Unless maybe it was configured to listen on a set of interfaces that includes one no longer present?

I come to the same conclusion.

Can you ctl+c past that?

Yes, and put me in a shell prompt.

It's probably going to be fastest to reinstall and restore the config. That way you can be sure it's not a hardware compatibility issue.

There are a lot of other small tunings in a system, so reinstalling would be latest option.

Steve, because I able to go to prompt may be possible to start pfSense's re-assigning shell script to assign interfaces to ports ? (Like starting from pfSense's Main Menu, options 2) Set interface(s) IP address

How to doing this and no breaking any other preferences?

Which script may I starting manually ?

stephenw10

You could try editing the config at the prompt to disable Unbound and then rebooting.

Or you could try to rerun /etc/rc.bootup.

What NIC did you replace and what did you put in instead?

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

You could try editing the config at the prompt to disable Unbound and then rebooting.

Could You be so friendly to write exactly the code that I must edit?

Or you could try to rerun /etc/rc.bootup.

Hm, stuck on a same point...
Starting DNS Resolver...

What NIC did you replace and what did you put in instead?

I just replace old cards on a new version, the same 2 head, the same PCIe connection, the same brand. Only chip & bounded electronic parts are new because of design.

Sergei_Shablovsky

@stephenw10
Steve, because I able to go to prompt may be possible to start pfSense's re-assigning shell script to assign interfaces to ports ? (Like starting from pfSense's Main Menu, options 2) Set interface(s) IP address

stephenw10

To disable Unbound remove the <enable> line from the config section in /cf/conf/config.xml:

	<unbound>
		<enable></enable>
		<dnssec></dnssec>
		<active_interface></active_interface>
		<outgoing_interface></outgoing_interface>
		<custom_options></custom_options>
		<hideidentity></hideidentity>
		<hideversion></hideversion>
		<dnssecstripped></dnssecstripped>
	</unbound>

There is not an interface mismatch there. It would have stopped at the prompt to reassign them before that point. Something else is preventing that boot.

To bring up the console menu run: /etc/rc.initial

Steve

Gertjan

@sergei_shablovsky said in Replace 2 port with 4 port nic card:

stuck on booting on Starting DNS Resolver...

With this unbound setting :

?

Still, if a WAN type interface is assigned during boot, that is, there is one according the systems configuration settings, but you removed the physical interface, it's a basic "maintaining the patient while doing open hart surgery". Normal that the patient needs some cardiac assistance while doing so.
Changing NICs is normally not done every morning ;)

Replace 2 port with 4 port nic card

Then Eric A. Borisch's port. sysutils/ethname

Then Eric A. Borisch's port.
sysutils/ethname