Replace 2 port with 4 port nic card

Sergei_Shablovsky

@gertjan said in Replace 2 port with 4 port nic card:

I did exactly that half a year ago :
I replace my 3 single 'realtek' 100 Mbits NICs for one quad NIC 1 Gbit Intel NIC.

I rebooted after the replacement ones, to see what happens, and assigned my new drivers em0 to em3. replacing the old re0, re1 and re3 with a new em0 em1 and a em2 - the em3 being a non assigned spare.
With this info, the current config file, I edited my previous backup up config, and imported it.
Done.

Btw using the CE version.

So, because I have several NICs installed on one motherboard, and need to replace only 2(TWO) of them,
At the first step I change one NIC, then correct Interface assigning, then reboot. All working properly on that stage.

At the second stage I replace another NIC and... pfSense stuck on booting on Starting DNS Resolver...

Rolling back on a previous NIC not changing situation: just stuck on the same Starting DNS Resolver....

Please help :)

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

For most users in that situation:

Shutdown

Replace the NIC(s)

Boot back up

Re-assign the interfaces

Done.

It's far more common to replace the hardware entirely but the same applies really if the interfaces are different.

Steve

System stuck on a “Starting DNS Resolver...”

stephenw10

It can wait there some time if the NIC you replaced was the WAN so it now has no valid upstream connection. It should timeout and move past that though.

Try hitting ctl+t what process is it waiting for?

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

It can wait there some time if the NIC you replaced was the WAN so it now has no valid upstream connection. It should timeout and move past that though.

No, NICs belong to internal LANs.

Waiting within 7, 10, 20, 45min,- no changes

Try hitting ctl+t what process is it waiting for?

load: 1.06 cmd: unbound 68727 [running] 462.68r 462.44u 0.07s 100% 98180k

stephenw10

Hmm, hard to imagine how changing a NIC could prevent Unbound starting correctly. Unless maybe it was configured to listen on a set of interfaces that includes one no longer present?

Can you ctl+c past that?

It's probably going to be fastest to reinstall and restore the config. That way you can be sure it's not a hardware compatibility issue.

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

Hmm, hard to imagine how changing a NIC could prevent Unbound starting correctly. Unless maybe it was configured to listen on a set of interfaces that includes one no longer present?

I come to the same conclusion.

Can you ctl+c past that?

Yes, and put me in a shell prompt.

It's probably going to be fastest to reinstall and restore the config. That way you can be sure it's not a hardware compatibility issue.

There are a lot of other small tunings in a system, so reinstalling would be latest option.

Steve, because I able to go to prompt may be possible to start pfSense's re-assigning shell script to assign interfaces to ports ? (Like starting from pfSense's Main Menu, options 2) Set interface(s) IP address

How to doing this and no breaking any other preferences?

Which script may I starting manually ?

stephenw10

You could try editing the config at the prompt to disable Unbound and then rebooting.

Or you could try to rerun /etc/rc.bootup.

What NIC did you replace and what did you put in instead?

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

You could try editing the config at the prompt to disable Unbound and then rebooting.

Could You be so friendly to write exactly the code that I must edit?

Or you could try to rerun /etc/rc.bootup.

Hm, stuck on a same point...
Starting DNS Resolver...

What NIC did you replace and what did you put in instead?

I just replace old cards on a new version, the same 2 head, the same PCIe connection, the same brand. Only chip & bounded electronic parts are new because of design.

Sergei_Shablovsky

@stephenw10
Steve, because I able to go to prompt may be possible to start pfSense's re-assigning shell script to assign interfaces to ports ? (Like starting from pfSense's Main Menu, options 2) Set interface(s) IP address

stephenw10

To disable Unbound remove the <enable> line from the config section in /cf/conf/config.xml:

	<unbound>
		<enable></enable>
		<dnssec></dnssec>
		<active_interface></active_interface>
		<outgoing_interface></outgoing_interface>
		<custom_options></custom_options>
		<hideidentity></hideidentity>
		<hideversion></hideversion>
		<dnssecstripped></dnssecstripped>
	</unbound>

There is not an interface mismatch there. It would have stopped at the prompt to reassign them before that point. Something else is preventing that boot.

To bring up the console menu run: /etc/rc.initial

Steve

Gertjan

@sergei_shablovsky said in Replace 2 port with 4 port nic card:

stuck on booting on Starting DNS Resolver...

With this unbound setting :

?

Still, if a WAN type interface is assigned during boot, that is, there is one according the systems configuration settings, but you removed the physical interface, it's a basic "maintaining the patient while doing open hart surgery". Normal that the patient needs some cardiac assistance while doing so.
Changing NICs is normally not done every morning ;)

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

Something else is preventing that boot.

In which part of system may I seek for a source of problem?

To bring up the console menu run: /etc/rc.initial

BSD told me that I have no sufficient permission...

Sergei_Shablovsky

@gertjan said in Replace 2 port with 4 port nic card:

Still, if a WAN type interface is assigned during boot, that is, there is one according the systems configuration settings, but you removed the physical interface, it's a basic "maintaining the patient while doing open hart surgery". Normal that the patient needs some cardiac assistance while doing so.

;) Thank You for trying to support me by making a little more smiley:) Now in Ukraine when You hearing sound alarm about bombing each 3-4h, hard to be normal...
So, because I need some “cardiac assistance”, I come to this “Hardware” section of Netgate forum :)

Changing NICs is normally not done every morning ;)
Yes, You are right, but as I wrote before in other threads:

time goes faster and faster each year, and 10 years ago we change NICs on a production once in 3-5 years, but now , after traffic become more and more heavy due streaming and hi-res cameras + social networks, we pushed to change NICs and applience each 2-3 years;
Netgate (even in CE edition) need to be responsible for product quality, and making ordinary maintenance procedures painless and as simply as possible both for newbies and for pro system administrators;

From that points of view most usable operation like changing NICs **must be troubles free and as simply as possible, and also well and detailed described in documentation”.

Anyway, from newbies, students and youth Technishian/ Sys Admins that using pfSense CE edition on a custom built DIY hardware at the first steps, after 2-3-5 years become a PRO sys admins / security admins in Enterprise and more and more buying Netgate official hardware and support plans.

So, well and detailed documented common maintenance operation + good code in pfSense that support painful maintenance become investments in a future company market grow and revenue

Agree?

stephenw10

@sergei_shablovsky said in Replace 2 port with 4 port nic card:

To bring up the console menu run: /etc/rc.initial

BSD told me that I have no sufficient permission...

Hmm, not a root shell then.

Were you able to disable Unbound in the config? Is it stuck somewhere else now?

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

You could try editing the config at the prompt to disable Unbound and then rebooting.

I manually disable Unbound and have access from desired (office_lan) port to internet.
BUT in webGUI all installed packages disappear and no ability to check pfSense updates through GUI menu and from CLI by pkg update / pkg upgrade

Or you could try to rerun /etc/rc.

Doing this after disabling Unbound and rebooting. Double twice checking all connections and port naming, assigning, etc.

What NIC did you replace and what did you put in instead?

2-head NIC
on
2-head NIC

The same manufacturer
The same place on motherboard
The same speed 1G

only MACs are different, of coarse

stephenw10

The fact it's failing to boot could just be coincidence. When was it last rebooted before this?
Replacing a NIC with the same type should make no difference to pfSense. And indeed it is booting past the interfaces check.
Did you try enabling Unbound again once it booted? Did it show any errors?

Steve

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

The fact it's failing to boot could just be coincidence. When was it last rebooted before this?
Replacing a NIC with the same type should make no difference to pfSense. And indeed it is booting past the interfaces check.

I also disappointed, so wrote question to this forum. :)

Did you try enabling Unbound again once it booted? Did it show any errors?

No any errors in log.

Anyway, after a lot of trying, I

backup all that have in a backup list;
clean system from dust (anyway that may be done by schedule) and clean all contacts of all components;
completely retest hardware (by own test soft, from embedded tests to special pack)
NO ANY ISSUES

So, than try to install pfSense on bare metal server. And what I found: 2.6.0-STABLE unable to install, stuck in the moment of interfere with NICs, but 2.7.x - installed without any problems

Also at this point I see that 2.7.x-DEVELOPMENT

recognize details in NICs (but anyway igb(4) driver used);
give a little bit less loading on system;

Thank You, Steve and all others here for help.

stephenw10

@sergei_shablovsky said in Replace 2 port with 4 port nic card:

2.6.0-STABLE unable to install, stuck in the moment of interfere with NICs

Actually in the installer or first boot after install?

The installer doesn't doesn't use the NICs so if it;s failing to boot the installer it must be some pretty low level incompatibility. Somehow that doesn't happen in 2.7 so maybe a PCI bridge the needs a newer driver? Just a guess.

Sergei_Shablovsky

@stephenw10 said in Replace 2 port with 4 port nic card:

@sergei_shablovsky said in Replace 2 port with 4 port nic card:

2.6.0-STABLE unable to install, stuck in the moment of interfere with NICs

Actually in the installer or first boot after install?

Right after booting from pfSense memstick CE. So even not possible to doing installing.

The installer doesn't doesn't use the NICs so if it;s failing to boot the installer it must be some pretty low level incompatibility. Somehow that doesn't happen in 2.7 so maybe a PCI bridge the needs a newer driver? Just a guess.

I come to the same conclusion as a You. But anyway that are strange because the first NIC I replace not impact on pfSense so dramatically: all become not so good after I replace the next NIC...

So spending hours to “playing” with different settings in a BIOS that may be related / impact on PCI at all. But as You may read, unsuccessfully at all.
So, after my patience gone out, I just create fresh memstick with pfSense 2.7.0-DEV and install it.

Anyway, time to shift to 2.7.X...

Sergei_Shablovsky

@stephenw10
Some new problem with upgrade come, please read Unable to update and package install (ERROR: It was not possible to identify which pfSense kernel is installed) in “Installing...” section ;) Thank You!