2.6.0 clean install & config restore fails <SOLVED>

manicmoose

Attempted a clean install off USB stick utilising ZFS and auto-config reload (from 2.5.2) on a Qotom 4-port device.

First time, the package re-installation mostly worked but the following services refused to start:

sshd
avahi-daemon (seems it's been renamed to 'avahi', so I had to install that manually later and then it started)
apcupsd - it tries to start but fails - have not fully investigated yet, but the error on the dashboard is:

"Network Information Server (NIS) not running, in order to run apcaccess on localhost, you need to enable it on APCupsd General settings."

Having sshd down made it difficult for me to analyse/troubleshoot so, I tried it a second time (clean ZFS install/restore config).

The 2nd time:

sshd failed to start again
avahi-daemon (see above)
not all packages were re-installed automatically (seemed to get hung up on pfblockerNG-devel)

Most everything else (network interfaces, multiple gateways, VLANs, FW rules, Aliases, etc) seemed to restore correctly, but with sshd down and no error messages in the log this makes it tricky to work on.

All I get is this in the Notices section:
pfSense has started creating missing SSH keys. SSH Startup will be delayed. Please note that reloading the filter rules and changes will be delayed until this operation is completed.

At this stage with SSHD so broken I'm struggling for ideas so I'm hoping someone else has one/some.

Otherwise it looks like I'll have to try reloading 2.5.2....

EDIT

Tried a package re-install for apcupsd and it seems to be happier now...
RRD charts are all empty - they did not restore - the data is in the xml so I might try to restore that.
Traffic Totals datas are all zero'd - no idea if the xml backup even contains the data for them.

Tried full reboot - still no SSHD
Disabled/re-enabled SSHD from the console menu....still nothing.

SOLUTION:
Found someone else's gem here
...turns out the '/etc/ssh/ssh_host_rsa_key' (public & private) files didn't get created at install time. Strange this happened twice with brand new fresh installs..
Anyway, a quick:

cd /etc/ssh
ssh-keygen -N '' -t rsa -f ssh_host_rsa_key
/usr/sbin/sshd

and we're back in business. Even survived a reboot.

Cheers.

manicmoose

@jimp
Not sure why you moved this out of the installation/upgrade section?
It's 100% produced by a fresh installation.
Others might have the same and miss it.

jimp

Read the stickies in that category. It is not for problems found post-installation. Only problems with the installation and upgrade process itself.

I also can't reproduce this. Fresh installs or config restores, I always get SSH keys when the SSH service is enabled.

manicmoose

@jimp Fwoaaar....splitting hairs a bit here aren't we?
I've done many in-place upgrades without such problems, so I would say that SSH keys not being generated during a clean install (twice!) is an installation topic. But, whatever.

jimp

Keys aren't generated during the installation process (ever). They're either made when you first enable the SSH daemon on a fresh config or when you restore a configuration with SSH enabled.

Whatever is happening appears to only be happening to you, and It's 100% not related to the installation process either way.

It may seem like nitpicking but that category is busy enough without extra unrelated topics being there.

e-1-1

@jimp not just to @manicmoose , it happened to me today when reinstalling a 2.6 box in order to get the new ZFS layout.

Steps to reproduce:

a 2.6 box (VM on ESXi) with older ZFS layout, upgraded since 2.4 series
install 2.6 over it, choose to recover the old config
(note there are no keys in /etc/ssh after the installer finishes)
after first and subsequent boots, the sshd keys are not regenerating, and clicking "Start" on the SSH service yields nothing. Only starting from CLI reveals the issue (missing keys).

Regen via CLI (almost instant)

 cd /etc/ssh
 ssh-keygen -N '' -t rsa -f ssh_host_rsa_key
 ssh-keygen -N '' -t ed25519 -f ssh_host_ed25519_key

adapted from here fixes it.