[solved] Question regarding MIM: Controller not having a static *public* IP possible?

[Bob.Dig]

Maybe some last thought about this in this thread. I now had the chance to use a third Plus-Installation, it is on a VPS (2) too.
It worked right away to register it to the controller at home, while the other VPS (1) still didn't worked.

I then removed all peers and toggled the controller on all instances and registered all peers to the controller on VPS 2. This worked out of the box. There is not much difference on those two VPS other than that VPS 1 is very slow and that VPN connections towards the pfSense at home differs.

One problem I noticed with that new setup (on VPS 2) is that VPS 1 is still trying to connect at every second towards Home over one WireGuard tunnel. And I can't stop it other than blocking it. I will open another thread about this.

[Bob.Dig]

OT but didn't deserve its own thread I think: On 24.11-RC I noticed that on an installation (with 1 Gig RAM) the pfnet-controller Service wasn't running. I did a reboot, saw the same thing again. Starting it manually works. That controller is the actual controller of two other instances.
Only thing I noticed:

Spoiler

No problem with my main-machine though. The service is running but it is not the controller of other instances.

[marcosm]

Are the logs any different if you increase the log level?

[Bob.Dig]

@marcosm Not really.

Nov 14 17:07:10 	kernel 		controltun0: link state changed to DOWN 

Nov 14 17:07:09 	pfnet-controller 	57413 	controltun0 packet from 185.*.*.*:*
Nov 14 17:07:09 	pfnet-controller 	57413 	Unknown peerId in received packet from peer 4ae627
Nov 14 17:07:09 	pfnet-controller 	57413 	NG System busy, will challenge peer
Nov 14 17:07:09 	pfnet-controller 	57413 	NG Handshake received from 185.*.*.*:*
Nov 14 17:07:09 	pfnet-controller 	57413 	controltun0 packet from 185.*.*.*:*
Nov 14 17:07:09 	pfnet-controller 	57413 	Unknown peerId in received packet from peer 4ae627 

Nothing after that for the MIM log.

That IP is my very slow instance...

[marcosm]

Would you please reproduce the issue, generate diagnostic data while it's still stopped, and upload it here?

Also, are there any interesting logs from the client(s) when that happens?

[Bob.Dig]

@marcosm said in [solved] Question regarding MIM: Controller not having a static *public* IP possible?:

Would you please reproduce the issue

Done. This time even more services hadn't started... It is a VPS in the oracle cloud with just 1 GB RAM.

Edit2: Now it stopped again, I can't even get it working... While I see it now is supporting DDNS, I haven't used that and it is way less stable. Maybe I am supposed to make a new config(db) for it? No, latter doesn't change anything.

And on my main machine at home I have a WireGuard problem it seems, service isn't running although most of the tunnels do... And I see this, not the first time.

Spoiler

So I guess I am looking forward to an all new Install-Image to start over... At least CrystalDiskInfo isn't showing any problems.

Edit: A Reboot fixed this for my main machine, still, I have never seen this before on it. Also did a RAM-check, no problems found.

[stephenw10]

You should be able to install the RC directly with Net Installer if that's an option for you.

[Bob.Dig]

@stephenw10 said in [solved] Question regarding MIM: Controller not having a static *public* IP possible?:

You should be able to install the RC directly with Net Installer if that's an option for you.

I will do this with the final release. And reboot fixed all problems with my home installation for now. It might just be a problem with FreeBSD on ZFS on Hyper-V on NTFS.

[marcosm]

The failing SCSI commands are telling. I've experienced similar issues before with VMs running on NFS shares. IMO the issue there is storage. I wouldn't trust it even with reinstalls.

[Bob.Dig]

@marcosm Something seems odd to me with me the RC. I did an "offline" disk check in Windows, no problems found. After rebooting the host, the pfSense VM didn't boot fully, or to be more precise, not all services where loaded and a message said, that boot verification hasn't completed. And because it didn't, I disabled MIM and rebooted the Host again. This time everything went well and no problems at all.
So this behavior I only have seen with the RC, now at home and before in the oracle cloud... I will disable MIM on all machines and have a look if these problems are related to it...

@marcosm said in [solved] Question regarding MIM: Controller not having a static *public* IP possible?:

I wouldn't trust it even with reinstalls.

I will let you know if you where right.

[marcosm]

If you're referring to the message:

Automatic boot verification is still running - wait a moment for boot to complete.

You'll need to wait for the boot process to finish. The GUI becomes available before all services are ready.

[Bob.Dig]

@marcosm said in [solved] Question regarding MIM: Controller not having a static *public* IP possible?:

You'll need to wait for the boot process to finish. The GUI becomes available before all services are ready.

Yeah, I did but it didn't.

[stephenw10]

Does it show 'bootup complete' at the console? If not how far does it get?

[Bob.Dig]

@stephenw10 It never finished so I went in, disabled MIM and rebooted (the whole host). Now, absolutely no problem. And this is my main machine, it is fast compared to those cheap VPS I got. I hope it is not my drive, and I have seen similar in OCI. So I hope it is MIM to be honest, I would like to see the final release soon (don't need MIM).

[stephenw10]

Mmm, me too!
This is the first report of this we've seen though so it's an edge case at most. I don't think this will hold a release.

Do you have the logs from that failed boot still?

[Bob.Dig]

@stephenw10 No, I have the RAM-disk enabled and I don't see anything before.