pfSense 2.6 problem with zombie processes

[gofaizen]

@stephenw10
I would like to get all info from qemu-ga. Like from Linux VM.
FreeBSD qemu-ga has strange behavior - in default config (when -m is not present) I see method - isa-serial. But in VM config - virtio-serial. And when I set virtio-serial in qemu-ga config, qemu-ga can't get any request from oVirt.

qemu-ga -d -v -l /var/log/qemu-ga.log -m virtio-serial - does not work
qemu-ga -d -v -l /var/log/qemu-ga.log - works with some restrictions in info

[stephenw10]

Ok, the next thing I would test here is whether it works as expected in FreeBSD 12.3.
Is this an upstream regression or something we are doing in pfSense specifically.

Steve

[gofaizen]

@stephenw10
Today I will try FreeBSD 12 on ovirt.
There is qcow image on download.freebsd.org
I will try clear iso and qcow image.

[gofaizen]

@stephenw10
I tried qemu-ga in FreeBSD 12.3 - working as expected.

[stephenw10]

Hmm, so something different about our filesystem perhaps?

Permissions issue?

Same qemu-agent version?

There can't be much different there.

[gofaizen]

@stephenw10
On FreeBSD 12.3 - qemu-ga -V
QEMU Guest Agent 5.0.1

On pfSense - qemu-ga -V
QEMU Guest Agent 5.0.1

Command line on FreeBSD - qemu-ga -m virtio-serial -d -v -l /var/log/qemu-ga.log

Command line on pfSense - /usr/local/bin/qemu-ga -d -v -l /var/log/qemu-ga.log -b guest-get-fsinfo

[gofaizen]

@stephenw10
In FreeBSD I have run /usr/local/bin/qemu-ga -d -v -l /var/log/qemu-ga.log -b guest-get-fsinfo.
And everything works fine.

[gofaizen]

@stephenw10
Everything is similar in FreeBSD and pfSense

pfSense :

qemu_guest_agent_enable="YES"
qemu_guest_agent_flags="-d -v -l /var/log/qemu-ga.log -b "guest-get-fsinfo""

/usr/local/bin/qemu-ga -d -v -l /var/log/qemu-ga.log -b guest-get-fsinfo -D
[general]
daemon=true
method=isa-serial
path=/dev/vtcon/org.qemu.guest_agent.0
logfile=/var/log/qemu-ga.log
pidfile=/var/run/qemu-ga.pid
statedir=/var/run
verbose=true
retry-path=false
blacklist=guest-get-fsinfo

ls -l /dev/vtcon
total 0
lrwxr-xr-x 1 root wheel 10 Apr 25 15:24 com.redhat.spice.0 -> ../ttyV0.3
lrwxr-xr-x 1 root wheel 10 Apr 25 15:24 org.qemu.guest_agent.0 -> ../ttyV0.2
lrwxr-xr-x 1 root wheel 10 Apr 25 15:24 ovirt-guest-agent.0 -> ../ttyV0.1

ls -l /dev/ttyV*
crw------- 1 root wheel 0x34 Apr 25 15:24 /dev/ttyV0.1
crw------- 1 root wheel 0x35 Apr 28 14:34 /dev/ttyV0.2
crw------- 1 root wheel 0x36 Apr 25 15:24 /dev/ttyV0.3

FreeBSD :

qemu_guest_agent_enable="YES"
qemu_guest_agent_flags="-d -v -l /var/log/qemu-ga.log -b guest-get-fsinfo"

/usr/local/bin/qemu-ga -d -v -l /var/log/qemu-ga.log -b guest-get-fsinfo -D
[general]
daemon=true
method=isa-serial
path=/dev/vtcon/org.qemu.guest_agent.0
logfile=/var/log/qemu-ga.log
pidfile=/var/run/qemu-ga.pid
statedir=/var/run
verbose=true
retry-path=false
blacklist=guest-get-fsinfo

ls -l /dev/vtcon
total 0
lrwxr-xr-x 1 root wheel 10 Apr 28 14:07 com.redhat.spice.0 -> ../ttyV0.3
lrwxr-xr-x 1 root wheel 10 Apr 28 14:07 org.qemu.guest_agent.0 -> ../ttyV0.2
lrwxr-xr-x 1 root wheel 10 Apr 28 14:07 ovirt-guest-agent.0 -> ../ttyV0.1

ls -l /dev/ttyV*
crw------- 1 root wheel 0x33 Apr 28 14:07 /dev/ttyV0.1
crw------- 1 root wheel 0x34 Apr 28 14:30 /dev/ttyV0.2
crw------- 1 root wheel 0x35 Apr 28 14:07 /dev/ttyV0.3

[stephenw10]

Ok, lets get a bug report open to track this.

What steps are required to replicate this?

[gofaizen]

@stephenw10
I have installed pfSense 2.5 from ISO on virtual machine in oVirt 4.4.5, then I have upgraded it to 2.6 from WEB interface. Then I added qemu-guest agent using this instruction - https://forum.netgate.com/topic/162083/pfsense-vm-on-proxmox-qemu-agent-installation
And got the problem, described above.

[gofaizen]

@stephenw10
I want to try fresh installation pfSense 2.6 and test qemu-guest-agent.

[gofaizen]

@stephenw10
In fresh installation of pfSense 2.6 same result

[stephenw10]

Hmm, struggling to replicate this in Proxmox.
Using the linked instructions the guest-agent starts and gives me IP information only.

Using the additional flags -b guest-get-fsinfo -D it fails to start at all.

I will say this is unsupported/untested. It's unlikely to see any developer time.

Steve

[stephenw10]

It returns correctly when run manually though:

root@pve:~# qm agent 110 get-fsinfo
[
   {
      "disk" : [],
      "mountpoint" : "/var/run",
      "name" : "tmpfs",
      "total-bytes" : 4194304,
      "type" : "tmpfs",
      "used-bytes" : 118784
   },
   {
      "disk" : [],
      "mountpoint" : "/",
      "name" : "/dev/mirror/pfSenseMirrors1a",
      "total-bytes" : 7259652096,
      "type" : "ufs",
      "used-bytes" : 1825689600
   }
]

Do you not see that?

[stephenw10]

....but that does seem to create zombie processes each time each run. OK.

[stephenw10]

Ok, this actually looks like a known issue that is still open for some reason:
https://github.com/aborche/qemu-guest-agent/issues/17

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260023

Waiting to be fixed upstream.

Steve

[gofaizen]

@stephenw10
Option -D for qemu-ga is to produce config only.
With this option qemu-ga shows config and quits.

[gofaizen]

@stephenw10
I don't understand - why qemu-ga in FreeBSD shows me everything in oVirt engine interface and pfSense shows only some information ?
They have identical config.

[stephenw10]

Mmm, the same port version though? 5.0.1?

From what I can see that fix should be in 5.0.1_1 and that hasn't arrived in FreeBSD yet.

How are you querying the agent?