userland calling deprecated sysctl, please rebuild world

itfsupport

[2.5.2-RELEASE][root@vrouter-dc.it.local]/var/log: cat dmesg.boot
Copyright (c) 1992-2020 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.2-STABLE fd0f54f44b5c(RELENG_2_5_0) pfSense amd64
FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2)
VT(vga): resolution 640x480
CPU: Intel(R) Xeon(R) CPU           E5649  @ 2.53GHz (2533.48-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping=2
  Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x83ba2223<SSE3,PCLMULQDQ,VMX,SSSE3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,HV>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  Structured Extended Features=0x2<TSCADJ>
  Structured Extended Features2=0x4<UMIP>
  Structured Extended Features3=0xac000000<IBPB,STIBP,ARCH_CAP,SSBD>
  IA32_ARCH_CAPS=0x48<SKIP_L1DFL_VME>
  AMD Extended Feature Extensions ID EBX=0x1009000
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
Hypervisor: Origin = "KVMKVMKVM"
real memory  = 2147483648 (2048 MB)
avail memory = 2019041280 (1925 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <BOCHS  BXPCAPIC>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
random: unblocking device.
ioapic0 <Version 1.1> irqs 0-23 on motherboard
Launching APs: 1
wlan: mac acl policy registered
random: entropy device external interface
ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff8073dd40, 0) error 1
ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff8073ddf0, 0) error 1
ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff8073dea0, 0) error 1
iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80765730, 0) error 1
iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff807657e0, 0) error 1
iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80765890, 0) error 1
000.000018 [4336] netmap_init               netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff8140c3e0, 0) error 19
kbd1 at kbdmux0
[ath_hal] loaded
mlx5en: Mellanox Ethernet driver 3.5.2 (September 2019)
nexus0
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
padlock0: No ACE support.
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS> on motherboard
acpi0: <BOCHS BXPCRSDT> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71,0x72-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 100000000 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> mem 0xfb000000-0xfbffffff,0xfea14000-0xfea14fff at device 1.0 on pci0
vgapci0: Boot video device
uhci0: <Intel 82801I (ICH9) USB controller> port 0x5040-0x505f irq 16 at device 26.0 on pci0
usbus0 on uhci0
usbus0: 12Mbps Full Speed USB v1.0
uhci1: <Intel 82801I (ICH9) USB controller> port 0x5060-0x507f irq 17 at device 26.1 on pci0
usbus1 on uhci1
usbus1: 12Mbps Full Speed USB v1.0
uhci2: <Intel 82801I (ICH9) USB controller> port 0x5080-0x509f irq 18 at device 26.2 on pci0
usbus2 on uhci2
usbus2: 12Mbps Full Speed USB v1.0
ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xfea15000-0xfea15fff irq 19 at device 26.7 on pci0
usbus3: EHCI version 1.0
usbus3 on ehci0
usbus3: 480Mbps High Speed USB v2.0
hdac0: <Intel 82801I HDA Controller> mem 0xfea10000-0xfea13fff irq 16 at device 27.0 on pci0
pcib1: <ACPI PCI-PCI bridge> mem 0xfea16000-0xfea16fff irq 16 at device 28.0 on pci0
pcib1: [GIANT-LOCKED]
pcib2: <PCI-PCI bridge> mem 0xfea17000-0xfea17fff irq 16 at device 28.1 on pci0
pcib2: [GIANT-LOCKED]
pcib3: <PCI-PCI bridge> mem 0xfea18000-0xfea18fff irq 16 at device 28.2 on pci0
pcib3: [GIANT-LOCKED]
pcib4: <PCI-PCI bridge> mem 0xfea19000-0xfea19fff irq 16 at device 28.3 on pci0
pcib4: [GIANT-LOCKED]
uhci3: <Intel 82801I (ICH9) USB controller> port 0x50a0-0x50bf irq 16 at device 29.0 on pci0
usbus4 on uhci3
uhci4: <Intel 82801I (ICH9) USB controller> port 0x50c0-0x50df irq 17 at device 29.1 on pci0
usbus5 on uhci4
uhci5: <Intel 82801I (ICH9) USB controller> port 0x50e0-0x50ff irq 18 at device 29.2 on pci0
usbus6 on uhci5
ehci1: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xfea1a000-0xfea1afff irq 19 at device 29.7 on pci0
usbus7: EHCI version 1.0
usbus7 on ehci1
pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib5
pcib6: <PCI-PCI bridge> mem 0xfe000000-0xfe0000ff irq 21 at device 1.0 on pci1
pci2: <PCI bus> on pcib6
virtio_pci0: <VirtIO PCI SCSI adapter> port 0x4000-0x403f mem 0xfde80000-0xfde80fff,0xfc600000-0xfc603fff irq 22 at device 5.0 on pci2
vtscsi0: <VirtIO SCSI Adapter> on virtio_pci0
virtio_pci1: <VirtIO PCI Network adapter> port 0x4040-0x405f mem 0xfde81000-0xfde81fff,0xfc604000-0xfc607fff irq 23 at device 18.0 on pci2
vtnet0: <VirtIO Networking Adapter> on virtio_pci1
vtnet0: Ethernet address: 76:b0:b6:34:46:ca
vtnet0: netmap queues/slots: TX 1/256, RX 1/128
000.000104 [ 445] vtnet_netmap_attach       vtnet attached txq=1, txd=256 rxq=1, rxd=128
virtio_pci2: <VirtIO PCI Network adapter> port 0x4060-0x407f mem 0xfde82000-0xfde82fff,0xfc608000-0xfc60bfff irq 20 at device 19.0 on pci2
vtnet1: <VirtIO Networking Adapter> on virtio_pci2
vtnet1: Ethernet address: 9a:0d:8e:1f:f7:ef
vtnet1: netmap queues/slots: TX 1/256, RX 1/128
000.000105 [ 445] vtnet_netmap_attach       vtnet attached txq=1, txd=256 rxq=1, rxd=128
pcib7: <PCI-PCI bridge> mem 0xfe001000-0xfe0010ff irq 22 at device 2.0 on pci1
pci3: <PCI bus> on pcib7
pcib8: <PCI-PCI bridge> mem 0xfe002000-0xfe0020ff irq 23 at device 3.0 on pci1
pci4: <PCI bus> on pcib8
pcib9: <PCI-PCI bridge> mem 0xfe003000-0xfe0030ff irq 20 at device 4.0 on pci1
pci5: <PCI bus> on pcib9
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel ICH9 AHCI SATA controller> port 0x5100-0x511f mem 0xfea1b000-0xfea1bfff irq 16 at device 31.2 on pci0
ahci0: AHCI v1.00 with 6 1.5Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
acpi_syscontainer0: <System Container> on acpi0
acpi_syscontainer1: <System Container> port 0xcd8-0xce3 on acpi0
acpi_syscontainer2: <System Container> port 0x620-0x62f on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4
orm0: <ISA Option ROM> at iomem 0xea800-0xeffff pnpid ORM0000 on isa0
attimer0: <AT timer> at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounters tick every 10.000 msec
usbus5: 12Mbps Full Speed USB v1.0
usbus4: 12Mbps Full Speed USB v1.0
ugen1.1: <Intel UHCI root HUB> at usbus1
usbus6: 12Mbps Full Speed USB v1.0
usbus7: 480Mbps High Speed USB v2.0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel UHCI root HUB> at usbus2
ugen5.1: <Intel UHCI root HUB> at usbus5
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
ugen0.1: <Intel UHCI root HUB> at usbus0
ugen3.1: <Intel EHCI root HUB> at usbus3
uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen6.1: <Intel UHCI root HUB> at usbus6
ugen7.1: <Intel EHCI root HUB> at usbus7
uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
uhub6: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus7
ugen4.1: <Intel UHCI root HUB> at usbus4
uhub7: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
Trying to mount root from ufs:/dev/gpt/root [rw,noatime]...
Root mount waiting for: usbus0 usbus1 usbus2 usbus3 usbus4 usbus5 usbus6 usbus7 CAM
uhub3: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub7: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
Root mount waiting for: usbus3 usbus7 CAM
Root mount waiting for: usbus3 usbus7 CAM
uhub4: 6 ports with 6 removable, self powered
uhub6: 6 ports with 6 removable, self powered
Root mount waiting for: usbus7 CAM
ugen7.2: <QEMU QEMU USB Tablet> at usbus7
uhid0 on uhub6
uhid0: <QEMU QEMU USB Tablet, class 0/0, rev 2.00/0.00, addr 2> on usbus7
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
da0 at vtscsi0 bus 0 scbus0 target 0 lun 0
da0: <QEMU QEMU HARDDISK 2.5+> Fixed Direct Access SPC-3 SCSI device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 16384MB (33554432 512 byte sectors)
cd0 at ahcich1 bus 0 scbus2 target 0 lun 0
cd0: <QEMU QEMU DVD-ROM 2.5+> Removable CD-ROM SCSI device
cd0: Serial Number QM00003
cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes)
cd0: 0MB (1 0 byte sectors)
mountroot: waiting for device /dev/gpt/root...
CPU: Intel(R) Xeon(R) CPU           E5649  @ 2.53GHz (2533.48-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping=2
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x83ba2223<SSE3,PCLMULQDQ,VMX,SSSE3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,HV>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  Structured Extended Features=0x2<TSCADJ>
  Structured Extended Features2=0x4<UMIP>
  Structured Extended Features3=0xac000000<IBPB,STIBP,ARCH_CAP,SSBD>
  IA32_ARCH_CAPS=0x48<SKIP_L1DFL_VME>
  AMD Extended Feature Extensions ID EBX=0x1009000
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
Hypervisor: Origin = "KVMKVMKVM"

stephenw10

There's no error shown in that log.

You are loading additional modules if you see htcp available. What else are you loading?

Do you have a /boot/loader.conf.local? What's in it?

Steve

johnpoz

I'm on 21.05.1 and not seeing that available

[21.05.1-RELEASE][admin@sg4860.local.lan]/root: sysctl net.inet.tcp.cc.available
net.inet.tcp.cc.available: newreno
[21.05.1-RELEASE][admin@sg4860.local.lan]/root:

stephenw10

Yup because in order to see that as an option you have to load the module:

[2.5.2-RELEASE][admin@252dev.stevew.lan]/root: kldload cc_htcp
[2.5.2-RELEASE][admin@252dev.stevew.lan]/root: sysctl net.inet.tcp.cc.available
net.inet.tcp.cc.available: newreno, htcp

It's not loaded by default because it's pointless for the vast majority of users.

So OP is loading it via some custom loader variable which begs; what else is being loaded?

Whatever it is it's probably generating that error. Which we've yet to see in context.

Steve

itfsupport

@stephenw10 said in userland calling deprecated sysctl, please rebuild world:

There's no error shown in that log.

You are loading additional modules if you see htcp available. What else are you loading?

Do you have a /boot/loader.conf.local? What's in it?

Steve

[2.5.2-RELEASE][root@vrouter-dc.itf.local]/root: sudo cat /boot/loader.conf.local
aesni_load="YES"
aio_load="YES"
hw.vga.textmode=0
kern.maxproc=100000
kern.panic_reboot_wait_time=30
kern.racct.enable=1
# mem settings
kern.dfldsiz=1099511627776
kern.dflssiz=1099511627776
kern.ipc.maxmbufmem=2034446336
kern.ipc.msgmnb=65536
kern.ipc.msgmni=32000
kern.ipc.semmni=2048
kern.ipc.semmns=32768
kern.ipc.semmnu=32768
kern.ipc.semmsl=256
kern.ipc.semopm=500
kern.ipc.shmmni=2048
kern.ipc.shmseg=2048
kern.maxdsiz=1099511627776
kern.maxssiz=1099511627776
vfs.maxbcachebuf=131072
# net settings
cc_htcp_load="YES"
hw.em.rx_abs_int_delay=1000
hw.em.rx_int_delay=100
hw.em.rxd=4096
hw.em.tx_abs_int_delay=1000
hw.em.tx_int_delay=100
hw.em.txd=4096
net.graph.maxdata=65536
net.graph.maxalloc=65536
net.inet.tcp.hostcache.bucketlimit=100
net.inet.tcp.hostcache.cachelimit=65536
net.inet.tcp.hostcache.hashsize=4096
net.inet.tcp.syncache.bucketlimit=100
net.inet.tcp.syncache.cachelimit=65536
net.inet.tcp.syncache.hashsize=1024
net.inet.tcp.tcbhashsize=65536
net.isr.bindthreads=1
net.isr.defaultqlimit=4096
net.isr.maxthreads=-1
net.link.ifqmaxlen=2048
[2.5.2-RELEASE][root@vrouter-dc.itf.local]/root:

stephenw10

Ok.

Why are you loading those things?

One of those is causing your error.

Steve

itfsupport

@stephenw10
Good day!
What specific parameter from the boot/loader.conf.local confuses you?

itfsupport

@stephenw10 said in userland calling deprecated sysctl, please rebuild world:

Yup because in order to see that as an option you have to load the module:
[2.5.2-RELEASE][admin@252dev.stevew.lan]/root: kldload cc_htcp
[2.5.2-RELEASE][admin@252dev.stevew.lan]/root: sysctl net.inet.tcp.cc.available
net.inet.tcp.cc.available: newreno, htcp
It's not loaded by default because it's pointless for the vast majority of users.

So OP is loading it via some custom loader variable which begs; what else us bring loaded?

Whatever it is it's probably generating that error. Which we've yet to see in context.

Steve

This information on tuning FreeBSD hosts connected at speeds of 1Gbps or higher.
FreeBSD supports TCP autotuning and advanced congestion control algorithms in all currently-supported releases. You can check which are available in the running kernel using:

sysctl net.inet.tcp.cc.available
To make additional algorithms available in the kernel, the appropriate module must be loaded. The congestion control modules are in /boot/kernel and begin with cc_ - they can be listed with the following command:

ls /boot/kernel/cc_* | grep -v symbols
To load a particular congestion control algorithm (for example htcp) you can use:

kldload cc_htcp
and you will then be able to set the congestion control algorithm using this:

sysctl net.inet.tcp.cc.algorithm=htcp

Enabling congestion control algorithms at boot time

Depending how your kernel is built, you may not have the algorithm you want available by default at boot time, and will need to load the driver during the boot process. To to this add the following to /boot/loader.conf:

cc_htcp_load="YES"
The htcp algorithm will then be loaded at boot time, and will be enabled when sysctl.conf is processed at boot time and the net.inet.tcp.cc.algorithm=htcp variable is set.

stephenw10

Right but a firewall/router is not a host. Almost all traffic passes through it and those TCP tunables apply to traffic terminated on the system.

Why have you added them? Were you seeing a problem? Did it help?

One of those is causing the error you are seeing. So if you want to not see it you need to stop applying that tunable.

Steve

provels

"Userland calling deprecated sysctl, please rebuild world"

~ An anthem for Modern Man

itfsupport

@stephenw10
Hello!
Now i delete all settings from boot/loader.conf.local
and i still have this error

itfsupport

Now i reset boot.loader.conf.local and syscl.conf to default and i have error

userland calling deprecated sysctl, please rebuild world

stephenw10

You still have not shown us the log containing this error for context so we can only guess what is calling it. You must have something custom on that box.
To confirm you have removed the custom sysctls from Sys > Adv > System tunables?
sysctl.conf will just get rebuilt at boot.

Steve

itfsupport

@stephenw10
sysctl.conf recovered after reboot
only this settings
kern.ipc.shmall=2097152
kern.ipc.shmmax=2147483648
kern.ipc.semmsl=250

itfsupport

@stephenw10
I don't really understand how to show you an error
There is a journal entry

Sep 26 23:17:21 pfSenseIllanM syslogd: kernel boot file is /boot/kernel/kernel
Sep 26 23:17:22 pfSenseIllanM nginx: 2021/09/26 23:17:22 [error] 71259#100112: send() failed (54: Connection reset by peer)
Sep 26 23:17:45 pfSenseIllanM kernel: userland calling deprecated sysctl, please rebuild world
Sep 26 23:17:45 pfSenseIllanM kernel: userland calling deprecated sysctl, please rebuild world
Sep 26 23:18:48 pfSenseIllanM kernel: userland calling deprecated sysctl, please rebuild world
Sep 26 23:18:48 pfSenseIllanM kernel: userland calling deprecated sysctl, please rebuild world
Sep 26 23:19:50 pfSenseIllanM kernel: userland calling deprecated sysctl, please rebuild world
Sep 26 23:19:50 pfSenseIllanM kernel: userland calling deprecated sysctl, please rebuild world

stephenw10

Ok, so it occurs multiple times and not just at boot? Some minutes apart?

Does it log that continually?

That's not a sysctl that is applied at boot then. You have something else on that system actively setting something that's no longer valid. Do you have something else running there? Some other custom script or manually added package?

Steve

itfsupport

@stephenw10 said in userland calling deprecated sysctl, please rebuild world:

Ok, so it occurs multiple times and not just at boot? Some minutes apart?

Does it log that continually?

That's not a sysctl that is applied at boot then. You have something else on that system actively setting something that's no longer valid. Do you have something else running there? Some other custom script or manually added package?

Steve

Good day!
After a long search for the error found the following.
The problem appears when enable an additional parameter in the Zabbix agent settings

EnableRemoteCommands=1

stephenw10

Nice find.