Time drift/system clock too fast on a PFSense VM



  • My pfSense VM's clock is running too fast for NTP to handle +11.5% or about +7 seconds every minute.

    I can fix this temporarily, by switching my kern.timecounter.hardware to ACPI-Fast:

    sysctl kern.timecounter.hardware="ACPI-fast"

    but this does not persist after a reboot.

    So, how can I make this setting stick between reboots and upgrades?

    What I have tried that has not worked:

    Adding the following to /etc/sysctl.conf
    kern.timecounter.hardware=ACPI-fast
    kern.timecounter.tc.TSC-low.quality=100

    Adding the following to /boot/loader.conf.local
    kern.timecounter.hardware="ACPI-fast"

    What I am trying next in my /boot/loader.conf.local
    kern.timecounter.tc.TSC-low.quality="100"

    Output that may help:

    sysctl kern.timecounter

    kern.timecounter.tsc_shift: 1
    kern.timecounter.smp_tsc_adjust: 0
    kern.timecounter.smp_tsc: 0
    kern.timecounter.invariant_tsc: 1
    kern.timecounter.fast_gettime: 1
    kern.timecounter.tick: 1
    kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) dummy(-1000000)
    kern.timecounter.hardware: ACPI-fast
    kern.timecounter.alloweddeviation: 5
    kern.timecounter.stepwarnings: 0
    kern.timecounter.tc.TSC-low.quality: 1000
    kern.timecounter.tc.TSC-low.frequency: 1247112000
    kern.timecounter.tc.TSC-low.counter: 326644757
    kern.timecounter.tc.TSC-low.mask: 4294967295
    kern.timecounter.tc.ACPI-fast.quality: 900
    kern.timecounter.tc.ACPI-fast.frequency: 3579545
    kern.timecounter.tc.ACPI-fast.counter: 4671981
    kern.timecounter.tc.ACPI-fast.mask: 16777215
    kern.timecounter.tc.i8254.quality: 0
    kern.timecounter.tc.i8254.frequency: 1193182
    kern.timecounter.tc.i8254.counter: 15941
    kern.timecounter.tc.i8254.mask: 65535

    sysctl kern.hz

    kern.hz: 1000

    edit:
    I have looked at a similar post form 2012, but they did not find a way to resolve this:
    https://forum.pfsense.org/index.php?topic=50994


  • Rebel Alliance Global Moderator

    What hardware is your host, what hypervisor?

    I have been running pfsense on esxi for long time, and have never seen any issues with the clock drifting..  Mine is set to HPET..

    [2.3.2-RELEASE][root@pfsense.local.lan]/: sysctl kern.timecounter                     
    kern.timecounter.tsc_shift: 1                                                         
    kern.timecounter.smp_tsc_adjust: 0                                                   
    kern.timecounter.smp_tsc: 0                                                           
    kern.timecounter.invariant_tsc: 1                                                     
    kern.timecounter.fast_gettime: 1                                                     
    kern.timecounter.tick: 1                                                             
    kern.timecounter.choice: TSC(-100) ACPI-fast(900) i8254(0) HPET(950) dummy(-1000000) 
    kern.timecounter.hardware: HPET                                                       
    kern.timecounter.alloweddeviation: 5                                                 
    kern.timecounter.stepwarnings: 0                                                     
    kern.timecounter.tc.TSC.quality: -100                                                 
    kern.timecounter.tc.TSC.frequency: 1497506000                                         
    kern.timecounter.tc.TSC.counter: 2212931071                                           
    kern.timecounter.tc.TSC.mask: 4294967295                                             
    kern.timecounter.tc.ACPI-fast.quality: 900                                           
    kern.timecounter.tc.ACPI-fast.frequency: 3579545                                     
    kern.timecounter.tc.ACPI-fast.counter: 5082544                                       
    kern.timecounter.tc.ACPI-fast.mask: 16777215                                         
    kern.timecounter.tc.i8254.quality: 0                                                 
    kern.timecounter.tc.i8254.frequency: 1193182                                         
    kern.timecounter.tc.i8254.counter: 1662                                               
    kern.timecounter.tc.i8254.mask: 65535                                                 
    kern.timecounter.tc.HPET.quality: 950                                                 
    kern.timecounter.tc.HPET.frequency: 14318180                                         
    kern.timecounter.tc.HPET.counter: 4259615338                                         
    kern.timecounter.tc.HPET.mask: 4294967295                                             
    [2.3.2-RELEASE][root@pfsense.local.lan]/:

    [2.3.2-RELEASE][root@pfsense.local.lan]/: sysctl kern.hz
    kern.hz: 100
    [2.3.2-RELEASE][root@pfsense.local.lan]/:



  • VMware ESXi, but it's in a hosted so not much I can change on the back-end.

    The issue is that I am not able to set sysctl values on the PFsense box… I can change them using a similar method for other BSD systems, but not this one.. so not sure what I need to change for the change to be persistent and survive upgrades.

    I can set the variable once it is running, but not at boot  :-(



  • I also run under ESXi and my timecounter.hardware is also HPET like John's.  I wonder who/why it get set differently on your end.



  • No clue, it should detect the best timer, but it is not.
    In this case it's best to set the quality manually, but not sure how to do this with pfSense…



  • Try add the following to /boot/loader.conf.local
    kern.timecounter.hardware=ACPI-fast

    If it does not work then you can use an own shell script.

    Something like

    
    #!/bin/sh
    
    sysctl kern.timecounter.hardware="ACPI-fast"
    
    

    Just put it into /usr/local/etc/rc.d change rights to 0755 and reboot



  • Damn, I tried the first one already.
    On my FreedBSD 10.3 system editing the /etc/sysctl.conf worked to change the timecounter.

    Not sure what needs to be set to make this change permanent…
    I added the script in /usr/local/etc/rc.d/fixtime, as w0w suggested, the permissions are set correctly and I can execute the script by running it, which works, but it did not get launched on reboot.

    so.. I'm still stuck.

    There must be something special pfsense is doing that freeBSD is not doing. I do see other threads in FreeBSD about this issue, and they are resolved with editing the /etc/sysctl.conf kern.timecounter.hardware... it's even in the manual
    https://www.freebsd.org/doc/faq/book.html

    Is there a pfSense manual for sysctl changes to the timecounter?


  • Banned

    No idea what are you doing with /boot/loader.conf.local, works just fine here. Use the Shellcmd package if it doesn't work for you.



  • [solved] I got it, it is done in the GUI:
    System -> Advanced
    System Tunables
    New
    Tunable kern.timecounter.hardware
    Value ACPI-fast
    Description

    So, basically most stuff that you would put in /etc/sysctl.conf or /boot/loader.conf.local goes in the GUI…

    Here is a link related to the issue:
    https://doc.pfsense.org/index.php/PfSense_on_Watchguard_Firebox#Further_Enhancements_3

    Enabling Speedstep: To get it up and running you need to do a few things:

    Set the timecounter to use the i8254 device with

    sysctl kern.timecounter.hardware=i8254

    To make this setting permanent add it to the system tunables table in the webgui:System: Advanced: System Tunables:

    Enable powerd in the webgui in System: Advanced: Miscellaneous:

    To force it to use EST rather than throttling or p4tcc add the following lines to loader.conf.local

    hint.p4tcc.0.disabled=1
    hint.acpi_throttle.0.disabled=1



  • I've tried in in two separate esxi 6 environments, built a fresh pfsense VM in each, and neither one lets you change this setting via /boot/loader.conf.local. Are you saying that they change to the file then rebooting worked for you to force a specific timecounter, or are you saying that the time did not drift?

    pfsense versions 2.3.2 and 2.3.2_p1



  • I think the script should be not just /usr/local/etc/rc.d/fixtime but
    /usr/local/etc/rc.d/fixtime.sh
    I have two scripts that are working fine on startup.
    May be its needed full path to sysctl to be added.
    /sbin/sysctl kern.timecounter.hardware="ACPI-fast"

    Anyway I glad you have solved it, but it looks a little bit strange that nothing works as it should just for you. I think the devil is in the details.