Console reload stalls on last interface



  • 2.0-RC3 (amd64)
    built on Wed Aug 10 18:06:58 EDT 2011
    nanobsd (2g)

    15 vlans
    16 interfaces
    arping, nmap and cron packages installed.

    After a fresh reboot the console menu loads quickly, right to the prompt within about 2 seconds. Then, after a couple days uptime, I notice that the console load/reload stalls with the cursor blinking at the end of the line of the last interface, in the position to the right of the IP address. Currently it flashes there about 5 times, then the numbered menu appears all in a quick flash. This pause of ~5 seconds has been as long as 25 seconds in the past few weeks, and it appears to increase approximatley 1-1.5 second per day of uptime.

    Any idea what is causing this creeping delay between interfaces and options on the console load?


  • Rebel Alliance Developer Netgate

    The menu is drawn in a shell script (really fast), the banner at the top with the IPs is a PHP script. The part that prints the IPs just interates over the list of interfaces you have configured. There isn't anything in there I see that would have any relation to uptime or anything dynamic.

    If it's stopping at the end of that, something may be slowing PHP down in general on the system. Have you looked at the CPU usage graphs or Status > System Activity to see if you have something consuming a lot of CPU?



  • It doesn't look loaded to me. I've attached the RRD graph in case I'm reading it wrong.

    I've also noticed that when loading the Dashboard, the traffic graph applet loads the graphs for all my interfaces within a few seconds, but that last interface takes several times longer before it begins graphing or showing a throughput number. Maybe related?



  • Rebel Alliance Developer Netgate

    It could be - is your last interface something unusual? What does "ifconfig -a" look like? (feel free to x out the IPs)



  • em1_vlan240 is the interface that stalls in the console and the Traffic Graph applet. It's a public non-Natted subnet, but then so is em1_vlan172 and it doesn't exhibit the same stalls.

    ifconfig.txt


  • Rebel Alliance Developer Netgate

    yeah nothing there looks out of sorts. Unless there is something quirky with that card causing a delay when querying info about that one vlan… not sure how/why though.

    Nothing in the system log that looks odd?



  • Nothing out of the ordinary in the system log.

    I ran top and logged into ssh in a separate terminal and saw the sockstat process consume 25-30% CPU exactly coinciding with the delay on the console load. This is on an Atom D510 with hyperthreading disabled.


  • Rebel Alliance Developer Netgate

    So then it may be this check going slow for some bizarre reason:

    SSHD=`/usr/bin/sockstat -4l | grep "*.22" | wc -l`
    

    I suppose if there were thousands of active sockets on the box it might slow down, but that would have to be on the system itself, not connections passing through.



  • I reordered my interfaces in config.xml and it's still the last interface (now em0_vlan208) that's seeing the pause, so it apparently has nothing to do with any specific interface.


Locked